2024-03-28T16:35:54.213 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-28T16:35:54.214 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381 --verbose --description rgw:singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} --owner scheduled_soumyakoduri@teuthology --name soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi -- /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/orig.config.yaml 2024-03-28T16:35:54.245 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T16:35:54.325 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381 branch: wip_thotz_update_attrs_from_state_setattrs description: rgw:singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} email: null first_in_suite: false job_id: '7628381' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip_thotz_update_attrs_from_state_setattrs ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: acb3247c84f48e9ce91359973bf22ebb44ec5034 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: acb3247c84f48e9ce91359973bf22ebb44ec5034 rgw: ec-data-pool: false frontend: beast thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip_thotz_update_attrs_from_state_setattrs sha1: acb3247c84f48e9ce91359973bf22ebb44ec5034 owner: scheduled_soumyakoduri@teuthology priority: 50 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 sha1: acb3247c84f48e9ce91359973bf22ebb44ec5034 sleep_before_teardown: 0 suite: rgw:singleton suite_branch: wip_thotz_update_attrs_from_state_setattrs suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_acb3247c84f48e9ce91359973bf22ebb44ec5034/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: acb3247c84f48e9ce91359973bf22ebb44ec5034 targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQ0qQqye1uKDXScKCoAVqSlWVI8KsfzUNX7YTdRQwYQ0cIEIhdtKHEz0DpNLvdjS7oZqx3yLvXv8aw5fbhdqxQ= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpRTtw3n0zBm75s+ZMdSyBbyD9vw++ZFvRFb9QaQy+3ImgeT9ikr+Xk+DubNhB4IL2pmsjpXnrF6tn4Bnu/bEc= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-28_16:27:01 tube: smithi user: soumyakoduri verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-28T16:35:54.325 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_acb3247c84f48e9ce91359973bf22ebb44ec5034/qa; will attempt to use it 2024-03-28T16:35:54.326 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_acb3247c84f48e9ce91359973bf22ebb44ec5034/qa/tasks 2024-03-28T16:35:54.326 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-28T16:35:54.327 INFO:teuthology.task.internal:Checking packages... 2024-03-28T16:35:54.351 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'acb3247c84f48e9ce91359973bf22ebb44ec5034' 2024-03-28T16:35:54.352 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-28T16:35:54.352 INFO:teuthology.packaging:ref: None 2024-03-28T16:35:54.352 INFO:teuthology.packaging:tag: None 2024-03-28T16:35:54.352 INFO:teuthology.packaging:branch: wip_thotz_update_attrs_from_state_setattrs 2024-03-28T16:35:54.352 INFO:teuthology.packaging:sha1: acb3247c84f48e9ce91359973bf22ebb44ec5034 2024-03-28T16:35:54.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip_thotz_update_attrs_from_state_setattrs 2024-03-28T16:35:54.500 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:35:54.502 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-28T16:35:54.508 INFO:teuthology.task.internal:no buildpackages task found 2024-03-28T16:35:54.508 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-28T16:35:54.518 INFO:teuthology.task.internal:Saving configuration 2024-03-28T16:35:54.529 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-28T16:35:54.536 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T16:35:54.558 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 16:30:06.931945', 'locked_by': 'scheduled_soumyakoduri@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINfNmkhB/Vm/pE7rBr8G3YRx4Q9WndT0SL24ZvPML9wT'} 2024-03-28T16:35:54.580 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 16:30:06.933040', 'locked_by': 'scheduled_soumyakoduri@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPjWHsR1a8tLuPdWT+IWoTKBkVmzPh+fD53ZfOsqK5T8'} 2024-03-28T16:35:54.581 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-28T16:35:54.587 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-03-28T16:35:54.587 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-03-28T16:35:54.587 INFO:teuthology.run_tasks:Running task console_log... 2024-03-28T16:35:54.683 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9ad3ab1280>, signals=[15]) 2024-03-28T16:35:54.684 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-28T16:35:54.691 INFO:teuthology.task.internal:Opening connections... 2024-03-28T16:35:54.691 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T16:35:54.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:35:54.776 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:35:54.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:35:54.852 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-28T16:35:54.861 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-03-28T16:35:54.866 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-03-28T16:35:54.867 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Ubuntu" 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="22.04" 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_CODENAME=jammy 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:ID=ubuntu 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE=debian 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-28T16:35:54.914 INFO:teuthology.orchestra.run.smithi119.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-28T16:35:54.915 INFO:teuthology.orchestra.run.smithi119.stdout:UBUNTU_CODENAME=jammy 2024-03-28T16:35:54.915 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-03-28T16:35:54.936 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-03-28T16:35:54.941 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-03-28T16:35:54.941 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-03-28T16:35:54.989 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-28T16:35:54.989 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu" 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="22.04" 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=jammy 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-28T16:35:54.990 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=jammy 2024-03-28T16:35:54.991 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-03-28T16:35:55.010 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-28T16:35:55.021 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-28T16:35:55.087 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-28T16:35:55.088 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-03-28T16:35:55.091 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-03-28T16:35:55.096 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-28T16:35:55.104 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-28T16:35:55.104 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-03-28T16:35:55.140 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-03-28T16:35:55.163 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-28T16:35:55.214 INFO:teuthology.run_tasks:Running task kernel... 2024-03-28T16:35:55.232 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-28T16:35:55.232 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-28T16:35:55.232 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T16:35:55.233 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T16:35:55.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:35:55.238 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-03-28T16:35:55.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:35:55.239 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-03-28T16:35:55.286 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-56-generic 2024-03-28T16:35:55.286 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.15.0-56-generic 2024-03-28T16:35:55.286 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-03-28T16:35:55.287 INFO:teuthology.orchestra.run.smithi119.stdout:5.15.0-56-generic 2024-03-28T16:35:55.287 INFO:teuthology.task.kernel:Running kernel on smithi119: 5.15.0-56-generic 2024-03-28T16:35:55.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get clean 2024-03-28T16:35:55.427 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-03-28T16:35:55.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-03-28T16:35:55.600 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-28T16:35:55.600 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-28T16:35:55.605 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-28T16:35:55.691 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-28T16:35:55.695 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-28T16:35:55.695 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-28T16:35:55.760 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-28T16:35:55.832 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,298 kB] 2024-03-28T16:35:55.905 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-03-28T16:35:55.942 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [435 kB] 2024-03-28T16:35:55.948 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [232 kB] 2024-03-28T16:35:55.951 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-28T16:35:55.951 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-28T16:35:55.952 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,605 kB] 2024-03-28T16:35:55.974 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [270 kB] 2024-03-28T16:35:55.977 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-28T16:35:55.977 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-28T16:35:55.980 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,518 kB] 2024-03-28T16:35:55.985 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-03-28T16:35:55.996 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-03-28T16:35:55.998 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-28T16:35:56.022 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-28T16:35:56.023 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-28T16:35:56.023 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-28T16:35:56.024 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-28T16:35:56.024 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-03-28T16:35:56.030 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-28T16:35:56.031 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-03-28T16:35:56.051 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-28T16:35:56.051 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-03-28T16:35:56.054 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-28T16:35:56.055 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-03-28T16:35:56.066 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-03-28T16:35:56.073 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-03-28T16:35:56.076 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-28T16:35:56.076 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-03-28T16:35:56.077 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-03-28T16:35:56.094 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-03-28T16:35:56.094 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-28T16:35:56.095 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-28T16:35:56.095 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-28T16:35:56.097 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-28T16:35:56.097 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-28T16:35:56.098 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-28T16:35:56.098 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-28T16:35:56.115 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-28T16:35:56.116 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-28T16:35:56.118 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,298 kB] 2024-03-28T16:35:56.122 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-28T16:35:56.259 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,518 kB] 2024-03-28T16:35:56.532 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [435 kB] 2024-03-28T16:35:56.550 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [232 kB] 2024-03-28T16:35:56.560 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-28T16:35:56.560 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-28T16:35:56.562 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,605 kB] 2024-03-28T16:35:56.563 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-03-28T16:35:56.601 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-03-28T16:35:56.618 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-28T16:35:56.618 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-03-28T16:35:56.653 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [270 kB] 2024-03-28T16:35:56.664 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-28T16:35:56.664 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-03-28T16:35:56.705 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-28T16:35:56.705 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-28T16:35:56.705 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-03-28T16:35:56.711 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-28T16:35:56.875 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-03-28T16:35:56.880 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-03-28T16:35:57.277 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-28T16:35:57.283 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-28T16:35:57.296 INFO:teuthology.orchestra.run.smithi119.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-28T16:35:57.298 INFO:teuthology.orchestra.run.smithi119.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-28T16:35:57.300 INFO:teuthology.orchestra.run.smithi119.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-28T16:35:57.510 INFO:teuthology.orchestra.run.smithi119.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-03-28T16:35:57.548 INFO:teuthology.orchestra.run.smithi119.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-03-28T16:35:57.561 INFO:teuthology.orchestra.run.smithi119.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-28T16:35:57.562 INFO:teuthology.orchestra.run.smithi119.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-03-28T16:35:57.563 INFO:teuthology.orchestra.run.smithi119.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-03-28T16:35:57.565 INFO:teuthology.orchestra.run.smithi119.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-03-28T16:35:57.566 INFO:teuthology.orchestra.run.smithi119.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-28T16:35:57.566 INFO:teuthology.orchestra.run.smithi119.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-28T16:35:57.592 INFO:teuthology.orchestra.run.smithi119.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-28T16:35:57.596 INFO:teuthology.orchestra.run.smithi119.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-28T16:35:57.761 INFO:teuthology.orchestra.run.smithi119.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-28T16:35:57.845 INFO:teuthology.orchestra.run.smithi119.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-28T16:35:57.952 INFO:teuthology.orchestra.run.smithi119.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-28T16:35:58.009 INFO:teuthology.orchestra.run.smithi119.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-28T16:35:58.043 INFO:teuthology.orchestra.run.smithi119.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-28T16:36:01.566 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 12.6 MB in 2s (7,133 kB/s) 2024-03-28T16:36:02.283 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 12.6 MB in 3s (4,932 kB/s) 2024-03-28T16:36:02.891 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-03-28T16:36:02.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-28T16:36:02.969 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-03-28T16:36:03.162 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-03-28T16:36:03.163 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-03-28T16:36:03.335 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-03-28T16:36:03.336 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-28T16:36:03.336 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-28T16:36:03.336 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-28T16:36:03.336 INFO:teuthology.orchestra.run.smithi170.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-28T16:36:03.336 INFO:teuthology.orchestra.run.smithi170.stdout: python2.7-minimal 2024-03-28T16:36:03.336 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-28T16:36:03.338 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-03-28T16:36:03.338 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-28T16:36:03.338 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-28T16:36:03.338 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-28T16:36:03.339 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-03-28T16:36:03.339 INFO:teuthology.orchestra.run.smithi170.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-28T16:36:03.339 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-03-28T16:36:03.340 INFO:teuthology.orchestra.run.smithi170.stdout: thermald 2024-03-28T16:36:03.392 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-03-28T16:36:03.392 INFO:teuthology.orchestra.run.smithi170.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-28T16:36:03.392 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-28T16:36:03.392 INFO:teuthology.orchestra.run.smithi170.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-28T16:36:03.393 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages will be upgraded: 2024-03-28T16:36:03.394 INFO:teuthology.orchestra.run.smithi170.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-28T16:36:03.574 INFO:teuthology.orchestra.run.smithi170.stdout:3 upgraded, 5 newly installed, 0 to remove and 311 not upgraded. 2024-03-28T16:36:03.574 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 113 MB of archives. 2024-03-28T16:36:03.574 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-28T16:36:03.574 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-28T16:36:03.619 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-03-28T16:36:03.636 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-28T16:36:03.729 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-03-28T16:36:03.902 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-03-28T16:36:03.903 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-03-28T16:36:04.053 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-03-28T16:36:04.053 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-28T16:36:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-28T16:36:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-28T16:36:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-28T16:36:04.054 INFO:teuthology.orchestra.run.smithi119.stdout: python2.7-minimal 2024-03-28T16:36:04.054 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-28T16:36:04.055 INFO:teuthology.orchestra.run.smithi119.stdout:The following additional packages will be installed: 2024-03-28T16:36:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-28T16:36:04.055 INFO:teuthology.orchestra.run.smithi119.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-28T16:36:04.056 INFO:teuthology.orchestra.run.smithi119.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-28T16:36:04.057 INFO:teuthology.orchestra.run.smithi119.stdout:Suggested packages: 2024-03-28T16:36:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-28T16:36:04.057 INFO:teuthology.orchestra.run.smithi119.stdout:Recommended packages: 2024-03-28T16:36:04.057 INFO:teuthology.orchestra.run.smithi119.stdout: thermald 2024-03-28T16:36:04.104 INFO:teuthology.orchestra.run.smithi119.stdout:The following NEW packages will be installed: 2024-03-28T16:36:04.104 INFO:teuthology.orchestra.run.smithi119.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-28T16:36:04.105 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-28T16:36:04.105 INFO:teuthology.orchestra.run.smithi119.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-28T16:36:04.106 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages will be upgraded: 2024-03-28T16:36:04.106 INFO:teuthology.orchestra.run.smithi119.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-28T16:36:04.286 INFO:teuthology.orchestra.run.smithi119.stdout:3 upgraded, 5 newly installed, 0 to remove and 311 not upgraded. 2024-03-28T16:36:04.286 INFO:teuthology.orchestra.run.smithi119.stdout:Need to get 113 MB of archives. 2024-03-28T16:36:04.286 INFO:teuthology.orchestra.run.smithi119.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-28T16:36:04.286 INFO:teuthology.orchestra.run.smithi119.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-28T16:36:04.956 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-28T16:36:05.306 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-28T16:36:05.713 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-28T16:36:06.065 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-28T16:36:07.522 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-28T16:36:07.523 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-28T16:36:07.523 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-28T16:36:08.329 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-28T16:36:08.330 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-28T16:36:08.330 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-28T16:36:08.470 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-28T16:36:08.680 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-28T16:36:09.079 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 113 MB in 5s (21.4 MB/s) 2024-03-28T16:36:09.232 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-28T16:36:09.249 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-28T16:36:09.362 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-28T16:36:09.775 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 113 MB in 5s (21.6 MB/s) 2024-03-28T16:36:09.960 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-28T16:36:11.285 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-28T16:36:11.290 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:11.413 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:12.003 INFO:teuthology.orchestra.run.smithi119.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-28T16:36:12.003 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:12.065 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:13.615 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-28T16:36:13.636 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:13.721 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:14.201 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-28T16:36:14.215 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:14.260 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:14.502 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-28T16:36:14.523 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:14.602 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:15.080 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-28T16:36:15.100 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:15.138 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:21.335 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-28T16:36:21.461 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-28T16:36:21.520 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-28T16:36:21.622 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-28T16:36:21.785 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-28T16:36:21.890 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-28T16:36:21.989 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-28T16:36:22.117 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-28T16:36:22.209 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-28T16:36:22.234 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-28T16:36:22.276 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-28T16:36:22.412 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-28T16:36:22.434 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-28T16:36:22.478 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-28T16:36:27.876 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-28T16:36:27.913 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:27.938 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-28T16:36:27.951 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:27.975 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-28T16:36:28.013 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:30.570 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-28T16:36:30.695 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-28T16:36:31.036 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-28T16:36:31.867 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:31.993 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-28T16:36:32.078 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-28T16:36:32.130 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:32.200 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-28T16:36:32.575 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-28T16:36:32.697 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:32.839 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-28T16:36:32.994 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:33.470 INFO:teuthology.orchestra.run.smithi119.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-28T16:36:33.471 INFO:teuthology.orchestra.run.smithi119.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-28T16:36:33.471 INFO:teuthology.orchestra.run.smithi119.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-28T16:36:33.471 INFO:teuthology.orchestra.run.smithi119.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:33.624 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:34.324 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-28T16:36:34.324 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-28T16:36:34.324 INFO:teuthology.orchestra.run.smithi170.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-28T16:36:34.324 INFO:teuthology.orchestra.run.smithi170.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:34.488 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:34.999 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:35.922 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:36.359 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-28T16:36:36.485 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-28T16:36:36.611 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:36.671 INFO:teuthology.orchestra.run.smithi119.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-28T16:36:36.672 INFO:teuthology.orchestra.run.smithi119.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:37.715 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-28T16:36:37.849 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-28T16:36:38.009 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-28T16:36:38.068 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-28T16:36:38.068 INFO:teuthology.orchestra.run.smithi170.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:50.545 INFO:teuthology.orchestra.run.smithi119.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-28T16:36:50.546 INFO:teuthology.orchestra.run.smithi119.stdout:Sourcing file `/etc/default/grub' 2024-03-28T16:36:50.557 INFO:teuthology.orchestra.run.smithi119.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-28T16:36:50.589 INFO:teuthology.orchestra.run.smithi119.stdout:Generating grub configuration file ... 2024-03-28T16:36:50.873 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-28T16:36:50.888 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:51.163 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-28T16:36:51.169 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-28T16:36:51.241 INFO:teuthology.orchestra.run.smithi119.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-28T16:36:51.247 INFO:teuthology.orchestra.run.smithi119.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-28T16:36:51.403 INFO:teuthology.orchestra.run.smithi119.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-28T16:36:51.403 INFO:teuthology.orchestra.run.smithi119.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-28T16:36:51.403 INFO:teuthology.orchestra.run.smithi119.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-28T16:36:51.451 INFO:teuthology.orchestra.run.smithi119.stdout:done 2024-03-28T16:36:51.873 DEBUG:teuthology.orchestra.run.smithi119:> dpkg -s linux-image-generic 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Package: linux-image-generic 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Status: install ok installed 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Priority: optional 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Section: kernel 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Installed-Size: 21 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Maintainer: Ubuntu Kernel Team 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: amd64 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Source: linux-meta 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Version: 5.15.0.101.98 2024-03-28T16:36:51.894 INFO:teuthology.orchestra.run.smithi119.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-28T16:36:51.895 INFO:teuthology.orchestra.run.smithi119.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-28T16:36:51.895 INFO:teuthology.orchestra.run.smithi119.stdout:Recommends: thermald 2024-03-28T16:36:51.895 INFO:teuthology.orchestra.run.smithi119.stdout:Description: Generic Linux kernel image 2024-03-28T16:36:51.895 INFO:teuthology.orchestra.run.smithi119.stdout: This package will always depend on the latest generic kernel image 2024-03-28T16:36:51.895 INFO:teuthology.orchestra.run.smithi119.stdout: available. 2024-03-28T16:36:51.895 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-28T16:36:51.895 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-28T16:36:51.895 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-28T16:36:51.895 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-28T16:36:51.895 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=None, version=distro) 2024-03-28T16:36:51.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get clean 2024-03-28T16:36:52.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-03-28T16:36:52.271 INFO:teuthology.orchestra.run.smithi170.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-28T16:36:52.271 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub' 2024-03-28T16:36:52.298 INFO:teuthology.orchestra.run.smithi170.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-28T16:36:52.314 INFO:teuthology.orchestra.run.smithi170.stdout:Generating grub configuration file ... 2024-03-28T16:36:52.352 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-28T16:36:52.352 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-28T16:36:52.423 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-28T16:36:52.505 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-28T16:36:52.708 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-28T16:36:52.734 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:53.119 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-28T16:36:53.124 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-28T16:36:53.194 INFO:teuthology.orchestra.run.smithi170.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-28T16:36:53.200 INFO:teuthology.orchestra.run.smithi170.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-28T16:36:53.376 INFO:teuthology.orchestra.run.smithi170.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-28T16:36:53.376 INFO:teuthology.orchestra.run.smithi170.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-28T16:36:53.376 INFO:teuthology.orchestra.run.smithi170.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-28T16:36:53.415 INFO:teuthology.orchestra.run.smithi170.stdout:done 2024-03-28T16:36:53.764 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-03-28T16:36:53.781 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-28T16:36:53.832 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-03-28T16:36:53.889 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-03-28T16:36:53.907 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-03-28T16:36:53.907 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-03-28T16:36:53.907 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-03-28T16:36:53.907 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-03-28T16:36:53.907 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-03-28T16:36:53.907 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.101.98 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-03-28T16:36:53.908 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-03-28T16:36:53.908 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-28T16:36:53.908 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-28T16:36:53.908 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-28T16:36:53.909 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-03-28T16:36:53.909 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-03-28T16:36:53.909 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get clean 2024-03-28T16:36:54.008 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-03-28T16:36:54.008 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-03-28T16:36:54.079 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-03-28T16:36:54.161 INFO:teuthology.orchestra.run.smithi119.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-28T16:36:54.161 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-03-28T16:36:54.161 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-28T16:36:54.161 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-28T16:36:54.161 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-28T16:36:54.161 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-28T16:36:54.162 INFO:teuthology.orchestra.run.smithi119.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-28T16:36:54.162 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-28T16:36:54.199 INFO:teuthology.orchestra.run.smithi119.stdout:0 upgraded, 0 newly installed, 0 to remove and 311 not upgraded. 2024-03-28T16:36:54.200 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-28T16:36:54.200 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-28T16:36:54.201 DEBUG:teuthology.orchestra.run.smithi119:> dpkg -s linux-image-generic 2024-03-28T16:36:54.212 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-28T16:36:54.232 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Package: linux-image-generic 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Status: install ok installed 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Priority: optional 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Section: kernel 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Installed-Size: 21 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Maintainer: Ubuntu Kernel Team 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: amd64 2024-03-28T16:36:54.260 INFO:teuthology.orchestra.run.smithi119.stdout:Source: linux-meta 2024-03-28T16:36:54.261 INFO:teuthology.orchestra.run.smithi119.stdout:Version: 5.15.0.101.98 2024-03-28T16:36:54.261 INFO:teuthology.orchestra.run.smithi119.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-28T16:36:54.261 INFO:teuthology.orchestra.run.smithi119.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-28T16:36:54.261 INFO:teuthology.orchestra.run.smithi119.stdout:Recommends: thermald 2024-03-28T16:36:54.261 INFO:teuthology.orchestra.run.smithi119.stdout:Description: Generic Linux kernel image 2024-03-28T16:36:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: This package will always depend on the latest generic kernel image 2024-03-28T16:36:54.261 INFO:teuthology.orchestra.run.smithi119.stdout: available. 2024-03-28T16:36:54.261 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-03-28T16:36:54.309 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.d4SY2gRMlj 2024-03-28T16:36:54.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub/grub.cfg /tmp/tmp.d4SY2gRMlj 2024-03-28T16:36:54.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.d4SY2gRMlj 2024-03-28T16:36:54.498 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.d4SY2gRMlj is 10KB 2024-03-28T16:36:54.511 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.d4SY2gRMlj 2024-03-28T16:36:54.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-28T16:36:54.569 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:36:54.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-28T16:36:54.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-28T16:36:54.637 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-28T16:36:54.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo update-grub 2024-03-28T16:36:55.546 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-03-28T16:36:55.564 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-28T16:36:55.614 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-03-28T16:36:55.770 INFO:teuthology.orchestra.run.smithi119.stderr:Sourcing file `/etc/default/grub' 2024-03-28T16:36:55.771 INFO:teuthology.orchestra.run.smithi119.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-28T16:36:55.773 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2024-03-28T16:36:55.797 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-03-28T16:36:55.797 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-28T16:36:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-28T16:36:55.955 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-28T16:36:55.962 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:55.988 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 0 newly installed, 0 to remove and 311 not upgraded. 2024-03-28T16:36:55.990 DEBUG:teuthology.orchestra.run.smithi170:> dpkg -s linux-image-generic 2024-03-28T16:36:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:Package: linux-image-generic 2024-03-28T16:36:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:Status: install ok installed 2024-03-28T16:36:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:Priority: optional 2024-03-28T16:36:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:Section: kernel 2024-03-28T16:36:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:Installed-Size: 21 2024-03-28T16:36:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:Maintainer: Ubuntu Kernel Team 2024-03-28T16:36:56.048 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: amd64 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout:Source: linux-meta 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout:Version: 5.15.0.101.98 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout:Recommends: thermald 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout:Description: Generic Linux kernel image 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout: This package will always depend on the latest generic kernel image 2024-03-28T16:36:56.049 INFO:teuthology.orchestra.run.smithi170.stdout: available. 2024-03-28T16:36:56.049 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-03-28T16:36:56.097 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.4Gqh1JVxRW 2024-03-28T16:36:56.097 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub/grub.cfg /tmp/tmp.4Gqh1JVxRW 2024-03-28T16:36:56.159 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.4Gqh1JVxRW 2024-03-28T16:36:56.189 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-28T16:36:56.194 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-28T16:36:56.237 INFO:teuthology.orchestra.run.smithi119.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-28T16:36:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-28T16:36:56.268 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.4Gqh1JVxRW is 10KB 2024-03-28T16:36:56.281 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.4Gqh1JVxRW 2024-03-28T16:36:56.287 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-28T16:36:56.340 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:36:56.340 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-28T16:36:56.340 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-28T16:36:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-28T16:36:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-28T16:36:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-28T16:36:56.385 INFO:teuthology.orchestra.run.smithi119.stderr:done 2024-03-28T16:36:56.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2024-03-28T16:36:56.408 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-28T16:36:56.408 DEBUG:teuthology.orchestra.run.smithi170:> sudo update-grub 2024-03-28T16:36:57.738 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub' 2024-03-28T16:36:57.739 INFO:teuthology.orchestra.run.smithi170.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-28T16:36:57.745 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-03-28T16:36:57.922 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-28T16:36:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-28T16:36:58.250 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-28T16:36:58.255 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-28T16:36:58.297 INFO:teuthology.orchestra.run.smithi170.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-28T16:36:58.302 INFO:teuthology.orchestra.run.smithi170.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-28T16:36:58.414 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-28T16:36:58.415 INFO:teuthology.orchestra.run.smithi170.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-28T16:36:58.415 INFO:teuthology.orchestra.run.smithi170.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-28T16:36:58.426 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-03-28T16:36:58.444 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-03-28T16:37:26.423 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-28T16:37:26.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T16:37:26.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:37:28.449 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-03-28T16:37:28.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-28T16:37:28.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:37:44.867 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-28T16:37:46.914 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-28T16:37:53.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T16:37:53.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:37:55.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-28T16:37:55.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:37:56.930 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-28T16:37:58.979 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-03-28T16:38:08.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T16:38:08.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:38:10.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-28T16:38:10.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:38:40.415 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2024-03-28T16:38:42.729 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-03-28T16:38:43.339 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-28T16:38:43.339 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.15.0-101-generic"... 2024-03-28T16:38:43.339 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-03-28T16:38:43.387 INFO:teuthology.orchestra.run.smithi170.stdout:5.15.0-101-generic 2024-03-28T16:38:43.387 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-28T16:38:43.387 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T16:38:43.387 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-28T16:38:44.388 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-03-28T16:38:44.389 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T16:38:44.480 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-03-28T16:38:44.508 DEBUG:teuthology.parallel:result is None 2024-03-28T16:38:55.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T16:38:55.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:38:55.756 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-03-28T16:38:56.289 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T16:38:56.289 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-03-28T16:38:56.289 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2024-03-28T16:38:56.335 INFO:teuthology.orchestra.run.smithi119.stdout:5.15.0-101-generic 2024-03-28T16:38:56.335 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-28T16:38:56.335 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T16:38:56.335 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-28T16:38:57.336 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-28T16:38:57.336 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-28T16:38:57.714 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2024-03-28T16:38:57.737 DEBUG:teuthology.parallel:result is None 2024-03-28T16:38:57.738 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-28T16:38:57.745 INFO:teuthology.task.internal:Creating test directory... 2024-03-28T16:38:57.745 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T16:38:57.748 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T16:38:57.754 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-28T16:38:57.762 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-28T16:38:57.772 INFO:teuthology.task.internal:Creating archive directory... 2024-03-28T16:38:57.773 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T16:38:57.800 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T16:38:57.881 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-28T16:38:57.888 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-28T16:38:57.888 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T16:38:57.891 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T16:38:57.912 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T16:38:57.920 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T16:38:57.923 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T16:38:57.931 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T16:38:57.933 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-28T16:38:57.940 INFO:teuthology.task.internal:Configuring sudo... 2024-03-28T16:38:57.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T16:38:57.976 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T16:38:58.008 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-28T16:38:58.018 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-28T16:38:58.018 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T16:38:58.032 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T16:38:58.055 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T16:38:58.084 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T16:38:58.131 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:38:58.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T16:38:58.193 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T16:38:58.199 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T16:38:58.247 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:38:58.247 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T16:38:58.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-03-28T16:38:58.311 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-03-28T16:38:58.381 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-28T16:38:58.389 INFO:teuthology.task.internal:Starting timer... 2024-03-28T16:38:58.389 INFO:teuthology.run_tasks:Running task pcp... 2024-03-28T16:38:58.399 INFO:teuthology.run_tasks:Running task selinux... 2024-03-28T16:38:58.408 DEBUG:teuthology.task.selinux:Excluding smithi119: OS 'ubuntu' does not support SELinux 2024-03-28T16:38:58.408 DEBUG:teuthology.task.selinux:Excluding smithi170: OS 'ubuntu' does not support SELinux 2024-03-28T16:38:58.408 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-28T16:38:58.408 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-28T16:38:58.408 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-28T16:38:58.408 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-28T16:38:58.417 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-28T16:38:58.423 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-28T16:38:58.541 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-28T16:38:58.559 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-28T16:38:58.560 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-28T16:45:58.088 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-03-28T16:45:58.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T16:45:58.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:45:58.169 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-03-28T16:45:58.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-03-28T16:45:58.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-28T16:45:58.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T16:45:58.324 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-03-28T16:45:58.398 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-28T16:45:58.398 INFO:teuthology.run_tasks:Running task clock... 2024-03-28T16:45:58.409 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-28T16:45:58.409 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T16:45:58.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T16:45:58.413 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T16:45:58.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Command line: ntpd -gq 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: ---------------------------------------------------- 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: ntp-4 is maintained by Network Time Foundation, 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: corporation. Support and training for ntp-4 are 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: available at https://www.nwtime.org/support 2024-03-28T16:45:58.441 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: ---------------------------------------------------- 2024-03-28T16:45:58.443 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: proto: precision = 0.046 usec (-24) 2024-03-28T16:45:58.443 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: basedate set to 2022-02-04 2024-03-28T16:45:58.443 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: gps base set to 2022-02-06 (week 2196) 2024-03-28T16:45:58.443 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-28T16:45:58.443 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: restrict ::: KOD does nothing without LIMITED. 2024-03-28T16:45:58.443 INFO:teuthology.orchestra.run.smithi119.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-28T16:45:58.443 INFO:teuthology.orchestra.run.smithi119.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-28T16:45:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Listen and drop on 0 v6wildcard [::]:123 2024-03-28T16:45:58.444 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-28T16:45:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Listen normally on 2 lo 127.0.0.1:123 2024-03-28T16:45:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Listen normally on 3 enp3s0f1 172.21.15.119:123 2024-03-28T16:45:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Listen normally on 4 lo [::1]:123 2024-03-28T16:45:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5ef%5]:123 2024-03-28T16:45:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:45:58 ntpd[17163]: Listening on routing socket on fd #22 for interface updates 2024-03-28T16:45:58.466 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-28T16:45:58.467 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Command line: ntpd -gq 2024-03-28T16:45:58.467 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: ---------------------------------------------------- 2024-03-28T16:45:58.467 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: ntp-4 is maintained by Network Time Foundation, 2024-03-28T16:45:58.467 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-28T16:45:58.467 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: corporation. Support and training for ntp-4 are 2024-03-28T16:45:58.467 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: available at https://www.nwtime.org/support 2024-03-28T16:45:58.467 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: ---------------------------------------------------- 2024-03-28T16:45:58.468 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: proto: precision = 0.055 usec (-24) 2024-03-28T16:45:58.468 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: basedate set to 2022-02-04 2024-03-28T16:45:58.468 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: gps base set to 2022-02-06 (week 2196) 2024-03-28T16:45:58.468 INFO:teuthology.orchestra.run.smithi170.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-28T16:45:58.469 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-28T16:45:58.469 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: restrict ::: KOD does nothing without LIMITED. 2024-03-28T16:45:58.469 INFO:teuthology.orchestra.run.smithi170.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-28T16:45:58.470 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Listen and drop on 0 v6wildcard [::]:123 2024-03-28T16:45:58.470 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-28T16:45:58.470 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Listen normally on 2 lo 127.0.0.1:123 2024-03-28T16:45:58.470 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Listen normally on 3 enp3s0f1 172.21.15.170:123 2024-03-28T16:45:58.470 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Listen normally on 4 lo [::1]:123 2024-03-28T16:45:58.470 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7255%5]:123 2024-03-28T16:45:58.470 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:45:58 ntpd[17164]: Listening on routing socket on fd #22 for interface updates 2024-03-28T16:45:59.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:45:59 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:45:59.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:45:59 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:45:59.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:45:59 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:45:59.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:45:59 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:45:59.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:45:59 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:45:59.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:45:59 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:00.441 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:00 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:00.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:00 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:01 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:01 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:01 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:01 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:01 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:01 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:01.467 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:01 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:01.467 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:01 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:01.467 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:01 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:01.467 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:01 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:01.467 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:01 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:01.467 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:01 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:02.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:02.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:02 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:02.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:02.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:02.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:02.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:02.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:02 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:03.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:03 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:03.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:03 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:03.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:03 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:03.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:03 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:03.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:03 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:03.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:03 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:03.467 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:03 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:03.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:03 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:03.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:03 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:03.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:03 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:03.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:03 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:03.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:03 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:04.444 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:04 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:04.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:04 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:05.443 INFO:teuthology.orchestra.run.smithi119.stdout:28 Mar 16:46:05 ntpd[17163]: ntpd: time slew -0.002112 s 2024-03-28T16:46:05.443 INFO:teuthology.orchestra.run.smithi119.stdout:ntpd: time slew -0.002112s 2024-03-28T16:46:05.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-28T16:46:05.443 INFO:teuthology.orchestra.run.smithi119.stderr:28 Mar 16:46:05 ntpd[17163]: can't open /var/log/ntpstats/loopstats.20240328: Permission denied 2024-03-28T16:46:05.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:05 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:05.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:05 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:05.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:05 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:05.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:05 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:05.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:05 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:05.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:05 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:05.499 INFO:teuthology.orchestra.run.smithi119.stdout: remote refid st t when poll reach delay offset jitter 2024-03-28T16:46:05.499 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================== 2024-03-28T16:46:05.499 INFO:teuthology.orchestra.run.smithi119.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:05.499 INFO:teuthology.orchestra.run.smithi119.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:05.499 INFO:teuthology.orchestra.run.smithi119.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:05.499 INFO:teuthology.orchestra.run.smithi119.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:06.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:06.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:06.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:06.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:06.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:06 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: can't open /var/log/ntpstats/rawstats.20240328: Permission denied 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: can't open /var/log/ntpstats/peerstats.20240328: Permission denied 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stdout:28 Mar 16:46:07 ntpd[17164]: ntpd: time slew -0.002206 s 2024-03-28T16:46:07.468 INFO:teuthology.orchestra.run.smithi170.stdout:ntpd: time slew -0.002206s 2024-03-28T16:46:07.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-28T16:46:07.469 INFO:teuthology.orchestra.run.smithi170.stderr:28 Mar 16:46:07 ntpd[17164]: can't open /var/log/ntpstats/loopstats.20240328: Permission denied 2024-03-28T16:46:07.518 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-03-28T16:46:07.518 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-03-28T16:46:07.518 INFO:teuthology.orchestra.run.smithi170.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:07.518 INFO:teuthology.orchestra.run.smithi170.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:07.518 INFO:teuthology.orchestra.run.smithi170.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:07.518 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:46:07.519 INFO:teuthology.run_tasks:Running task install... 2024-03-28T16:46:07.528 DEBUG:teuthology.task.install:project ceph 2024-03-28T16:46:07.528 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034'}} 2024-03-28T16:46:07.528 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034'} 2024-03-28T16:46:07.528 INFO:teuthology.task.install:Using flavor: default 2024-03-28T16:46:07.533 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-28T16:46:07.533 INFO:teuthology.task.install:extra packages: [] 2024-03-28T16:46:07.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-key list | grep Ceph 2024-03-28T16:46:07.534 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-key list | grep Ceph 2024-03-28T16:46:07.599 INFO:teuthology.orchestra.run.smithi119.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-28T16:46:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-28T16:46:07.626 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-28T16:46:07.626 INFO:teuthology.orchestra.run.smithi119.stdout:uid [ unknown] Ceph.com (release key) 2024-03-28T16:46:07.627 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-28T16:46:07.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=acb3247c84f48e9ce91359973bf22ebb44ec5034 2024-03-28T16:46:07.643 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-28T16:46:07.643 INFO:teuthology.orchestra.run.smithi170.stdout:uid [ unknown] Ceph.com (release key) 2024-03-28T16:46:07.643 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-28T16:46:07.644 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=acb3247c84f48e9ce91359973bf22ebb44ec5034 2024-03-28T16:46:07.783 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default/ 2024-03-28T16:46:07.784 INFO:teuthology.task.install.deb:Package version is 19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:46:07.900 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default/ 2024-03-28T16:46:07.901 INFO:teuthology.task.install.deb:Package version is 19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:46:07.976 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:46:07.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-28T16:46:07.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo apt-get update 2024-03-28T16:46:08.011 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:46:08.011 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-28T16:46:08.025 DEBUG:teuthology.orchestra.run.smithi170:> sudo apt-get update 2024-03-28T16:46:08.208 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-28T16:46:08.213 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy InRelease 2024-03-28T16:46:08.219 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-28T16:46:08.235 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-28T16:46:08.239 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-28T16:46:08.242 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy InRelease 2024-03-28T16:46:08.258 INFO:teuthology.orchestra.run.smithi119.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-28T16:46:08.263 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-28T16:46:08.287 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-28T16:46:08.287 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-28T16:46:08.288 INFO:teuthology.orchestra.run.smithi170.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-28T16:46:08.300 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-28T16:46:08.309 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-28T16:46:08.314 INFO:teuthology.orchestra.run.smithi170.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-28T16:46:08.318 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-03-28T16:46:08.336 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-03-28T16:46:08.360 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-28T16:46:08.444 INFO:teuthology.orchestra.run.smithi119.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-28T16:46:08.551 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 28.8 kB in 0s (82.0 kB/s) 2024-03-28T16:46:08.652 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 28.8 kB in 0s (59.9 kB/s) 2024-03-28T16:46:09.444 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-03-28T16:46:09.459 DEBUG:teuthology.orchestra.run.smithi170:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1426-gacb3247c-1jammy cephadm=19.0.0-1426-gacb3247c-1jammy ceph-mds=19.0.0-1426-gacb3247c-1jammy ceph-mgr=19.0.0-1426-gacb3247c-1jammy ceph-common=19.0.0-1426-gacb3247c-1jammy ceph-fuse=19.0.0-1426-gacb3247c-1jammy ceph-test=19.0.0-1426-gacb3247c-1jammy ceph-volume=19.0.0-1426-gacb3247c-1jammy radosgw=19.0.0-1426-gacb3247c-1jammy python3-rados=19.0.0-1426-gacb3247c-1jammy python3-rgw=19.0.0-1426-gacb3247c-1jammy python3-cephfs=19.0.0-1426-gacb3247c-1jammy python3-rbd=19.0.0-1426-gacb3247c-1jammy libcephfs2=19.0.0-1426-gacb3247c-1jammy libcephfs-dev=19.0.0-1426-gacb3247c-1jammy librados2=19.0.0-1426-gacb3247c-1jammy librbd1=19.0.0-1426-gacb3247c-1jammy rbd-fuse=19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:46:09.520 INFO:teuthology.orchestra.run.smithi170.stdout:Reading package lists... 2024-03-28T16:46:09.587 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-03-28T16:46:09.602 DEBUG:teuthology.orchestra.run.smithi119:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1426-gacb3247c-1jammy cephadm=19.0.0-1426-gacb3247c-1jammy ceph-mds=19.0.0-1426-gacb3247c-1jammy ceph-mgr=19.0.0-1426-gacb3247c-1jammy ceph-common=19.0.0-1426-gacb3247c-1jammy ceph-fuse=19.0.0-1426-gacb3247c-1jammy ceph-test=19.0.0-1426-gacb3247c-1jammy ceph-volume=19.0.0-1426-gacb3247c-1jammy radosgw=19.0.0-1426-gacb3247c-1jammy python3-rados=19.0.0-1426-gacb3247c-1jammy python3-rgw=19.0.0-1426-gacb3247c-1jammy python3-cephfs=19.0.0-1426-gacb3247c-1jammy python3-rbd=19.0.0-1426-gacb3247c-1jammy libcephfs2=19.0.0-1426-gacb3247c-1jammy libcephfs-dev=19.0.0-1426-gacb3247c-1jammy librados2=19.0.0-1426-gacb3247c-1jammy librbd1=19.0.0-1426-gacb3247c-1jammy rbd-fuse=19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:46:09.663 INFO:teuthology.orchestra.run.smithi119.stdout:Reading package lists... 2024-03-28T16:46:09.715 INFO:teuthology.orchestra.run.smithi170.stdout:Building dependency tree... 2024-03-28T16:46:09.716 INFO:teuthology.orchestra.run.smithi170.stdout:Reading state information... 2024-03-28T16:46:09.860 INFO:teuthology.orchestra.run.smithi119.stdout:Building dependency tree... 2024-03-28T16:46:09.860 INFO:teuthology.orchestra.run.smithi119.stdout:Reading state information... 2024-03-28T16:46:09.896 INFO:teuthology.orchestra.run.smithi170.stdout:The following packages were automatically installed and are no longer required: 2024-03-28T16:46:09.896 INFO:teuthology.orchestra.run.smithi170.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-28T16:46:09.896 INFO:teuthology.orchestra.run.smithi170.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-28T16:46:09.896 INFO:teuthology.orchestra.run.smithi170.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-28T16:46:09.896 INFO:teuthology.orchestra.run.smithi170.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-28T16:46:09.897 INFO:teuthology.orchestra.run.smithi170.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-28T16:46:09.897 INFO:teuthology.orchestra.run.smithi170.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-28T16:46:09.897 INFO:teuthology.orchestra.run.smithi170.stdout:The following additional packages will be installed: 2024-03-28T16:46:09.897 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-28T16:46:09.898 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-28T16:46:09.899 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-28T16:46:09.900 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet zip 2024-03-28T16:46:09.901 INFO:teuthology.orchestra.run.smithi170.stdout:Suggested packages: 2024-03-28T16:46:09.901 INFO:teuthology.orchestra.run.smithi170.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout:Recommended packages: 2024-03-28T16:46:09.902 INFO:teuthology.orchestra.run.smithi170.stdout: btrfs-tools 2024-03-28T16:46:10.001 INFO:teuthology.orchestra.run.smithi170.stdout:The following NEW packages will be installed: 2024-03-28T16:46:10.001 INFO:teuthology.orchestra.run.smithi170.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-28T16:46:10.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-28T16:46:10.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-28T16:46:10.002 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-28T16:46:10.002 INFO:teuthology.orchestra.run.smithi170.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-28T16:46:10.002 INFO:teuthology.orchestra.run.smithi170.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-28T16:46:10.003 INFO:teuthology.orchestra.run.smithi170.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-28T16:46:10.003 INFO:teuthology.orchestra.run.smithi170.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-28T16:46:10.003 INFO:teuthology.orchestra.run.smithi170.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-28T16:46:10.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-28T16:46:10.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-28T16:46:10.004 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-28T16:46:10.005 INFO:teuthology.orchestra.run.smithi170.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-28T16:46:10.041 INFO:teuthology.orchestra.run.smithi119.stdout:The following packages were automatically installed and are no longer required: 2024-03-28T16:46:10.041 INFO:teuthology.orchestra.run.smithi119.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-28T16:46:10.041 INFO:teuthology.orchestra.run.smithi119.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-28T16:46:10.041 INFO:teuthology.orchestra.run.smithi119.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-28T16:46:10.041 INFO:teuthology.orchestra.run.smithi119.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout:The following additional packages will be installed: 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-28T16:46:10.045 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-28T16:46:10.046 INFO:teuthology.orchestra.run.smithi119.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet zip 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout:Suggested packages: 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout:Recommended packages: 2024-03-28T16:46:10.047 INFO:teuthology.orchestra.run.smithi119.stdout: btrfs-tools 2024-03-28T16:46:10.069 INFO:teuthology.orchestra.run.smithi170.stdout:0 upgraded, 118 newly installed, 0 to remove and 298 not upgraded. 2024-03-28T16:46:10.069 INFO:teuthology.orchestra.run.smithi170.stdout:Need to get 176 MB of archives. 2024-03-28T16:46:10.069 INFO:teuthology.orchestra.run.smithi170.stdout:After this operation, 729 MB of additional disk space will be used. 2024-03-28T16:46:10.069 INFO:teuthology.orchestra.run.smithi170.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-28T16:46:10.144 INFO:teuthology.orchestra.run.smithi119.stdout:The following NEW packages will be installed: 2024-03-28T16:46:10.145 INFO:teuthology.orchestra.run.smithi119.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-28T16:46:10.145 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-28T16:46:10.145 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-28T16:46:10.145 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-28T16:46:10.145 INFO:teuthology.orchestra.run.smithi119.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-28T16:46:10.145 INFO:teuthology.orchestra.run.smithi119.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-28T16:46:10.146 INFO:teuthology.orchestra.run.smithi119.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-28T16:46:10.146 INFO:teuthology.orchestra.run.smithi119.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-28T16:46:10.146 INFO:teuthology.orchestra.run.smithi119.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-28T16:46:10.146 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-28T16:46:10.146 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-28T16:46:10.146 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-28T16:46:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-28T16:46:10.187 INFO:teuthology.orchestra.run.smithi170.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-28T16:46:10.189 INFO:teuthology.orchestra.run.smithi170.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-28T16:46:10.217 INFO:teuthology.orchestra.run.smithi170.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-28T16:46:10.226 INFO:teuthology.orchestra.run.smithi170.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1426-gacb3247c-1jammy [3,532 kB] 2024-03-28T16:46:10.287 INFO:teuthology.orchestra.run.smithi170.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-28T16:46:10.292 INFO:teuthology.orchestra.run.smithi170.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-28T16:46:10.306 INFO:teuthology.orchestra.run.smithi170.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-28T16:46:10.311 INFO:teuthology.orchestra.run.smithi170.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-28T16:46:10.311 INFO:teuthology.orchestra.run.smithi170.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-28T16:46:10.312 INFO:teuthology.orchestra.run.smithi170.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-28T16:46:10.321 INFO:teuthology.orchestra.run.smithi170.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-28T16:46:10.323 INFO:teuthology.orchestra.run.smithi170.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-28T16:46:10.325 INFO:teuthology.orchestra.run.smithi170.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-28T16:46:10.327 INFO:teuthology.orchestra.run.smithi170.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-28T16:46:10.333 INFO:teuthology.orchestra.run.smithi119.stdout:0 upgraded, 118 newly installed, 0 to remove and 298 not upgraded. 2024-03-28T16:46:10.333 INFO:teuthology.orchestra.run.smithi119.stdout:Need to get 176 MB of archives. 2024-03-28T16:46:10.334 INFO:teuthology.orchestra.run.smithi119.stdout:After this operation, 729 MB of additional disk space will be used. 2024-03-28T16:46:10.334 INFO:teuthology.orchestra.run.smithi119.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-28T16:46:10.339 INFO:teuthology.orchestra.run.smithi170.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-28T16:46:10.341 INFO:teuthology.orchestra.run.smithi170.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-28T16:46:10.343 INFO:teuthology.orchestra.run.smithi170.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-28T16:46:10.345 INFO:teuthology.orchestra.run.smithi170.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-28T16:46:10.346 INFO:teuthology.orchestra.run.smithi170.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-28T16:46:10.359 INFO:teuthology.orchestra.run.smithi170.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-28T16:46:10.360 INFO:teuthology.orchestra.run.smithi170.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-28T16:46:10.360 INFO:teuthology.orchestra.run.smithi170.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-28T16:46:10.361 INFO:teuthology.orchestra.run.smithi170.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-28T16:46:10.361 INFO:teuthology.orchestra.run.smithi170.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-28T16:46:10.375 INFO:teuthology.orchestra.run.smithi119.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1426-gacb3247c-1jammy [3,532 kB] 2024-03-28T16:46:10.378 INFO:teuthology.orchestra.run.smithi170.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-28T16:46:10.379 INFO:teuthology.orchestra.run.smithi170.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-28T16:46:10.379 INFO:teuthology.orchestra.run.smithi170.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-28T16:46:10.382 INFO:teuthology.orchestra.run.smithi170.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-28T16:46:10.382 INFO:teuthology.orchestra.run.smithi170.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-28T16:46:10.398 INFO:teuthology.orchestra.run.smithi170.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-28T16:46:10.398 INFO:teuthology.orchestra.run.smithi170.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-28T16:46:10.399 INFO:teuthology.orchestra.run.smithi170.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-28T16:46:10.400 INFO:teuthology.orchestra.run.smithi170.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-28T16:46:10.400 INFO:teuthology.orchestra.run.smithi170.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-28T16:46:10.420 INFO:teuthology.orchestra.run.smithi170.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-28T16:46:10.420 INFO:teuthology.orchestra.run.smithi170.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-28T16:46:10.421 INFO:teuthology.orchestra.run.smithi170.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-28T16:46:10.426 INFO:teuthology.orchestra.run.smithi170.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-28T16:46:10.427 INFO:teuthology.orchestra.run.smithi170.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-28T16:46:10.439 INFO:teuthology.orchestra.run.smithi170.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-28T16:46:10.440 INFO:teuthology.orchestra.run.smithi170.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-28T16:46:10.441 INFO:teuthology.orchestra.run.smithi170.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-28T16:46:10.443 INFO:teuthology.orchestra.run.smithi170.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-28T16:46:10.444 INFO:teuthology.orchestra.run.smithi170.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-28T16:46:10.459 INFO:teuthology.orchestra.run.smithi170.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-28T16:46:10.460 INFO:teuthology.orchestra.run.smithi170.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-28T16:46:10.462 INFO:teuthology.orchestra.run.smithi170.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-28T16:46:10.462 INFO:teuthology.orchestra.run.smithi170.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-28T16:46:10.463 INFO:teuthology.orchestra.run.smithi170.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-28T16:46:10.478 INFO:teuthology.orchestra.run.smithi170.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-28T16:46:10.560 INFO:teuthology.orchestra.run.smithi170.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-28T16:46:10.562 INFO:teuthology.orchestra.run.smithi170.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-28T16:46:10.564 INFO:teuthology.orchestra.run.smithi170.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-28T16:46:10.575 INFO:teuthology.orchestra.run.smithi170.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-28T16:46:10.577 INFO:teuthology.orchestra.run.smithi170.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-28T16:46:10.604 INFO:teuthology.orchestra.run.smithi170.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-28T16:46:10.606 INFO:teuthology.orchestra.run.smithi170.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-28T16:46:10.607 INFO:teuthology.orchestra.run.smithi170.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-28T16:46:10.633 INFO:teuthology.orchestra.run.smithi170.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-28T16:46:10.634 INFO:teuthology.orchestra.run.smithi170.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-28T16:46:10.636 INFO:teuthology.orchestra.run.smithi170.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1426-gacb3247c-1jammy [3,211 kB] 2024-03-28T16:46:10.657 INFO:teuthology.orchestra.run.smithi170.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-28T16:46:10.657 INFO:teuthology.orchestra.run.smithi170.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-28T16:46:10.658 INFO:teuthology.orchestra.run.smithi170.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-28T16:46:10.659 INFO:teuthology.orchestra.run.smithi170.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-28T16:46:10.659 INFO:teuthology.orchestra.run.smithi170.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-28T16:46:10.660 INFO:teuthology.orchestra.run.smithi170.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-28T16:46:10.664 INFO:teuthology.orchestra.run.smithi170.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-28T16:46:10.687 INFO:teuthology.orchestra.run.smithi170.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-28T16:46:10.689 INFO:teuthology.orchestra.run.smithi170.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-28T16:46:10.691 INFO:teuthology.orchestra.run.smithi170.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-28T16:46:10.719 INFO:teuthology.orchestra.run.smithi170.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-28T16:46:10.723 INFO:teuthology.orchestra.run.smithi170.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-28T16:46:10.725 INFO:teuthology.orchestra.run.smithi170.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-28T16:46:10.726 INFO:teuthology.orchestra.run.smithi170.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-28T16:46:10.733 INFO:teuthology.orchestra.run.smithi170.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-28T16:46:10.734 INFO:teuthology.orchestra.run.smithi170.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-28T16:46:10.739 INFO:teuthology.orchestra.run.smithi170.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-28T16:46:10.752 INFO:teuthology.orchestra.run.smithi170.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-28T16:46:10.753 INFO:teuthology.orchestra.run.smithi170.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-28T16:46:10.754 INFO:teuthology.orchestra.run.smithi170.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-28T16:46:10.755 INFO:teuthology.orchestra.run.smithi119.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-28T16:46:10.767 INFO:teuthology.orchestra.run.smithi119.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-28T16:46:10.777 INFO:teuthology.orchestra.run.smithi170.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-28T16:46:10.779 INFO:teuthology.orchestra.run.smithi170.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-28T16:46:10.782 INFO:teuthology.orchestra.run.smithi170.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1426-gacb3247c-1jammy [710 kB] 2024-03-28T16:46:10.790 INFO:teuthology.orchestra.run.smithi170.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-28T16:46:10.794 INFO:teuthology.orchestra.run.smithi170.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-28T16:46:10.794 INFO:teuthology.orchestra.run.smithi170.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-28T16:46:10.795 INFO:teuthology.orchestra.run.smithi170.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-28T16:46:10.796 INFO:teuthology.orchestra.run.smithi170.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-28T16:46:10.820 INFO:teuthology.orchestra.run.smithi170.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1426-gacb3247c-1jammy [356 kB] 2024-03-28T16:46:10.846 INFO:teuthology.orchestra.run.smithi170.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1426-gacb3247c-1jammy [32.8 kB] 2024-03-28T16:46:10.847 INFO:teuthology.orchestra.run.smithi170.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1426-gacb3247c-1jammy [180 kB] 2024-03-28T16:46:10.848 INFO:teuthology.orchestra.run.smithi119.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-28T16:46:10.853 INFO:teuthology.orchestra.run.smithi170.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1426-gacb3247c-1jammy [64.8 kB] 2024-03-28T16:46:10.856 INFO:teuthology.orchestra.run.smithi170.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1426-gacb3247c-1jammy [330 kB] 2024-03-28T16:46:10.864 INFO:teuthology.orchestra.run.smithi170.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1426-gacb3247c-1jammy [6,560 kB] 2024-03-28T16:46:11.079 INFO:teuthology.orchestra.run.smithi119.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-28T16:46:11.090 INFO:teuthology.orchestra.run.smithi119.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-28T16:46:11.095 INFO:teuthology.orchestra.run.smithi170.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1426-gacb3247c-1jammy [112 kB] 2024-03-28T16:46:11.098 INFO:teuthology.orchestra.run.smithi170.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1426-gacb3247c-1jammy [465 kB] 2024-03-28T16:46:11.110 INFO:teuthology.orchestra.run.smithi170.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1426-gacb3247c-1jammy [24.1 MB] 2024-03-28T16:46:11.121 INFO:teuthology.orchestra.run.smithi119.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-28T16:46:11.130 INFO:teuthology.orchestra.run.smithi119.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-28T16:46:11.131 INFO:teuthology.orchestra.run.smithi119.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-28T16:46:11.132 INFO:teuthology.orchestra.run.smithi119.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-28T16:46:11.153 INFO:teuthology.orchestra.run.smithi119.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-28T16:46:11.321 INFO:teuthology.orchestra.run.smithi119.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-28T16:46:11.725 INFO:teuthology.orchestra.run.smithi119.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-28T16:46:11.760 INFO:teuthology.orchestra.run.smithi119.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-28T16:46:11.761 INFO:teuthology.orchestra.run.smithi119.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-28T16:46:11.841 INFO:teuthology.orchestra.run.smithi119.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-28T16:46:11.893 INFO:teuthology.orchestra.run.smithi119.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-28T16:46:11.914 INFO:teuthology.orchestra.run.smithi119.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-28T16:46:11.920 INFO:teuthology.orchestra.run.smithi119.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-28T16:46:11.922 INFO:teuthology.orchestra.run.smithi119.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-28T16:46:11.929 INFO:teuthology.orchestra.run.smithi119.stdout:Get:22 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1426-gacb3247c-1jammy [3,211 kB] 2024-03-28T16:46:11.933 INFO:teuthology.orchestra.run.smithi119.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-28T16:46:11.934 INFO:teuthology.orchestra.run.smithi119.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-28T16:46:12.099 INFO:teuthology.orchestra.run.smithi119.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-28T16:46:12.182 INFO:teuthology.orchestra.run.smithi119.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-28T16:46:12.210 INFO:teuthology.orchestra.run.smithi119.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-28T16:46:12.227 INFO:teuthology.orchestra.run.smithi119.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-28T16:46:12.272 INFO:teuthology.orchestra.run.smithi119.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-28T16:46:12.336 INFO:teuthology.orchestra.run.smithi170.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1426-gacb3247c-1jammy [5,073 kB] 2024-03-28T16:46:12.482 INFO:teuthology.orchestra.run.smithi119.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-28T16:46:12.490 INFO:teuthology.orchestra.run.smithi170.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1426-gacb3247c-1jammy [243 kB] 2024-03-28T16:46:12.496 INFO:teuthology.orchestra.run.smithi170.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1426-gacb3247c-1jammy [124 kB] 2024-03-28T16:46:12.497 INFO:teuthology.orchestra.run.smithi119.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-28T16:46:12.500 INFO:teuthology.orchestra.run.smithi170.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1426-gacb3247c-1jammy [1,518 kB] 2024-03-28T16:46:12.505 INFO:teuthology.orchestra.run.smithi119.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-28T16:46:12.510 INFO:teuthology.orchestra.run.smithi119.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-28T16:46:12.531 INFO:teuthology.orchestra.run.smithi119.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-28T16:46:12.534 INFO:teuthology.orchestra.run.smithi119.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-28T16:46:12.561 INFO:teuthology.orchestra.run.smithi170.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1426-gacb3247c-1jammy [6,240 kB] 2024-03-28T16:46:12.661 INFO:teuthology.orchestra.run.smithi119.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1426-gacb3247c-1jammy [710 kB] 2024-03-28T16:46:12.701 INFO:teuthology.orchestra.run.smithi119.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-28T16:46:12.802 INFO:teuthology.orchestra.run.smithi170.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1426-gacb3247c-1jammy [22.7 MB] 2024-03-28T16:46:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1426-gacb3247c-1jammy [356 kB] 2024-03-28T16:46:12.887 INFO:teuthology.orchestra.run.smithi119.stdout:Get:39 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1426-gacb3247c-1jammy [32.8 kB] 2024-03-28T16:46:12.889 INFO:teuthology.orchestra.run.smithi119.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1426-gacb3247c-1jammy [180 kB] 2024-03-28T16:46:12.926 INFO:teuthology.orchestra.run.smithi119.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1426-gacb3247c-1jammy [64.8 kB] 2024-03-28T16:46:12.928 INFO:teuthology.orchestra.run.smithi119.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1426-gacb3247c-1jammy [330 kB] 2024-03-28T16:46:13.003 INFO:teuthology.orchestra.run.smithi119.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1426-gacb3247c-1jammy [6,560 kB] 2024-03-28T16:46:13.033 INFO:teuthology.orchestra.run.smithi119.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-28T16:46:13.064 INFO:teuthology.orchestra.run.smithi119.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-28T16:46:13.075 INFO:teuthology.orchestra.run.smithi119.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-28T16:46:13.277 INFO:teuthology.orchestra.run.smithi119.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-28T16:46:13.278 INFO:teuthology.orchestra.run.smithi119.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-28T16:46:13.285 INFO:teuthology.orchestra.run.smithi119.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-28T16:46:13.291 INFO:teuthology.orchestra.run.smithi119.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-28T16:46:13.306 INFO:teuthology.orchestra.run.smithi119.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-28T16:46:13.327 INFO:teuthology.orchestra.run.smithi119.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-28T16:46:13.334 INFO:teuthology.orchestra.run.smithi119.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-28T16:46:13.508 INFO:teuthology.orchestra.run.smithi119.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-28T16:46:13.761 INFO:teuthology.orchestra.run.smithi119.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-28T16:46:13.797 INFO:teuthology.orchestra.run.smithi170.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1426-gacb3247c-1jammy [14.0 kB] 2024-03-28T16:46:13.798 INFO:teuthology.orchestra.run.smithi170.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1426-gacb3247c-1jammy [865 kB] 2024-03-28T16:46:13.814 INFO:teuthology.orchestra.run.smithi170.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1426-gacb3247c-1jammy [2,185 kB] 2024-03-28T16:46:13.868 INFO:teuthology.orchestra.run.smithi170.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1426-gacb3247c-1jammy [776 kB] 2024-03-28T16:46:13.888 INFO:teuthology.orchestra.run.smithi170.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1426-gacb3247c-1jammy [144 kB] 2024-03-28T16:46:13.893 INFO:teuthology.orchestra.run.smithi170.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1426-gacb3247c-1jammy [1,920 kB] 2024-03-28T16:46:13.938 INFO:teuthology.orchestra.run.smithi119.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-28T16:46:13.952 INFO:teuthology.orchestra.run.smithi119.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-28T16:46:13.972 INFO:teuthology.orchestra.run.smithi170.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1426-gacb3247c-1jammy [8,625 kB] 2024-03-28T16:46:13.972 INFO:teuthology.orchestra.run.smithi119.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-28T16:46:14.036 INFO:teuthology.orchestra.run.smithi119.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-28T16:46:14.285 INFO:teuthology.orchestra.run.smithi119.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1426-gacb3247c-1jammy [112 kB] 2024-03-28T16:46:14.317 INFO:teuthology.orchestra.run.smithi119.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1426-gacb3247c-1jammy [465 kB] 2024-03-28T16:46:14.325 INFO:teuthology.orchestra.run.smithi170.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1426-gacb3247c-1jammy [14.2 kB] 2024-03-28T16:46:14.326 INFO:teuthology.orchestra.run.smithi170.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1426-gacb3247c-1jammy [49.9 MB] 2024-03-28T16:46:14.326 INFO:teuthology.orchestra.run.smithi119.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-28T16:46:14.328 INFO:teuthology.orchestra.run.smithi119.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-28T16:46:14.331 INFO:teuthology.orchestra.run.smithi119.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-28T16:46:14.332 INFO:teuthology.orchestra.run.smithi119.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-28T16:46:14.334 INFO:teuthology.orchestra.run.smithi119.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-28T16:46:14.401 INFO:teuthology.orchestra.run.smithi119.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1426-gacb3247c-1jammy [24.1 MB] 2024-03-28T16:46:14.551 INFO:teuthology.orchestra.run.smithi119.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-28T16:46:14.989 INFO:teuthology.orchestra.run.smithi119.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-28T16:46:14.996 INFO:teuthology.orchestra.run.smithi119.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-28T16:46:15.286 INFO:teuthology.orchestra.run.smithi119.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-28T16:46:15.286 INFO:teuthology.orchestra.run.smithi119.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-28T16:46:15.288 INFO:teuthology.orchestra.run.smithi119.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-28T16:46:15.290 INFO:teuthology.orchestra.run.smithi119.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-28T16:46:15.292 INFO:teuthology.orchestra.run.smithi119.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-28T16:46:15.294 INFO:teuthology.orchestra.run.smithi119.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-28T16:46:15.295 INFO:teuthology.orchestra.run.smithi119.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-28T16:46:15.297 INFO:teuthology.orchestra.run.smithi119.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-28T16:46:15.479 INFO:teuthology.orchestra.run.smithi119.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-28T16:46:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-28T16:46:15.933 INFO:teuthology.orchestra.run.smithi119.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-28T16:46:15.952 INFO:teuthology.orchestra.run.smithi119.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-28T16:46:16.064 INFO:teuthology.orchestra.run.smithi119.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-28T16:46:16.104 INFO:teuthology.orchestra.run.smithi119.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-28T16:46:16.116 INFO:teuthology.orchestra.run.smithi119.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-28T16:46:16.121 INFO:teuthology.orchestra.run.smithi119.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-28T16:46:16.168 INFO:teuthology.orchestra.run.smithi119.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-28T16:46:16.171 INFO:teuthology.orchestra.run.smithi119.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-28T16:46:16.193 INFO:teuthology.orchestra.run.smithi119.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-28T16:46:16.357 INFO:teuthology.orchestra.run.smithi119.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-28T16:46:16.537 INFO:teuthology.orchestra.run.smithi170.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1426-gacb3247c-1jammy [121 kB] 2024-03-28T16:46:16.539 INFO:teuthology.orchestra.run.smithi170.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1426-gacb3247c-1jammy [38.4 kB] 2024-03-28T16:46:16.540 INFO:teuthology.orchestra.run.smithi170.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1426-gacb3247c-1jammy [13.0 MB] 2024-03-28T16:46:16.622 INFO:teuthology.orchestra.run.smithi119.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-28T16:46:16.801 INFO:teuthology.orchestra.run.smithi119.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-28T16:46:17.097 INFO:teuthology.orchestra.run.smithi170.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1426-gacb3247c-1jammy [91.2 kB] 2024-03-28T16:46:17.195 INFO:teuthology.orchestra.run.smithi119.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-28T16:46:17.305 INFO:teuthology.orchestra.run.smithi119.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-28T16:46:17.593 INFO:teuthology.orchestra.run.smithi119.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-28T16:46:17.668 INFO:teuthology.orchestra.run.smithi170.stdout:Fetched 176 MB in 7s (24.8 MB/s) 2024-03-28T16:46:17.676 INFO:teuthology.orchestra.run.smithi119.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-28T16:46:17.759 INFO:teuthology.orchestra.run.smithi119.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-28T16:46:17.824 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-28T16:46:17.854 INFO:teuthology.orchestra.run.smithi119.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-28T16:46:17.925 INFO:teuthology.orchestra.run.smithi170.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-28T16:46:17.931 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-28T16:46:17.939 INFO:teuthology.orchestra.run.smithi119.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-28T16:46:18.014 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-28T16:46:18.384 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-28T16:46:18.411 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-28T16:46:18.451 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-28T16:46:18.644 INFO:teuthology.orchestra.run.smithi119.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1426-gacb3247c-1jammy [5,073 kB] 2024-03-28T16:46:18.771 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-28T16:46:18.798 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-28T16:46:18.829 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-28T16:46:19.274 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-28T16:46:19.284 INFO:teuthology.orchestra.run.smithi119.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1426-gacb3247c-1jammy [243 kB] 2024-03-28T16:46:19.301 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-28T16:46:19.318 INFO:teuthology.orchestra.run.smithi119.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1426-gacb3247c-1jammy [124 kB] 2024-03-28T16:46:19.324 INFO:teuthology.orchestra.run.smithi119.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1426-gacb3247c-1jammy [1,518 kB] 2024-03-28T16:46:19.349 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:46:19.506 INFO:teuthology.orchestra.run.smithi119.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1426-gacb3247c-1jammy [6,240 kB] 2024-03-28T16:46:19.736 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-28T16:46:19.762 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-28T16:46:19.802 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:46:20.139 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-28T16:46:20.166 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-28T16:46:20.205 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:46:20.299 INFO:teuthology.orchestra.run.smithi119.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1426-gacb3247c-1jammy [22.7 MB] 2024-03-28T16:46:20.525 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-28T16:46:20.552 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-28T16:46:20.583 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-28T16:46:20.886 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librados2. 2024-03-28T16:46:20.913 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../007-librados2_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:20.952 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librados2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:21.447 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librbd1. 2024-03-28T16:46:21.474 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../008-librbd1_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:21.514 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librbd1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:21.950 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs2. 2024-03-28T16:46:21.977 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:22.016 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:22.269 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rados. 2024-03-28T16:46:22.284 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:22.318 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rados (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:22.630 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-28T16:46:22.657 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:22.696 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-argparse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:22.949 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cephfs. 2024-03-28T16:46:22.975 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:23.015 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cephfs (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:23.318 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-28T16:46:23.344 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:23.385 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:23.697 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-28T16:46:23.723 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-28T16:46:23.754 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-28T16:46:24.057 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-prettytable. 2024-03-28T16:46:24.083 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-28T16:46:24.124 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-28T16:46:24.385 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rbd. 2024-03-28T16:46:24.411 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:24.451 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rbd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:24.779 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-28T16:46:24.806 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-28T16:46:24.846 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-28T16:46:25.115 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-28T16:46:25.131 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-28T16:46:25.190 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-28T16:46:25.476 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-28T16:46:25.491 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-28T16:46:25.534 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-28T16:46:25.846 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua5.1. 2024-03-28T16:46:25.861 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-28T16:46:25.904 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-28T16:46:26.224 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-any. 2024-03-28T16:46:26.251 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-28T16:46:26.291 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-28T16:46:26.560 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package zip. 2024-03-28T16:46:26.587 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-28T16:46:26.627 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking zip (3.0-12build2) ... 2024-03-28T16:46:26.913 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package unzip. 2024-03-28T16:46:26.940 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-28T16:46:26.980 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-28T16:46:27.342 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package luarocks. 2024-03-28T16:46:27.369 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-28T16:46:27.408 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-28T16:46:27.783 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package librgw2. 2024-03-28T16:46:27.810 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../025-librgw2_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:27.849 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking librgw2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:28.360 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rgw. 2024-03-28T16:46:28.376 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:28.410 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:28.730 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-28T16:46:28.757 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-28T16:46:28.796 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-28T16:46:29.107 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libradosstriper1. 2024-03-28T16:46:29.134 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:29.174 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libradosstriper1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:29.611 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-common. 2024-03-28T16:46:29.638 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:29.677 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:29.681 INFO:teuthology.orchestra.run.smithi119.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1426-gacb3247c-1jammy [14.0 kB] 2024-03-28T16:46:29.682 INFO:teuthology.orchestra.run.smithi119.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1426-gacb3247c-1jammy [865 kB] 2024-03-28T16:46:29.881 INFO:teuthology.orchestra.run.smithi119.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1426-gacb3247c-1jammy [2,185 kB] 2024-03-28T16:46:30.378 INFO:teuthology.orchestra.run.smithi119.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1426-gacb3247c-1jammy [776 kB] 2024-03-28T16:46:30.529 INFO:teuthology.orchestra.run.smithi119.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1426-gacb3247c-1jammy [144 kB] 2024-03-28T16:46:30.546 INFO:teuthology.orchestra.run.smithi119.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1426-gacb3247c-1jammy [1,920 kB] 2024-03-28T16:46:30.974 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-base. 2024-03-28T16:46:31.001 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:31.019 INFO:teuthology.orchestra.run.smithi119.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1426-gacb3247c-1jammy [8,625 kB] 2024-03-28T16:46:31.050 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-base (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:31.645 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-28T16:46:31.671 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-28T16:46:31.711 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-28T16:46:31.989 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cheroot. 2024-03-28T16:46:32.016 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-28T16:46:32.056 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-28T16:46:32.230 INFO:teuthology.orchestra.run.smithi119.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1426-gacb3247c-1jammy [14.2 kB] 2024-03-28T16:46:32.231 INFO:teuthology.orchestra.run.smithi119.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1426-gacb3247c-1jammy [49.9 MB] 2024-03-28T16:46:32.384 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-28T16:46:32.411 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-28T16:46:32.450 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-28T16:46:32.753 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-28T16:46:32.780 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-28T16:46:32.820 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-28T16:46:33.123 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-28T16:46:33.150 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-28T16:46:33.189 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-28T16:46:33.484 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempora. 2024-03-28T16:46:33.511 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-28T16:46:33.550 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-28T16:46:33.853 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-portend. 2024-03-28T16:46:33.880 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-28T16:46:33.919 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-28T16:46:34.190 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-28T16:46:34.216 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-28T16:46:34.256 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-28T16:46:34.559 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-28T16:46:34.586 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-28T16:46:34.625 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-28T16:46:35.004 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-natsort. 2024-03-28T16:46:35.031 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-28T16:46:35.070 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-28T16:46:35.382 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-logutils. 2024-03-28T16:46:35.409 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-28T16:46:35.448 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-28T16:46:35.684 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-28T16:46:35.700 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-28T16:46:35.743 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-28T16:46:36.329 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-mako. 2024-03-28T16:46:36.356 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-28T16:46:36.395 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-28T16:46:36.724 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-28T16:46:36.751 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-28T16:46:36.790 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-28T16:46:37.093 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-28T16:46:37.120 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-28T16:46:37.159 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-28T16:46:37.421 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webob. 2024-03-28T16:46:37.448 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-28T16:46:37.470 INFO:teuthology.orchestra.run.smithi119.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1426-gacb3247c-1jammy [121 kB] 2024-03-28T16:46:37.473 INFO:teuthology.orchestra.run.smithi119.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1426-gacb3247c-1jammy [38.4 kB] 2024-03-28T16:46:37.474 INFO:teuthology.orchestra.run.smithi119.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1426-gacb3247c-1jammy [13.0 MB] 2024-03-28T16:46:37.487 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-28T16:46:37.815 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-waitress. 2024-03-28T16:46:37.842 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-28T16:46:37.885 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-28T16:46:38.193 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-tempita. 2024-03-28T16:46:38.220 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-28T16:46:38.276 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-28T16:46:38.481 INFO:teuthology.orchestra.run.smithi119.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip_thotz_update_attrs_from_state_setattrs/acb3247c84f48e9ce91359973bf22ebb44ec5034/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1426-gacb3247c-1jammy [91.2 kB] 2024-03-28T16:46:38.579 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-paste. 2024-03-28T16:46:38.606 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-28T16:46:38.646 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-28T16:46:38.949 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-28T16:46:38.965 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-28T16:46:39.007 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-28T16:46:39.039 INFO:teuthology.orchestra.run.smithi119.stdout:Fetched 176 MB in 28s (6,204 kB/s) 2024-03-28T16:46:39.165 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-28T16:46:39.263 INFO:teuthology.orchestra.run.smithi119.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-28T16:46:39.269 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-28T16:46:39.310 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-28T16:46:39.337 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-28T16:46:39.338 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-28T16:46:39.376 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-28T16:46:39.634 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-28T16:46:39.660 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-28T16:46:39.671 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-webtest. 2024-03-28T16:46:39.698 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-28T16:46:39.700 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-28T16:46:39.738 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-28T16:46:40.020 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-28T16:46:40.024 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pecan. 2024-03-28T16:46:40.047 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-28T16:46:40.051 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-28T16:46:40.082 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-28T16:46:40.086 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-28T16:46:40.395 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-certifi. 2024-03-28T16:46:40.411 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-28T16:46:40.453 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-28T16:46:40.532 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-28T16:46:40.558 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-28T16:46:40.606 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:46:40.782 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-idna. 2024-03-28T16:46:40.809 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-28T16:46:40.848 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-28T16:46:41.001 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-28T16:46:41.028 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-28T16:46:41.060 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:46:41.168 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-urllib3. 2024-03-28T16:46:41.195 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-28T16:46:41.234 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-28T16:46:41.379 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-28T16:46:41.406 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-28T16:46:41.446 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:46:41.596 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests. 2024-03-28T16:46:41.622 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-28T16:46:41.662 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-28T16:46:41.982 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-28T16:46:42.008 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-28T16:46:42.048 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-28T16:46:42.335 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-28T16:46:42.361 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:42.401 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:42.657 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-28T16:46:42.684 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-28T16:46:42.724 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-28T16:46:42.848 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-28T16:46:42.875 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:42.931 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:43.052 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librados2. 2024-03-28T16:46:43.079 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../007-librados2_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:43.119 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librados2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:43.242 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-28T16:46:43.269 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-28T16:46:43.308 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-28T16:46:43.571 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librbd1. 2024-03-28T16:46:43.598 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../008-librbd1_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:43.638 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librbd1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:44.049 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs2. 2024-03-28T16:46:44.075 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:44.115 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:44.404 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rados. 2024-03-28T16:46:44.431 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:44.437 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-openssl. 2024-03-28T16:46:44.464 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-28T16:46:44.470 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rados (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:44.503 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-28T16:46:44.806 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-28T16:46:44.833 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:44.873 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-argparse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:44.906 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr. 2024-03-28T16:46:44.934 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:44.981 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:45.134 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cephfs. 2024-03-28T16:46:45.160 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:45.200 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cephfs (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:45.393 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mon. 2024-03-28T16:46:45.420 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:45.459 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mon (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:45.486 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-28T16:46:45.513 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:45.553 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:45.881 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-28T16:46:45.908 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-28T16:46:45.948 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-28T16:46:46.071 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-osd. 2024-03-28T16:46:46.098 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:46.137 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-osd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:46.276 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-prettytable. 2024-03-28T16:46:46.303 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-28T16:46:46.342 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-28T16:46:46.603 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rbd. 2024-03-28T16:46:46.619 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:46.653 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rbd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:46.965 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-28T16:46:46.991 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-28T16:46:47.032 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-28T16:46:47.218 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph. 2024-03-28T16:46:47.245 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../066-ceph_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:47.284 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:47.292 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-28T16:46:47.319 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-28T16:46:47.470 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-28T16:46:47.604 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-fuse. 2024-03-28T16:46:47.631 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:47.662 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:47.745 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-28T16:46:47.768 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-28T16:46:47.812 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-28T16:46:48.032 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mds. 2024-03-28T16:46:48.059 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:48.098 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mds (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:48.165 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua5.1. 2024-03-28T16:46:48.191 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-28T16:46:48.232 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-28T16:46:48.502 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package cephadm. 2024-03-28T16:46:48.529 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../069-cephadm_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:48.551 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-any. 2024-03-28T16:46:48.566 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-28T16:46:48.568 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:48.609 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-28T16:46:48.845 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package zip. 2024-03-28T16:46:48.854 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-28T16:46:48.861 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-28T16:46:48.870 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-28T16:46:48.895 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking zip (3.0-12build2) ... 2024-03-28T16:46:48.904 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-28T16:46:49.215 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package unzip. 2024-03-28T16:46:49.232 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-28T16:46:49.242 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-28T16:46:49.262 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-28T16:46:49.282 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-28T16:46:49.307 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-28T16:46:49.669 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package luarocks. 2024-03-28T16:46:49.695 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-28T16:46:49.735 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-28T16:46:50.144 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package librgw2. 2024-03-28T16:46:50.170 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../025-librgw2_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:50.210 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking librgw2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:50.304 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-babel. 2024-03-28T16:46:50.320 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-28T16:46:50.353 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-28T16:46:50.657 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-jinja2. 2024-03-28T16:46:50.685 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-28T16:46:50.723 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-28T16:46:50.730 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rgw. 2024-03-28T16:46:50.745 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:50.788 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:51.043 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-28T16:46:51.070 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:51.110 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:51.115 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-28T16:46:51.142 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-28T16:46:51.182 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-28T16:46:51.422 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-28T16:46:51.449 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-28T16:46:51.468 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libradosstriper1. 2024-03-28T16:46:51.488 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-28T16:46:51.494 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:51.526 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libradosstriper1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:51.947 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-common. 2024-03-28T16:46:51.973 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:52.013 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:52.091 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-routes. 2024-03-28T16:46:52.118 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-28T16:46:52.157 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-28T16:46:52.503 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-28T16:46:52.530 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:52.569 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:53.006 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-28T16:46:53.022 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-28T16:46:53.055 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-28T16:46:53.385 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-base. 2024-03-28T16:46:53.411 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:46:53.461 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-base (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:53.543 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-joblib. 2024-03-28T16:46:53.570 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-28T16:46:53.609 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-28T16:46:54.013 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-28T16:46:54.040 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-28T16:46:54.073 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-28T16:46:54.079 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-28T16:46:54.099 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-28T16:46:54.139 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-28T16:46:54.374 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-sklearn. 2024-03-28T16:46:54.400 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cheroot. 2024-03-28T16:46:54.401 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-28T16:46:54.426 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-28T16:46:54.440 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-28T16:46:54.467 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-28T16:46:54.778 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-28T16:46:54.805 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-28T16:46:54.853 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-28T16:46:55.156 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-28T16:46:55.166 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-28T16:46:55.182 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-28T16:46:55.193 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:55.222 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-28T16:46:55.318 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:46:55.517 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-28T16:46:55.544 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-28T16:46:55.584 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-28T16:46:55.853 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tempora. 2024-03-28T16:46:55.880 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-28T16:46:55.928 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-28T16:46:56.232 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-portend. 2024-03-28T16:46:56.258 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-28T16:46:56.298 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-28T16:46:56.313 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-cachetools. 2024-03-28T16:46:56.340 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-28T16:46:56.379 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-28T16:46:56.601 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-28T16:46:56.630 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-28T16:46:56.666 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-28T16:46:56.667 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-28T16:46:56.693 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-28T16:46:56.733 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-28T16:46:56.970 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-28T16:46:56.996 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-28T16:46:57.036 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-28T16:46:57.045 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-28T16:46:57.063 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-28T16:46:57.102 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-28T16:46:57.397 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-rsa. 2024-03-28T16:46:57.415 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-natsort. 2024-03-28T16:46:57.425 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-28T16:46:57.442 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-28T16:46:57.455 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-28T16:46:57.490 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-28T16:46:57.777 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-google-auth. 2024-03-28T16:46:57.804 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-28T16:46:57.835 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-logutils. 2024-03-28T16:46:57.843 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-28T16:46:57.862 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-28T16:46:57.902 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-28T16:46:58.197 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-28T16:46:58.224 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-28T16:46:58.265 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-28T16:46:58.372 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-28T16:46:58.398 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-28T16:46:58.438 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-28T16:46:58.641 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-websocket. 2024-03-28T16:46:58.668 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-28T16:46:58.716 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-28T16:46:58.758 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-mako. 2024-03-28T16:46:58.784 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-28T16:46:58.832 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-28T16:46:59.061 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-28T16:46:59.088 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-28T16:46:59.127 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-28T16:46:59.236 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-28T16:46:59.262 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-28T16:46:59.310 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-28T16:46:59.630 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-28T16:46:59.657 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-28T16:46:59.697 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-28T16:46:59.817 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-28T16:46:59.845 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:46:59.883 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:00.000 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webob. 2024-03-28T16:47:00.026 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-28T16:47:00.066 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-28T16:47:00.195 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-28T16:47:00.222 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-28T16:47:00.261 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-28T16:47:00.419 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-waitress. 2024-03-28T16:47:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-28T16:47:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-28T16:47:00.556 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-28T16:47:00.583 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-28T16:47:00.622 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-28T16:47:00.839 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-tempita. 2024-03-28T16:47:00.865 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-28T16:47:00.891 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package jq. 2024-03-28T16:47:00.907 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-28T16:47:00.914 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-28T16:47:00.958 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-28T16:47:01.211 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package socat. 2024-03-28T16:47:01.227 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-28T16:47:01.269 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-28T16:47:01.283 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-paste. 2024-03-28T16:47:01.310 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-28T16:47:01.358 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-28T16:47:01.564 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package xmlstarlet. 2024-03-28T16:47:01.591 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-28T16:47:01.647 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-28T16:47:01.686 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-28T16:47:01.713 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-28T16:47:01.753 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-28T16:47:01.985 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-test. 2024-03-28T16:47:02.001 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:02.060 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-test (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:02.072 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-28T16:47:02.099 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-28T16:47:02.139 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-28T16:47:02.442 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-webtest. 2024-03-28T16:47:02.469 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-28T16:47:02.509 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-28T16:47:02.778 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pecan. 2024-03-28T16:47:02.805 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-28T16:47:02.836 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-28T16:47:03.358 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-certifi. 2024-03-28T16:47:03.385 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-28T16:47:03.425 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-28T16:47:03.761 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-idna. 2024-03-28T16:47:03.788 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-28T16:47:03.836 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-28T16:47:04.223 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-urllib3. 2024-03-28T16:47:04.249 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-28T16:47:04.297 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-28T16:47:04.389 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package ceph-volume. 2024-03-28T16:47:04.417 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:47:04.456 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking ceph-volume (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:04.684 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-requests. 2024-03-28T16:47:04.711 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-28T16:47:04.759 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-28T16:47:04.760 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-28T16:47:04.787 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:04.826 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking libcephfs-dev (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:05.103 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-28T16:47:05.131 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-28T16:47:05.137 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-28T16:47:05.165 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-28T16:47:05.170 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-28T16:47:05.204 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-28T16:47:05.507 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-28T16:47:05.533 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:47:05.534 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-28T16:47:05.561 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-28T16:47:05.573 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:05.600 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-28T16:47:05.971 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package nvme-cli. 2024-03-28T16:47:05.998 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-28T16:47:06.003 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-28T16:47:06.030 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:06.037 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-28T16:47:06.077 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:06.464 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-28T16:47:06.476 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package pkg-config. 2024-03-28T16:47:06.491 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-28T16:47:06.503 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-28T16:47:06.539 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-28T16:47:06.542 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-28T16:47:06.795 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-28T16:47:06.823 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-28T16:47:06.851 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-openssl. 2024-03-28T16:47:06.862 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-28T16:47:06.877 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-28T16:47:06.917 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-28T16:47:07.253 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr. 2024-03-28T16:47:07.279 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:07.319 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:07.549 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-28T16:47:07.576 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-28T16:47:07.615 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-28T16:47:07.765 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mon. 2024-03-28T16:47:07.791 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:07.839 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mon (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:07.876 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pastescript. 2024-03-28T16:47:07.892 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-28T16:47:07.926 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-28T16:47:08.196 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pluggy. 2024-03-28T16:47:08.224 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-28T16:47:08.254 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-28T16:47:08.485 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-osd. 2024-03-28T16:47:08.511 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:08.551 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-osd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:08.565 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-psutil. 2024-03-28T16:47:08.593 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-28T16:47:08.632 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-28T16:47:08.977 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-py. 2024-03-28T16:47:09.005 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-28T16:47:09.043 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-28T16:47:09.355 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pygments. 2024-03-28T16:47:09.383 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-28T16:47:09.421 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-28T16:47:09.663 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph. 2024-03-28T16:47:09.690 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../066-ceph_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:09.729 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:09.859 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-28T16:47:09.875 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-28T16:47:09.909 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-28T16:47:09.999 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-fuse. 2024-03-28T16:47:10.014 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:10.048 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:10.254 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-toml. 2024-03-28T16:47:10.281 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-28T16:47:10.322 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-28T16:47:10.419 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mds. 2024-03-28T16:47:10.445 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:10.485 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mds (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:10.623 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-pytest. 2024-03-28T16:47:10.650 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-28T16:47:10.689 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-28T16:47:10.872 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package cephadm. 2024-03-28T16:47:10.898 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../069-cephadm_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:10.938 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:11.026 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package python3-simplejson. 2024-03-28T16:47:11.054 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-28T16:47:11.093 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-28T16:47:11.274 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-28T16:47:11.301 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-28T16:47:11.341 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-28T16:47:11.362 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-28T16:47:11.390 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-28T16:47:11.429 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-28T16:47:11.660 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-28T16:47:11.687 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-28T16:47:11.726 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-28T16:47:12.178 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package radosgw. 2024-03-28T16:47:12.202 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../116-radosgw_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:12.241 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking radosgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:12.723 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-babel. 2024-03-28T16:47:12.739 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-28T16:47:12.773 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-28T16:47:13.011 INFO:teuthology.orchestra.run.smithi170.stdout:Selecting previously unselected package rbd-fuse. 2024-03-28T16:47:13.039 INFO:teuthology.orchestra.run.smithi170.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:13.077 INFO:teuthology.orchestra.run.smithi170.stdout:Unpacking rbd-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:13.093 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-jinja2. 2024-03-28T16:47:13.120 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-28T16:47:13.151 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-28T16:47:13.381 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-28T16:47:13.487 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-28T16:47:13.503 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:47:13.546 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:13.707 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-28T16:47:13.833 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-28T16:47:13.841 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-28T16:47:13.868 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-28T16:47:13.899 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-28T16:47:14.092 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-28T16:47:14.185 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-routes. 2024-03-28T16:47:14.212 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-28T16:47:14.251 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-28T16:47:14.441 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-28T16:47:14.555 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-28T16:47:14.582 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:47:14.621 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:14.883 INFO:teuthology.orchestra.run.smithi170.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-28T16:47:14.944 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:15.008 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-28T16:47:15.024 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-28T16:47:15.066 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-28T16:47:15.487 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user cephadm....done 2024-03-28T16:47:15.587 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-joblib. 2024-03-28T16:47:15.614 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-28T16:47:15.642 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-28T16:47:15.653 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-28T16:47:15.910 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-28T16:47:15.990 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-28T16:47:16.017 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-28T16:47:16.056 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-28T16:47:16.144 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-28T16:47:16.270 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-28T16:47:16.309 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-sklearn. 2024-03-28T16:47:16.359 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-28T16:47:16.375 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-28T16:47:16.521 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-28T16:47:16.781 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-28T16:47:16.907 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-28T16:47:17.126 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-28T16:47:17.154 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:47:17.226 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-28T16:47:17.278 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:17.586 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-28T16:47:17.837 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-28T16:47:18.114 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-28T16:47:18.215 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-cachetools. 2024-03-28T16:47:18.242 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-28T16:47:18.281 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-28T16:47:18.382 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-28T16:47:18.568 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-28T16:47:18.595 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-28T16:47:18.634 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-28T16:47:18.650 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-28T16:47:18.918 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-28T16:47:18.946 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-28T16:47:18.973 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-28T16:47:19.012 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-28T16:47:19.235 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-argparse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:19.307 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-rsa. 2024-03-28T16:47:19.334 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-28T16:47:19.365 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-28T16:47:19.561 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-28T16:47:19.662 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-28T16:47:19.780 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-28T16:47:19.894 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-google-auth. 2024-03-28T16:47:19.906 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-28T16:47:19.921 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-28T16:47:19.961 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-28T16:47:20.157 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-28T16:47:20.306 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-28T16:47:20.333 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-28T16:47:20.372 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-28T16:47:20.497 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-28T16:47:20.530 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-28T16:47:20.594 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-28T16:47:20.695 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-28T16:47:20.700 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-websocket. 2024-03-28T16:47:20.727 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-28T16:47:20.766 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-28T16:47:21.038 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-28T16:47:21.112 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-28T16:47:21.139 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-28T16:47:21.164 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-28T16:47:21.186 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-28T16:47:21.432 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-28T16:47:21.691 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-28T16:47:21.877 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-28T16:47:21.894 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:47:21.935 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:22.001 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up zip (3.0-12build2) ... 2024-03-28T16:47:22.110 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-28T16:47:22.222 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-28T16:47:22.249 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-28T16:47:22.288 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-28T16:47:22.633 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-28T16:47:22.660 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-28T16:47:22.699 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-28T16:47:22.804 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-28T16:47:22.927 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package jq. 2024-03-28T16:47:22.943 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-28T16:47:22.977 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-28T16:47:23.055 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-28T16:47:23.198 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-28T16:47:23.221 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package socat. 2024-03-28T16:47:23.234 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-28T16:47:23.271 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-28T16:47:23.324 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-28T16:47:23.617 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package xmlstarlet. 2024-03-28T16:47:23.641 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-28T16:47:23.644 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-28T16:47:23.683 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-28T16:47:24.026 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-28T16:47:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-test. 2024-03-28T16:47:24.061 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:24.095 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-test (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:24.260 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-28T16:47:24.629 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-28T16:47:24.905 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-28T16:47:25.256 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-28T16:47:25.507 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-28T16:47:25.625 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:25.910 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-idna (3.3-1) ... 2024-03-28T16:47:26.141 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package ceph-volume. 2024-03-28T16:47:26.157 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1426-gacb3247c-1jammy_all.deb ... 2024-03-28T16:47:26.191 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking ceph-volume (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:26.211 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-28T16:47:26.445 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-28T16:47:26.461 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:26.494 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking libcephfs-dev (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:26.806 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-28T16:47:26.833 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-28T16:47:26.872 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-28T16:47:27.185 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-28T16:47:27.213 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-28T16:47:27.252 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-28T16:47:27.384 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-28T16:47:27.594 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:47:27.606 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package nvme-cli. 2024-03-28T16:47:27.633 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-28T16:47:27.672 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-28T16:47:27.720 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-28T16:47:28.061 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package pkg-config. 2024-03-28T16:47:28.089 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-28T16:47:28.119 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-28T16:47:28.204 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-28T16:47:28.330 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-28T16:47:28.364 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-28T16:47:28.391 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-28T16:47:28.430 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-28T16:47:28.473 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-28T16:47:28.758 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-28T16:47:28.859 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-28T16:47:28.886 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-28T16:47:28.925 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-28T16:47:29.009 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-28T16:47:29.228 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pastescript. 2024-03-28T16:47:29.252 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:47:29.255 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-28T16:47:29.294 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-28T16:47:29.369 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-28T16:47:29.631 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pluggy. 2024-03-28T16:47:29.659 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-28T16:47:29.679 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-28T16:47:29.697 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-28T16:47:29.930 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-28T16:47:29.992 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-psutil. 2024-03-28T16:47:30.020 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-28T16:47:30.059 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-28T16:47:30.190 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-28T16:47:30.449 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-28T16:47:30.703 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-py. 2024-03-28T16:47:30.710 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-28T16:47:30.730 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-28T16:47:30.770 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-28T16:47:30.970 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-28T16:47:31.088 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-28T16:47:31.123 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pygments. 2024-03-28T16:47:31.150 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-28T16:47:31.189 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-28T16:47:31.364 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-28T16:47:31.490 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-28T16:47:31.610 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-28T16:47:31.626 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-28T16:47:31.660 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-28T16:47:31.783 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-28T16:47:31.971 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-toml. 2024-03-28T16:47:31.999 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-28T16:47:32.038 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-28T16:47:32.092 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-28T16:47:32.340 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-pytest. 2024-03-28T16:47:32.343 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-28T16:47:32.368 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-28T16:47:32.415 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-28T16:47:32.452 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-28T16:47:32.579 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-28T16:47:32.668 INFO:teuthology.orchestra.run.smithi170.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-28T16:47:32.752 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package python3-simplejson. 2024-03-28T16:47:32.780 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-28T16:47:32.819 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-28T16:47:32.905 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-28T16:47:33.031 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-28T16:47:33.088 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-28T16:47:33.116 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-28T16:47:33.154 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-28T16:47:33.425 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-28T16:47:33.676 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-28T16:47:33.777 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-28T16:47:33.901 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package radosgw. 2024-03-28T16:47:33.928 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../116-radosgw_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:33.966 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking radosgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:34.011 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:47:34.137 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-28T16:47:34.447 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-28T16:47:34.748 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-28T16:47:34.770 INFO:teuthology.orchestra.run.smithi119.stdout:Selecting previously unselected package rbd-fuse. 2024-03-28T16:47:34.797 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1426-gacb3247c-1jammy_amd64.deb ... 2024-03-28T16:47:34.836 INFO:teuthology.orchestra.run.smithi119.stdout:Unpacking rbd-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:35.041 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-28T16:47:35.123 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-28T16:47:35.158 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-28T16:47:35.391 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-28T16:47:35.402 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-28T16:47:35.526 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-28T16:47:35.653 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-28T16:47:35.772 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-28T16:47:36.014 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-28T16:47:36.085 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-28T16:47:36.408 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-28T16:47:36.525 INFO:teuthology.orchestra.run.smithi119.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-28T16:47:36.603 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:36.685 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-28T16:47:37.012 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-28T16:47:37.155 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librados2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:37.163 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user cephadm....done 2024-03-28T16:47:37.281 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:37.284 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-28T16:47:37.407 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-28T16:47:37.543 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-28T16:47:37.718 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:37.778 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-28T16:47:37.827 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libradosstriper1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:37.904 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-28T16:47:37.937 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-28T16:47:38.163 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-28T16:47:38.222 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librbd1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:38.348 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-modules-core (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:38.423 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-28T16:47:38.474 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:38.549 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-28T16:47:38.678 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-28T16:47:38.678 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-28T16:47:38.834 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-28T16:47:39.175 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up libcephfs-dev (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:39.186 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-28T16:47:39.284 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rados (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:39.419 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up librgw2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:39.446 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-28T16:47:39.545 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-28T16:47:39.722 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-28T16:47:40.007 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-28T16:47:40.283 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-28T16:47:40.517 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-28T16:47:40.832 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rbd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:40.949 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up rbd-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:40.972 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-argparse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:41.050 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-rgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:41.152 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up python3-cephfs (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:41.286 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-28T16:47:41.287 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:41.412 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-28T16:47:41.538 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-28T16:47:41.681 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-28T16:47:41.788 INFO:teuthology.orchestra.run.smithi170.stdout:Adding group ceph....done 2024-03-28T16:47:41.949 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-28T16:47:42.164 INFO:teuthology.orchestra.run.smithi170.stdout:Adding system user ceph....done 2024-03-28T16:47:42.325 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-28T16:47:42.341 INFO:teuthology.orchestra.run.smithi170.stdout:Setting system user ceph properties....done 2024-03-28T16:47:42.356 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-28T16:47:42.357 INFO:teuthology.orchestra.run.smithi170.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-28T16:47:42.419 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-28T16:47:42.492 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-28T16:47:42.545 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-28T16:47:42.797 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-28T16:47:42.921 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-28T16:47:43.047 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-28T16:47:43.323 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-28T16:47:43.506 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-test (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:43.599 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-28T16:47:43.631 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up radosgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:43.917 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up zip (3.0-12build2) ... 2024-03-28T16:47:44.018 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-28T16:47:44.156 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-28T16:47:44.156 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-28T16:47:44.626 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-base (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:44.703 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-28T16:47:44.953 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-28T16:47:44.962 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-28T16:47:45.089 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-28T16:47:45.240 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-28T16:47:45.396 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mds (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:45.534 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-28T16:47:45.689 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-28T16:47:45.689 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-28T16:47:45.900 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-28T16:47:46.127 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-28T16:47:46.188 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:46.412 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-28T16:47:46.412 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-28T16:47:46.503 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-28T16:47:46.796 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-28T16:47:46.866 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-osd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:47.136 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-28T16:47:47.136 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-28T16:47:47.149 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-28T16:47:47.399 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-28T16:47:47.525 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:47.642 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:47.752 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:47.809 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-idna (3.3-1) ... 2024-03-28T16:47:47.911 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mon (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:48.102 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-28T16:47:48.122 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-28T16:47:48.122 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-28T16:47:48.749 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:48.983 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:49.159 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-mgr-dashboard (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:49.216 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-28T16:47:49.310 INFO:teuthology.orchestra.run.smithi170.stdout:Setting up ceph-volume (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:49.459 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:47:49.569 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-28T16:47:49.663 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-28T16:47:49.756 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-28T16:47:49.828 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-28T16:47:49.891 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-28T16:47:49.954 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-28T16:47:50.080 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-28T16:47:50.381 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-28T16:47:50.658 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-28T16:47:50.684 INFO:teuthology.orchestra.run.smithi170.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-28T16:47:50.900 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:47:51.001 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-28T16:47:51.294 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-28T16:47:51.347 INFO:teuthology.orchestra.run.smithi170.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-28T16:47:51.353 DEBUG:teuthology.parallel:result is None 2024-03-28T16:47:51.545 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-28T16:47:51.822 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-28T16:47:52.081 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-28T16:47:52.341 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-28T16:47:52.608 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-28T16:47:52.709 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-28T16:47:52.977 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-28T16:47:53.103 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-28T16:47:53.396 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-28T16:47:53.722 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-28T16:47:53.981 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-28T16:47:54.133 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-28T16:47:54.250 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-28T16:47:54.322 INFO:teuthology.orchestra.run.smithi119.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-28T16:47:54.543 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-28T16:47:54.661 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-28T16:47:55.038 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-28T16:47:55.297 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-28T16:47:55.423 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-28T16:47:55.674 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-28T16:47:55.775 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-28T16:47:56.068 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-28T16:47:56.344 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-28T16:47:56.603 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-28T16:47:56.729 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-28T16:47:56.997 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-28T16:47:57.274 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-28T16:47:57.610 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-28T16:47:58.003 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-28T16:47:58.288 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-28T16:47:58.625 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-28T16:47:58.726 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librados2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:58.835 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:58.961 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-28T16:47:59.297 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:59.423 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libradosstriper1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:59.574 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-28T16:47:59.842 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librbd1 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:47:59.969 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-modules-core (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:00.095 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:00.309 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-28T16:48:00.309 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-28T16:48:00.796 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up libcephfs-dev (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:00.922 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rados (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:01.048 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up librgw2 (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:01.166 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-28T16:48:02.353 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rbd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:02.479 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up rbd-fuse (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:02.605 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-rgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:02.730 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up python3-cephfs (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:02.857 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-common (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:03.374 INFO:teuthology.orchestra.run.smithi119.stdout:Adding group ceph....done 2024-03-28T16:48:03.743 INFO:teuthology.orchestra.run.smithi119.stdout:Adding system user ceph....done 2024-03-28T16:48:03.928 INFO:teuthology.orchestra.run.smithi119.stdout:Setting system user ceph properties....done 2024-03-28T16:48:03.940 INFO:teuthology.orchestra.run.smithi119.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-28T16:48:04.077 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-28T16:48:04.420 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-28T16:48:04.908 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-test (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:05.018 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up radosgw (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:05.486 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-28T16:48:05.486 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-28T16:48:05.936 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-base (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:06.273 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-28T16:48:06.771 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mds (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:06.982 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-28T16:48:06.983 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-28T16:48:07.499 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:07.710 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-28T16:48:07.711 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-28T16:48:08.167 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-osd (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:08.431 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-28T16:48:08.431 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-28T16:48:08.937 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:09.172 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:09.425 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mon (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:09.706 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-28T16:48:09.706 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-28T16:48:10.194 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-cephadm (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:10.353 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:10.479 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-mgr-dashboard (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:10.763 INFO:teuthology.orchestra.run.smithi119.stdout:Setting up ceph-volume (19.0.0-1426-gacb3247c-1jammy) ... 2024-03-28T16:48:11.116 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-28T16:48:11.209 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-28T16:48:11.536 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-28T16:48:12.486 INFO:teuthology.orchestra.run.smithi119.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-28T16:48:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-28T16:48:13.145 DEBUG:teuthology.parallel:result is None 2024-03-28T16:48:13.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=acb3247c84f48e9ce91359973bf22ebb44ec5034 2024-03-28T16:48:13.366 DEBUG:teuthology.orchestra.run.smithi119:> dpkg-query -W -f '${Version}' ceph 2024-03-28T16:48:13.385 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:48:13.385 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:48:13.385 INFO:teuthology.task.install:The correct ceph version 19.0.0-1426-gacb3247c-1jammy is installed. 2024-03-28T16:48:13.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=acb3247c84f48e9ce91359973bf22ebb44ec5034 2024-03-28T16:48:13.659 DEBUG:teuthology.orchestra.run.smithi170:> dpkg-query -W -f '${Version}' ceph 2024-03-28T16:48:13.678 INFO:teuthology.orchestra.run.smithi170.stdout:19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:48:13.679 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1426-gacb3247c-1jammy 2024-03-28T16:48:13.679 INFO:teuthology.task.install:The correct ceph version 19.0.0-1426-gacb3247c-1jammy is installed. 2024-03-28T16:48:13.680 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-28T16:48:13.680 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:13.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T16:48:13.695 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:13.695 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T16:48:13.735 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-28T16:48:13.735 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:13.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T16:48:13.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T16:48:13.805 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:13.806 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T16:48:13.819 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T16:48:13.873 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-28T16:48:13.874 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:13.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T16:48:13.886 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T16:48:13.938 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:13.939 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T16:48:13.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T16:48:14.006 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-28T16:48:14.007 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:14.007 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T16:48:14.019 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T16:48:14.073 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:14.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T16:48:14.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T16:48:14.141 INFO:teuthology.run_tasks:Running task ceph... 2024-03-28T16:48:14.228 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-28T16:48:14.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2024-03-28T16:48:14.230 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2024-03-28T16:48:14.243 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-28T16:48:14.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T16:48:14.284 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T16:48:14.297 INFO:tasks.ceph:Creating extra log directories... 2024-03-28T16:48:14.297 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T16:48:14.340 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T16:48:14.361 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-28T16:48:14.361 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-28T16:48:14.361 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381', 'branch': 'wip_thotz_update_attrs_from_state_setattrs', 'description': 'rgw:singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}}', 'email': None, 'first_in_suite': False, 'job_id': '7628381', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip_thotz_update_attrs_from_state_setattrs'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034'}}, 'rgw': {'ec-data-pool': False, 'frontend': 'beast'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip_thotz_update_attrs_from_state_setattrs', 'sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034'}}, 'owner': 'scheduled_soumyakoduri@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034', 'sleep_before_teardown': 0, 'suite': 'rgw:singleton', 'suite_branch': 'wip_thotz_update_attrs_from_state_setattrs', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_acb3247c84f48e9ce91359973bf22ebb44ec5034/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034', 'targets': {'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQ0qQqye1uKDXScKCoAVqSlWVI8KsfzUNX7YTdRQwYQ0cIEIhdtKHEz0DpNLvdjS7oZqx3yLvXv8aw5fbhdqxQ=', 'smithi170.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpRTtw3n0zBm75s+ZMdSyBbyD9vw++ZFvRFb9QaQy+3ImgeT9ikr+Xk+DubNhB4IL2pmsjpXnrF6tn4Bnu/bEc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'acb3247c84f48e9ce91359973bf22ebb44ec5034', 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-28_16:27:01', 'tube': 'smithi', 'user': 'soumyakoduri', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-28T16:48:14.361 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T16:48:14.396 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T16:48:14.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T16:48:14.444 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T16:48:14.458 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:14.458 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T16:48:14.503 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T16:48:14.503 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-03-28T16:48:14.551 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T16:48:14.551 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:14.552 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-03-28T16:48:14.552 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:14.552 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 16:42:41.210099178 +0000 2024-03-28T16:48:14.552 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 16:42:40.938101609 +0000 2024-03-28T16:48:14.552 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 16:42:40.938101609 +0000 2024-03-28T16:48:14.552 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-03-28T16:48:14.552 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T16:48:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T16:48:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T16:48:14.605 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000459867 s, 1.1 MB/s 2024-03-28T16:48:14.607 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T16:48:14.656 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-03-28T16:48:14.703 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T16:48:14.703 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:14.703 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-03-28T16:48:14.703 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:14.703 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 16:42:41.458096963 +0000 2024-03-28T16:48:14.703 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 16:42:41.454097000 +0000 2024-03-28T16:48:14.704 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 16:42:41.454097000 +0000 2024-03-28T16:48:14.704 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-03-28T16:48:14.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T16:48:14.758 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T16:48:14.758 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T16:48:14.758 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000349614 s, 1.5 MB/s 2024-03-28T16:48:14.760 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T16:48:14.809 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-03-28T16:48:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T16:48:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:14.859 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 919 Links: 1 2024-03-28T16:48:14.859 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:14.859 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 16:42:42.202090318 +0000 2024-03-28T16:48:14.859 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 16:42:41.914092891 +0000 2024-03-28T16:48:14.859 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 16:42:41.914092891 +0000 2024-03-28T16:48:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-03-28T16:48:14.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T16:48:14.914 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T16:48:14.914 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T16:48:14.914 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000370176 s, 1.4 MB/s 2024-03-28T16:48:14.915 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T16:48:14.964 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-03-28 16:42:42.742085496 +0000 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-03-28 16:42:42.466087961 +0000 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-03-28 16:42:42.466087961 +0000 2024-03-28T16:48:15.011 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2024-03-28T16:48:15.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T16:48:15.066 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-03-28T16:48:15.066 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-03-28T16:48:15.066 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000393282 s, 1.3 MB/s 2024-03-28T16:48:15.068 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T16:48:15.117 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-03-28T16:48:15.117 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:15.117 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T16:48:15.123 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T16:48:15.124 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-03-28T16:48:15.171 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T16:48:15.171 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 879 Links: 1 2024-03-28T16:48:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-28 16:42:41.734438015 +0000 2024-03-28T16:48:15.171 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-28 16:42:41.730438051 +0000 2024-03-28T16:48:15.172 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-28 16:42:41.730438051 +0000 2024-03-28T16:48:15.172 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-28T16:48:15.172 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T16:48:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-28T16:48:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-28T16:48:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000303103 s, 1.7 MB/s 2024-03-28T16:48:15.227 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T16:48:15.276 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-28 16:42:42.486431307 +0000 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-28 16:42:42.202433840 +0000 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-28 16:42:42.202433840 +0000 2024-03-28T16:48:15.323 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-28T16:48:15.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T16:48:15.377 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-28T16:48:15.377 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-28T16:48:15.377 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000250899 s, 2.0 MB/s 2024-03-28T16:48:15.378 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T16:48:15.428 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-28 16:42:42.698429416 +0000 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-28 16:42:42.694429452 +0000 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-28 16:42:42.694429452 +0000 2024-03-28T16:48:15.479 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-28T16:48:15.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T16:48:15.533 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-28T16:48:15.533 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-28T16:48:15.533 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000314034 s, 1.6 MB/s 2024-03-28T16:48:15.534 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T16:48:15.584 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-28 16:42:43.450422708 +0000 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-28 16:42:43.162425277 +0000 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-28 16:42:43.162425277 +0000 2024-03-28T16:48:15.631 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-28T16:48:15.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T16:48:15.684 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-28T16:48:15.684 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-28T16:48:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000202892 s, 2.5 MB/s 2024-03-28T16:48:15.686 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T16:48:15.736 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-03-28T16:48:15.736 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-03-28T16:48:15.736 INFO:tasks.ceph:Generating config... 2024-03-28T16:48:15.737 INFO:tasks.ceph:[client] debug ms = 1 2024-03-28T16:48:15.738 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-03-28T16:48:15.738 INFO:tasks.ceph:[client] debug rgw = 20 2024-03-28T16:48:15.738 INFO:tasks.ceph:[client] setgroup = ceph 2024-03-28T16:48:15.738 INFO:tasks.ceph:[client] setuser = ceph 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] debug objclass = 20 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-28T16:48:15.738 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-28T16:48:15.739 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-28T16:48:15.739 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-28T16:48:15.739 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-28T16:48:15.739 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-28T16:48:15.739 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-28T16:48:15.739 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-28T16:48:15.739 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-28T16:48:15.739 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-28T16:48:15.739 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-28T16:48:15.739 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-28T16:48:15.739 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-28T16:48:15.739 INFO:tasks.ceph:Setting up mon.a... 2024-03-28T16:48:15.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-28T16:48:15.773 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.keyring 2024-03-28T16:48:15.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-28T16:48:15.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T16:48:15.922 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.119')] 2024-03-28T16:48:15.922 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.119'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-03-28T16:48:15.923 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:15.923 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T16:48:15.971 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.119 --print /home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:16.050 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:48:16.050 INFO:teuthology.orchestra.run.smithi119.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:48:16.050 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: generated fsid 5a56263c-53d8-4d15-86cb-f8a4a2a85230 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = quincy 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 0 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:fsid 5a56263c-53d8-4d15-86cb-f8a4a2a85230 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:last_changed 2024-03-28T16:48:16.053162+0000 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:created 2024-03-28T16:48:16.053162+0000 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:min_mon_release 17 (quincy) 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:election_strategy: 1 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.a 2024-03-28T16:48:16.051 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-03-28T16:48:16.052 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T16:48:16.100 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5a56263c-53d8-4d15-86cb-f8a4a2a85230... 2024-03-28T16:48:16.100 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T16:48:16.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T16:48:16.172 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2024-03-28T16:48:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2024-03-28T16:48:16.172 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.173 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2024-03-28T16:48:16.174 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.175 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = 172.21.15.119 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 5a56263c-53d8-4d15-86cb-f8a4a2a85230 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.176 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2024-03-28T16:48:16.177 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2024-03-28T16:48:16.178 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: debug objclass = 20 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2024-03-28T16:48:16.179 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-03-28T16:48:16.180 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2024-03-28T16:48:16.181 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2024-03-28T16:48:16.182 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: rgw gc obj min wait = 15 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: debug rgw = 20 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: setgroup = ceph 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout: setuser = ceph 2024-03-28T16:48:16.183 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2024-03-28T16:48:16.184 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2024-03-28T16:48:16.184 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2024-03-28T16:48:16.184 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T16:48:16.184 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2024-03-28T16:48:16.184 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.184 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T16:48:16.185 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.186 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = 172.21.15.119 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = 5a56263c-53d8-4d15-86cb-f8a4a2a85230 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2024-03-28T16:48:16.187 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock profile = high_recovery_ops 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2024-03-28T16:48:16.188 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug trim objects = true 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2024-03-28T16:48:16.189 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: debug objclass = 20 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: bdev async discard = True 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: bdev enable discard = True 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore allocator = bitmap 2024-03-28T16:48:16.190 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 1/20 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 1/20 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 4/10 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2024-03-28T16:48:16.191 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2024-03-28T16:48:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2024-03-28T16:48:16.193 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: rgw gc obj min wait = 15 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: debug rgw = 20 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: setgroup = ceph 2024-03-28T16:48:16.194 INFO:teuthology.orchestra.run.smithi170.stdout: setuser = ceph 2024-03-28T16:48:16.195 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2024-03-28T16:48:16.196 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-28T16:48:16.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-28T16:48:16.291 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-28T16:48:16.291 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:16.291 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-28T16:48:16.296 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:16.297 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-28T16:48:16.343 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T16:48:16.343 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:16.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T16:48:16.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T16:48:16.407 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:16.407 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:16.455 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:48:16.456 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:16.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T16:48:16.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T16:48:16.478 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:16.478 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:16.527 INFO:tasks.ceph:Setting up mon nodes... 2024-03-28T16:48:16.527 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-28T16:48:16.527 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T16:48:16.611 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T16:48:16.627 INFO:tasks.ceph:Setting up mds nodes... 2024-03-28T16:48:16.627 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-28T16:48:16.627 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-28T16:48:16.701 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-28T16:48:16.710 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-28T16:48:16.710 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-03-28T16:48:16.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-28T16:48:16.724 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-03-28T16:48:16.724 INFO:tasks.ceph:role: osd.0 2024-03-28T16:48:16.724 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T16:48:16.725 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T16:48:16.787 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T16:48:16.787 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T16:48:16.788 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T16:48:17.239 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2024-03-28T16:48:17.241 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2024-03-28T16:48:17.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-28T16:48:17.342 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-28T16:48:17.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:48:17.353 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: /sbin/restorecon: command not found 2024-03-28T16:48:17.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:17.475 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:17.474+0000 7f4d14fa17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-28T16:48:17.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:17.478+0000 7f4d14fa17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-28T16:48:17.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:17.478+0000 7f4d14fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T16:48:17.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:17.478+0000 7f4d14fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T16:48:17.480 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:17.478+0000 7f4d14fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T16:48:17.481 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:17.478+0000 7f4d14fa17c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-28T16:48:20.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-28T16:48:20.474 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-28T16:48:20.488 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-03-28T16:48:20.488 INFO:tasks.ceph:role: osd.1 2024-03-28T16:48:20.488 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:48:20.488 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-28T16:48:20.543 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T16:48:20.544 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T16:48:20.567 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-03-28T16:48:20.569 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-03-28T16:48:20.569 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-03-28T16:48:20.630 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-28T16:48:20.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:48:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-03-28T16:48:20.685 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-28T16:48:20.737 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-03-28T16:48:20.737 INFO:tasks.ceph:role: osd.2 2024-03-28T16:48:20.737 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:48:20.737 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-28T16:48:20.790 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T16:48:20.790 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T16:48:20.790 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T16:48:20.790 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-28T16:48:20.791 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T16:48:20.791 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-03-28T16:48:20.791 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T16:48:20.791 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-28T16:48:20.791 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T16:48:20.791 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T16:48:20.814 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-03-28T16:48:20.816 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-03-28T16:48:20.816 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-03-28T16:48:20.877 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-28T16:48:20.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:48:20.932 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-03-28T16:48:20.932 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-28T16:48:20.986 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-03-28T16:48:20.986 INFO:tasks.ceph:role: osd.3 2024-03-28T16:48:20.986 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:48:20.986 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-28T16:48:21.042 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T16:48:21.042 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T16:48:21.043 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T16:48:21.065 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-03-28T16:48:21.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-03-28T16:48:21.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-03-28T16:48:21.125 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-28T16:48:21.180 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: /sbin/restorecon: command not found 2024-03-28T16:48:21.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:48:21.181 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:21.303+0000 7fdaff32e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-28T16:48:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:21.303+0000 7fdaff32e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-28T16:48:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:21.303+0000 7fdaff32e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T16:48:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:21.303+0000 7fdaff32e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T16:48:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:21.303+0000 7fdaff32e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T16:48:21.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:21.303+0000 7fdaff32e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-28T16:48:24.283 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-28T16:48:24.296 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:24.379+0000 7ff9372d37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-28T16:48:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:24.379+0000 7ff9372d37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-28T16:48:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:24.379+0000 7ff9372d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T16:48:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:24.379+0000 7ff9372d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T16:48:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:24.379+0000 7ff9372d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T16:48:24.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:24.379+0000 7ff9372d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-28T16:48:27.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-28T16:48:27.398 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:27.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:27.479+0000 7fced736b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-28T16:48:27.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:27.479+0000 7fced736b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-28T16:48:27.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:27.479+0000 7fced736b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T16:48:27.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:27.479+0000 7fced736b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T16:48:27.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:27.479+0000 7fced736b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T16:48:27.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:27.479+0000 7fced736b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-28T16:48:30.453 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-28T16:48:30.507 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-28T16:48:30.507 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:30.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-28T16:48:30.522 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:30.522 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-28T16:48:30.562 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:30.562 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-28T16:48:30.618 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:30.618 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-28T16:48:30.674 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:30.674 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-28T16:48:30.729 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:30.730 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-28T16:48:30.778 INFO:tasks.ceph:Adding keys to all mons... 2024-03-28T16:48:30.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBRnwVmR4KCHBAAw6D5qlIhf0vInN6gucsc9Q== 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBQnwVmeoWTJBAAEGAi+m8X1WHkvDJXqzhIFQ== 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBVnwVmPDhSEhAAEDCFA++eQU3QqlsI0DHHuA== 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBYnwVmwUDXFhAARSW0ddMuZIU1XIU8dcmrBg== 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2024-03-28T16:48:30.791 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBbnwVmvDDUHBAA3xuppcBJbEFwu+muD+3rxA== 2024-03-28T16:48:30.792 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2024-03-28T16:48:30.792 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBQnwVm2y7mKRAAmvCb5FYqmW000UvH1DdSYw== 2024-03-28T16:48:30.792 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T16:48:30.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T16:48:30.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T16:48:31.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T16:48:31.128 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T16:48:31.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T16:48:31.292 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-28T16:48:31.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-28T16:48:31.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T16:48:31.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-28T16:48:31.448 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-28T16:48:31.498 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-28T16:48:31.498 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-28T16:48:31.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-28T16:48:31.544 INFO:tasks.ceph.mon.a:Started 2024-03-28T16:48:31.544 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-28T16:48:31.544 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-28T16:48:31.544 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-28T16:48:31.547 INFO:tasks.ceph.mgr.x:Started 2024-03-28T16:48:31.547 INFO:tasks.ceph:Setting crush tunables to default 2024-03-28T16:48:31.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd crush tunables default 2024-03-28T16:48:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff4221b640 1 Processor -- start 2024-03-28T16:48:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff4221b640 1 -- start start 2024-03-28T16:48:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff4221b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 0x7eff3c133580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff4221b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff3c073250 con 0x7eff3c098910 2024-03-28T16:48:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff4221b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c096da0 con 0x7eff3c099400 2024-03-28T16:48:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff3affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 0x7eff3c133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:31.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff3affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 0x7eff3c133580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57132/0 (socket says 172.21.15.119:57132) 2024-03-28T16:48:31.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff3affd640 1 -- 172.21.15.119:0/1594918070 learned_addr learned my addr 172.21.15.119:0/1594918070 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:31.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff3affd640 1 -- 172.21.15.119:0/1594918070 >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 msgr2=0x7eff3c133580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-28T16:48:31.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff3affd640 1 -- 172.21.15.119:0/1594918070 >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 msgr2=0x7eff3c133580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-28T16:48:31.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff3affd640 1 --2- 172.21.15.119:0/1594918070 >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 0x7eff3c133580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T16:48:31.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.662+0000 7eff3affd640 1 --2- 172.21.15.119:0/1594918070 >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 0x7eff3c133580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017782414 0 0) 0x7eff3c073250 con 0x7eff3c098910 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff24003530 con 0x7eff3c098910 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666264914 0 0) 0x7eff24003530 con 0x7eff3c098910 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 msgr2=0x7eff3c133580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 --2- 172.21.15.119:0/1594918070 >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 0x7eff3c133580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c099ee0 con 0x7eff3c098910 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 2481508143 0 0) 0x7eff2c0015b0 con 0x7eff3c098910 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 >> v1:172.21.15.119:6789/0 conn(0x7eff3c098910 legacy=0x7eff3c098cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 --2- 172.21.15.119:0/1594918070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff24003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:31.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c096da0 con 0x7eff240039b0 2024-03-28T16:48:31.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3b7fe640 1 --2- 172.21.15.119:0/1594918070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff24003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:31.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.682+0000 7eff3b7fe640 1 -- 172.21.15.119:0/1594918070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24003530 con 0x7eff240039b0 2024-03-28T16:48:31.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3b7fe640 1 --2- 172.21.15.119:0/1594918070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff24003db0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007ec0 tx=0x7eff2c01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cb5922136573ea server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:31.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff2c0a5990 con 0x7eff240039b0 2024-03-28T16:48:31.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c0a5b10 con 0x7eff240039b0 2024-03-28T16:48:31.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3a7fc640 1 -- 172.21.15.119:0/1594918070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff2c0a5df0 con 0x7eff240039b0 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- 172.21.15.119:0/1594918070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 msgr2=0x7eff24003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 --2- 172.21.15.119:0/1594918070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff24003db0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007ec0 tx=0x7eff2c01cb60 comp rx=0 tx=0).stop 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- 172.21.15.119:0/1594918070 shutdown_connections 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 --2- 172.21.15.119:0/1594918070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff24003db0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 --2- 172.21.15.119:0/1594918070 >> v2:172.21.15.119:3300/0 conn(0x7eff3c099400 0x7eff3c133580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- 172.21.15.119:0/1594918070 >> 172.21.15.119:0/1594918070 conn(0x7eff3c09e900 msgr2=0x7eff3c09ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- 172.21.15.119:0/1594918070 shutdown_connections 2024-03-28T16:48:31.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- 172.21.15.119:0/1594918070 wait complete. 2024-03-28T16:48:31.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 Processor -- start 2024-03-28T16:48:31.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- start start 2024-03-28T16:48:31.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff3c092f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:31.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c096340 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff3c092f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff3c092f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57162/0 (socket says 172.21.15.119:57162) 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3b7fe640 1 -- 172.21.15.119:0/3914756883 learned_addr learned my addr 172.21.15.119:0/3914756883 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3b7fe640 1 -- 172.21.15.119:0/3914756883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c071ed0 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff3b7fe640 1 --2- 172.21.15.119:0/3914756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff3c092f60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7eff2c01cba0 tx=0x7eff24005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff23fff640 1 -- 172.21.15.119:0/3914756883 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff2c001440 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff23fff640 1 -- 172.21.15.119:0/3914756883 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c0015a0 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff23fff640 1 -- 172.21.15.119:0/3914756883 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff2c003750 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c095570 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c1255d0 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff23fff640 1 -- 172.21.15.119:0/3914756883 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7eff2c0b4040 con 0x7eff240039b0 2024-03-28T16:48:31.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.710+0000 7eff23fff640 1 -- 172.21.15.119:0/3914756883 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff2c0ad440 con 0x7eff240039b0 2024-03-28T16:48:31.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.714+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c0961a0 con 0x7eff240039b0 2024-03-28T16:48:31.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.714+0000 7eff23fff640 1 -- 172.21.15.119:0/3914756883 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7eff3c0961a0 con 0x7eff240039b0 2024-03-28T16:48:31.779 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7eff3c062860 con 0x7eff240039b0 2024-03-28T16:48:31.781 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff23fff640 1 -- 172.21.15.119:0/3914756883 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7eff2c0ba070 con 0x7eff240039b0 2024-03-28T16:48:31.781 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2024-03-28T16:48:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 msgr2=0x7eff3c092f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 --2- 172.21.15.119:0/3914756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff3c092f60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7eff2c01cba0 tx=0x7eff24005800 comp rx=0 tx=0).stop 2024-03-28T16:48:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 shutdown_connections 2024-03-28T16:48:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 --2- 172.21.15.119:0/3914756883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff240039b0 0x7eff3c092f60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 >> 172.21.15.119:0/3914756883 conn(0x7eff3c09e900 msgr2=0x7eff3c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 shutdown_connections 2024-03-28T16:48:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:31.778+0000 7eff4221b640 1 -- 172.21.15.119:0/3914756883 wait complete. 2024-03-28T16:48:31.798 INFO:tasks.ceph:check_enable_crimson: False 2024-03-28T16:48:31.798 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-28T16:48:31.799 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-03-28T16:48:31.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-28T16:48:31.811 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:31.811 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-28T16:48:31.826 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:31.826 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-28T16:48:31.882 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-28T16:48:31.882 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-28T16:48:31.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 75d33f3f-95d7-438e-8617-4161e6766019 0 2024-03-28T16:48:31.980 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:31.979+0000 7fa011d8e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3f7b4640 1 Processor -- start 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3f7b4640 1 -- start start 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3f7b4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e3806b7e0 0x7f9e3806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3f7b4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3811cd20 con 0x7f9e3806c100 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3f7b4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3811dee0 con 0x7f9e3806b7e0 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3d550640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e3806b7e0 0x7f9e3806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3cd4f640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9e3806c100 0x7f9e3811b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:52690/0 (socket says 172.21.15.170:52690) 2024-03-28T16:48:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3cd4f640 1 -- 172.21.15.170:0/4183419803 learned_addr learned my addr 172.21.15.170:0/4183419803 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:48:32.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3d550640 1 -- 172.21.15.170:0/4183419803 >> v1:172.21.15.119:6789/0 conn(0x7f9e3806c100 legacy=0x7f9e3811b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.099+0000 7f9e3d550640 1 -- 172.21.15.170:0/4183419803 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3811e050 con 0x7f9e3806b7e0 2024-03-28T16:48:32.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3d550640 1 --2- 172.21.15.170:0/4183419803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e3806bbc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009d90 tx=0x7f9e3001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819c3e6aeb1cdac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e2ffff640 1 -- 172.21.15.170:0/4183419803 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e300a58e0 con 0x7f9e3806b7e0 2024-03-28T16:48:32.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e2ffff640 1 -- 172.21.15.170:0/4183419803 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e300a5a40 con 0x7f9e3806b7e0 2024-03-28T16:48:32.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e2ffff640 1 -- 172.21.15.170:0/4183419803 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e300a5d00 con 0x7f9e3806b7e0 2024-03-28T16:48:32.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/4183419803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 msgr2=0x7f9e3806bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3f7b4640 1 --2- 172.21.15.170:0/4183419803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e3806bbc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009d90 tx=0x7f9e3001c950 comp rx=0 tx=0).stop 2024-03-28T16:48:32.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/4183419803 shutdown_connections 2024-03-28T16:48:32.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3f7b4640 1 --2- 172.21.15.170:0/4183419803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e3806bbc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/4183419803 >> 172.21.15.170:0/4183419803 conn(0x7f9e3806f540 msgr2=0x7f9e38071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/4183419803 shutdown_connections 2024-03-28T16:48:32.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.103+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/4183419803 wait complete. 2024-03-28T16:48:32.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3f7b4640 1 Processor -- start 2024-03-28T16:48:32.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3f7b4640 1 -- start start 2024-03-28T16:48:32.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3f7b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e381a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3f7b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e381a8d20 con 0x7f9e3806b7e0 2024-03-28T16:48:32.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3d550640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e381a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3d550640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e381a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:41810/0 (socket says 172.21.15.170:41810) 2024-03-28T16:48:32.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3d550640 1 -- 172.21.15.170:0/1742122258 learned_addr learned my addr 172.21.15.170:0/1742122258 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:32.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3d550640 1 -- 172.21.15.170:0/1742122258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30009a70 con 0x7f9e3806b7e0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3d550640 1 --2- 172.21.15.170:0/1742122258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e381a87e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e3001a580 tx=0x7f9e300196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e2dffb640 1 -- 172.21.15.170:0/1742122258 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e300addb0 con 0x7f9e3806b7e0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e2dffb640 1 -- 172.21.15.170:0/1742122258 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e300b7410 con 0x7f9e3806b7e0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e2dffb640 1 -- 172.21.15.170:0/1742122258 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e300b7710 con 0x7f9e3806b7e0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e381a8f20 con 0x7f9e3806b7e0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e381ad0b0 con 0x7f9e3806b7e0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e2dffb640 1 -- 172.21.15.170:0/1742122258 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9e300b4050 con 0x7f9e3806b7e0 2024-03-28T16:48:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e2dffb640 1 -- 172.21.15.170:0/1742122258 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9e300c2cc0 con 0x7f9e3806b7e0 2024-03-28T16:48:32.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.107+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e38062860 con 0x7f9e3806b7e0 2024-03-28T16:48:32.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.119+0000 7f9e2dffb640 1 -- 172.21.15.170:0/1742122258 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9e300c7060 con 0x7f9e3806b7e0 2024-03-28T16:48:32.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.187+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "75d33f3f-95d7-438e-8617-4161e6766019", "id": 0} v 0) v1 -- 0x7f9e38067620 con 0x7f9e3806b7e0 2024-03-28T16:48:32.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e2dffb640 1 -- 172.21.15.170:0/1742122258 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "75d33f3f-95d7-438e-8617-4161e6766019", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9e300bd070 con 0x7f9e3806b7e0 2024-03-28T16:48:32.191 INFO:teuthology.orchestra.run.smithi170.stdout:0 2024-03-28T16:48:32.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 msgr2=0x7f9e381a87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e3f7b4640 1 --2- 172.21.15.170:0/1742122258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e381a87e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e3001a580 tx=0x7f9e300196f0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 shutdown_connections 2024-03-28T16:48:32.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e3f7b4640 1 --2- 172.21.15.170:0/1742122258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e3806b7e0 0x7f9e381a87e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 >> 172.21.15.170:0/1742122258 conn(0x7f9e3806f540 msgr2=0x7f9e38118cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 shutdown_connections 2024-03-28T16:48:32.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.191+0000 7f9e3f7b4640 1 -- 172.21.15.170:0/1742122258 wait complete. 2024-03-28T16:48:32.208 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 29d55d30-92be-4309-a398-2412a4b9674f 1 2024-03-28T16:48:32.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23dc983640 1 Processor -- start 2024-03-28T16:48:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23dc983640 1 -- start start 2024-03-28T16:48:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23dc983640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23d811a350 0x7f23d811c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23dc983640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d811cc80 con 0x7f23d8116500 2024-03-28T16:48:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23dc983640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d811de20 con 0x7f23d811a350 2024-03-28T16:48:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23d5d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23d811a350 0x7f23d811c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23d659c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f23d8116500 0x7f23d81168e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:52704/0 (socket says 172.21.15.170:52704) 2024-03-28T16:48:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23d659c640 1 -- 172.21.15.170:0/2284783277 learned_addr learned my addr 172.21.15.170:0/2284783277 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:48:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23d5d9b640 1 -- 172.21.15.170:0/2284783277 >> v1:172.21.15.119:6789/0 conn(0x7f23d8116500 legacy=0x7f23d81168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.323+0000 7f23d5d9b640 1 -- 172.21.15.170:0/2284783277 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d811df60 con 0x7f23d811a350 2024-03-28T16:48:32.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23d5d9b640 1 --2- 172.21.15.170:0/2284783277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d811a350 0x7f23d811c740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009740 tx=0x7f23c801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59f1a0aea54f643 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23d559a640 1 -- 172.21.15.170:0/2284783277 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23c80a8050 con 0x7f23d811a350 2024-03-28T16:48:32.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23d559a640 1 -- 172.21.15.170:0/2284783277 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c80ac520 con 0x7f23d811a350 2024-03-28T16:48:32.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23d559a640 1 -- 172.21.15.170:0/2284783277 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23c80ac7e0 con 0x7f23d811a350 2024-03-28T16:48:32.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23dc983640 1 -- 172.21.15.170:0/2284783277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d811a350 msgr2=0x7f23d811c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23dc983640 1 --2- 172.21.15.170:0/2284783277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d811a350 0x7f23d811c740 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009740 tx=0x7f23c801c370 comp rx=0 tx=0).stop 2024-03-28T16:48:32.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23dc983640 1 -- 172.21.15.170:0/2284783277 shutdown_connections 2024-03-28T16:48:32.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23dc983640 1 --2- 172.21.15.170:0/2284783277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d811a350 0x7f23d811c740 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23dc983640 1 -- 172.21.15.170:0/2284783277 >> 172.21.15.170:0/2284783277 conn(0x7f23d806dba0 msgr2=0x7f23d806dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23dc983640 1 -- 172.21.15.170:0/2284783277 shutdown_connections 2024-03-28T16:48:32.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.327+0000 7f23dc983640 1 -- 172.21.15.170:0/2284783277 wait complete. 2024-03-28T16:48:32.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23dc983640 1 Processor -- start 2024-03-28T16:48:32.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23dc983640 1 -- start start 2024-03-28T16:48:32.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23dc983640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d8116500 0x7f23d81ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23dc983640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81aea70 con 0x7f23d8116500 2024-03-28T16:48:32.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23d659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d8116500 0x7f23d81ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23d659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d8116500 0x7f23d81ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:41838/0 (socket says 172.21.15.170:41838) 2024-03-28T16:48:32.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23d659c640 1 -- 172.21.15.170:0/916163452 learned_addr learned my addr 172.21.15.170:0/916163452 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:32.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23d659c640 1 -- 172.21.15.170:0/916163452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8009420 con 0x7f23d8116500 2024-03-28T16:48:32.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23d659c640 1 --2- 172.21.15.170:0/916163452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d8116500 0x7f23d81ae4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23c000c3a0 tx=0x7f23c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23ceffd640 1 -- 172.21.15.170:0/916163452 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23c0005c20 con 0x7f23d8116500 2024-03-28T16:48:32.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23ceffd640 1 -- 172.21.15.170:0/916163452 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0005d80 con 0x7f23d8116500 2024-03-28T16:48:32.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23ceffd640 1 -- 172.21.15.170:0/916163452 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23c0007200 con 0x7f23d8116500 2024-03-28T16:48:32.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d81aecd0 con 0x7f23d8116500 2024-03-28T16:48:32.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d81a83e0 con 0x7f23d8116500 2024-03-28T16:48:32.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23ceffd640 1 -- 172.21.15.170:0/916163452 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f23c0007360 con 0x7f23d8116500 2024-03-28T16:48:32.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23ceffd640 1 -- 172.21.15.170:0/916163452 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f23c00127b0 con 0x7f23d8116500 2024-03-28T16:48:32.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.331+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d8062860 con 0x7f23d8116500 2024-03-28T16:48:32.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.339+0000 7f23ceffd640 1 -- 172.21.15.170:0/916163452 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f23c0021060 con 0x7f23d8116500 2024-03-28T16:48:32.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.407+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "29d55d30-92be-4309-a398-2412a4b9674f", "id": 1} v 0) v1 -- 0x7f23d8067620 con 0x7f23d8116500 2024-03-28T16:48:32.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.407+0000 7f23ceffd640 1 -- 172.21.15.170:0/916163452 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "29d55d30-92be-4309-a398-2412a4b9674f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f23c0017070 con 0x7f23d8116500 2024-03-28T16:48:32.410 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-03-28T16:48:32.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.411+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d8116500 msgr2=0x7f23d81ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.411+0000 7f23dc983640 1 --2- 172.21.15.170:0/916163452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d8116500 0x7f23d81ae4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23c000c3a0 tx=0x7f23c000c860 comp rx=0 tx=0).stop 2024-03-28T16:48:32.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.411+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 shutdown_connections 2024-03-28T16:48:32.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.411+0000 7f23dc983640 1 --2- 172.21.15.170:0/916163452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23d8116500 0x7f23d81ae4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.411+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 >> 172.21.15.170:0/916163452 conn(0x7f23d806dba0 msgr2=0x7f23d8118f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.411+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 shutdown_connections 2024-03-28T16:48:32.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.411+0000 7f23dc983640 1 -- 172.21.15.170:0/916163452 wait complete. 2024-03-28T16:48:32.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 60322449-dbf6-4117-9bc4-10ac42982aad 2 2024-03-28T16:48:32.527 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:32.527+0000 7fa011d8e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T16:48:32.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d986d640 1 Processor -- start 2024-03-28T16:48:32.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d986d640 1 -- start start 2024-03-28T16:48:32.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d986d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2d4118160 0x7fc2d411a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d986d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d411ab20 con 0x7fc2d41153c0 2024-03-28T16:48:32.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d986d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d411bcc0 con 0x7fc2d4118160 2024-03-28T16:48:32.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d27fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2d4118160 0x7fc2d411a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d2ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc2d41153c0 0x7fc2d41157a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:52718/0 (socket says 172.21.15.170:52718) 2024-03-28T16:48:32.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d27fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2d4118160 0x7fc2d411a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:41854/0 (socket says 172.21.15.170:41854) 2024-03-28T16:48:32.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d2ffd640 1 -- 172.21.15.170:0/1700813133 learned_addr learned my addr 172.21.15.170:0/1700813133 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:48:32.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d27fc640 1 -- 172.21.15.170:0/1700813133 >> v1:172.21.15.119:6789/0 conn(0x7fc2d41153c0 legacy=0x7fc2d41157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.535+0000 7fc2d27fc640 1 -- 172.21.15.170:0/1700813133 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d411be00 con 0x7fc2d4118160 2024-03-28T16:48:32.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d27fc640 1 --2- 172.21.15.170:0/1700813133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d4118160 0x7fc2d411a550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009740 tx=0x7fc2bc01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb568353cc39742b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d1ffb640 1 -- 172.21.15.170:0/1700813133 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2bc0a8050 con 0x7fc2d4118160 2024-03-28T16:48:32.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d1ffb640 1 -- 172.21.15.170:0/1700813133 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc0ac520 con 0x7fc2d4118160 2024-03-28T16:48:32.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d1ffb640 1 -- 172.21.15.170:0/1700813133 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2bc0ac7e0 con 0x7fc2d4118160 2024-03-28T16:48:32.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d986d640 1 -- 172.21.15.170:0/1700813133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d4118160 msgr2=0x7fc2d411a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d986d640 1 --2- 172.21.15.170:0/1700813133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d4118160 0x7fc2d411a550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009740 tx=0x7fc2bc01c370 comp rx=0 tx=0).stop 2024-03-28T16:48:32.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d986d640 1 -- 172.21.15.170:0/1700813133 shutdown_connections 2024-03-28T16:48:32.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d986d640 1 --2- 172.21.15.170:0/1700813133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d4118160 0x7fc2d411a550 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d986d640 1 -- 172.21.15.170:0/1700813133 >> 172.21.15.170:0/1700813133 conn(0x7fc2d40702e0 msgr2=0x7fc2d4072700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d986d640 1 -- 172.21.15.170:0/1700813133 shutdown_connections 2024-03-28T16:48:32.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.539+0000 7fc2d986d640 1 -- 172.21.15.170:0/1700813133 wait complete. 2024-03-28T16:48:32.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d986d640 1 Processor -- start 2024-03-28T16:48:32.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d986d640 1 -- start start 2024-03-28T16:48:32.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d986d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d41153c0 0x7fc2d406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d986d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d406cd20 con 0x7fc2d41153c0 2024-03-28T16:48:32.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d41153c0 0x7fc2d406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d41153c0 0x7fc2d406c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:41870/0 (socket says 172.21.15.170:41870) 2024-03-28T16:48:32.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d2ffd640 1 -- 172.21.15.170:0/2190280999 learned_addr learned my addr 172.21.15.170:0/2190280999 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:32.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d2ffd640 1 -- 172.21.15.170:0/2190280999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc009420 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d2ffd640 1 --2- 172.21.15.170:0/2190280999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d41153c0 0x7fc2d406c750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400c3a0 tx=0x7fc2c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2cbfff640 1 -- 172.21.15.170:0/2190280999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2c4005c20 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d411bb30 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2cbfff640 1 -- 172.21.15.170:0/2190280999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c4005d80 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2cbfff640 1 -- 172.21.15.170:0/2190280999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2c4007200 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d406d290 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2cbfff640 1 -- 172.21.15.170:0/2190280999 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc2c4007360 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2cbfff640 1 -- 172.21.15.170:0/2190280999 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fc2c4012820 con 0x7fc2d41153c0 2024-03-28T16:48:32.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.543+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d406d590 con 0x7fc2d41153c0 2024-03-28T16:48:32.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.551+0000 7fc2cbfff640 1 -- 172.21.15.170:0/2190280999 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc2c4021060 con 0x7fc2d41153c0 2024-03-28T16:48:32.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.619+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "60322449-dbf6-4117-9bc4-10ac42982aad", "id": 2} v 0) v1 -- 0x7fc2d4067620 con 0x7fc2d41153c0 2024-03-28T16:48:32.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.619+0000 7fc2cbfff640 1 -- 172.21.15.170:0/2190280999 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "60322449-dbf6-4117-9bc4-10ac42982aad", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc2c402a0b0 con 0x7fc2d41153c0 2024-03-28T16:48:32.621 INFO:teuthology.orchestra.run.smithi170.stdout:2 2024-03-28T16:48:32.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.623+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d41153c0 msgr2=0x7fc2d406c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.623+0000 7fc2d986d640 1 --2- 172.21.15.170:0/2190280999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d41153c0 0x7fc2d406c750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400c3a0 tx=0x7fc2c400c860 comp rx=0 tx=0).stop 2024-03-28T16:48:32.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.623+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 shutdown_connections 2024-03-28T16:48:32.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.623+0000 7fc2d986d640 1 --2- 172.21.15.170:0/2190280999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d41153c0 0x7fc2d406c750 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.623+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 >> 172.21.15.170:0/2190280999 conn(0x7fc2d40702e0 msgr2=0x7fc2d4119b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.623+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 shutdown_connections 2024-03-28T16:48:32.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.623+0000 7fc2d986d640 1 -- 172.21.15.170:0/2190280999 wait complete. 2024-03-28T16:48:32.639 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new df1546de-4221-40e6-b2b6-cdc2d7f86a3c 3 2024-03-28T16:48:32.740 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:32.739+0000 7fa011d8e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T16:48:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e8b60640 1 Processor -- start 2024-03-28T16:48:32.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e8b60640 1 -- start start 2024-03-28T16:48:32.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e8b60640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f83e41180f0 0x7f83e411a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e8b60640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e411aab0 con 0x7f83e4115460 2024-03-28T16:48:32.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e8b60640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e411bc50 con 0x7f83e41180f0 2024-03-28T16:48:32.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e1d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f83e41180f0 0x7f83e411a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f83e4115460 0x7f83e4115840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:52732/0 (socket says 172.21.15.170:52732) 2024-03-28T16:48:32.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.751+0000 7f83e259c640 1 -- 172.21.15.170:0/2016274902 learned_addr learned my addr 172.21.15.170:0/2016274902 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:48:32.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.755+0000 7f83e1d9b640 1 -- 172.21.15.170:0/2016274902 >> v1:172.21.15.119:6789/0 conn(0x7f83e4115460 legacy=0x7f83e4115840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.755+0000 7f83e1d9b640 1 -- 172.21.15.170:0/2016274902 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e411bdc0 con 0x7f83e41180f0 2024-03-28T16:48:32.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.755+0000 7f83e1d9b640 1 --2- 172.21.15.170:0/2016274902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e41180f0 0x7f83e411a4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009740 tx=0x7f83d801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833f74c9e91d30ea server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.755+0000 7f83e159a640 1 -- 172.21.15.170:0/2016274902 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83d80a8050 con 0x7f83e41180f0 2024-03-28T16:48:32.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.755+0000 7f83e159a640 1 -- 172.21.15.170:0/2016274902 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80ac520 con 0x7f83e41180f0 2024-03-28T16:48:32.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.755+0000 7f83e159a640 1 -- 172.21.15.170:0/2016274902 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83d80ac7e0 con 0x7f83e41180f0 2024-03-28T16:48:32.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/2016274902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e41180f0 msgr2=0x7f83e411a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 --2- 172.21.15.170:0/2016274902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e41180f0 0x7f83e411a4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009740 tx=0x7f83d801c370 comp rx=0 tx=0).stop 2024-03-28T16:48:32.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/2016274902 shutdown_connections 2024-03-28T16:48:32.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 --2- 172.21.15.170:0/2016274902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e41180f0 0x7f83e411a4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/2016274902 >> 172.21.15.170:0/2016274902 conn(0x7f83e406f4d0 msgr2=0x7f83e40718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/2016274902 shutdown_connections 2024-03-28T16:48:32.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/2016274902 wait complete. 2024-03-28T16:48:32.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 Processor -- start 2024-03-28T16:48:32.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- start start 2024-03-28T16:48:32.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4115460 0x7f83e41ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e41ac920 con 0x7f83e4115460 2024-03-28T16:48:32.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4115460 0x7f83e41ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4115460 0x7f83e41ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:41890/0 (socket says 172.21.15.170:41890) 2024-03-28T16:48:32.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e259c640 1 -- 172.21.15.170:0/3586795458 learned_addr learned my addr 172.21.15.170:0/3586795458 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:32.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e259c640 1 -- 172.21.15.170:0/3586795458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d8009420 con 0x7f83e4115460 2024-03-28T16:48:32.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e259c640 1 --2- 172.21.15.170:0/3586795458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4115460 0x7f83e41ac350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83d400c480 tx=0x7f83d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83d2ffd640 1 -- 172.21.15.170:0/3586795458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83d4005c20 con 0x7f83e4115460 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e41acb20 con 0x7f83e4115460 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83d2ffd640 1 -- 172.21.15.170:0/3586795458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d4005d80 con 0x7f83e4115460 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83d2ffd640 1 -- 172.21.15.170:0/3586795458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83d40071a0 con 0x7f83e4115460 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e41a61e0 con 0x7f83e4115460 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83d2ffd640 1 -- 172.21.15.170:0/3586795458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f83d4007300 con 0x7f83e4115460 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83d2ffd640 1 -- 172.21.15.170:0/3586795458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f83d4012890 con 0x7f83e4115460 2024-03-28T16:48:32.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.759+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e4062860 con 0x7f83e4115460 2024-03-28T16:48:32.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.767+0000 7f83d2ffd640 1 -- 172.21.15.170:0/3586795458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f83d4021060 con 0x7f83e4115460 2024-03-28T16:48:32.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.831+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df1546de-4221-40e6-b2b6-cdc2d7f86a3c", "id": 3} v 0) v1 -- 0x7f83e4067620 con 0x7f83e4115460 2024-03-28T16:48:32.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.831+0000 7f83d2ffd640 1 -- 172.21.15.170:0/3586795458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df1546de-4221-40e6-b2b6-cdc2d7f86a3c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f83d4017070 con 0x7f83e4115460 2024-03-28T16:48:32.833 INFO:teuthology.orchestra.run.smithi170.stdout:3 2024-03-28T16:48:32.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.835+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4115460 msgr2=0x7f83e41ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.835+0000 7f83e8b60640 1 --2- 172.21.15.170:0/3586795458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4115460 0x7f83e41ac350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83d400c480 tx=0x7f83d400c940 comp rx=0 tx=0).stop 2024-03-28T16:48:32.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.835+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 shutdown_connections 2024-03-28T16:48:32.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.835+0000 7f83e8b60640 1 --2- 172.21.15.170:0/3586795458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e4115460 0x7f83e41ac350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.835+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 >> 172.21.15.170:0/3586795458 conn(0x7f83e406f4d0 msgr2=0x7f83e4118920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.835+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 shutdown_connections 2024-03-28T16:48:32.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:32.835+0000 7f83e8b60640 1 -- 172.21.15.170:0/3586795458 wait complete. 2024-03-28T16:48:32.850 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-28T16:48:32.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-28T16:48:32.853 INFO:tasks.ceph.osd.0:Started 2024-03-28T16:48:32.853 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-28T16:48:32.853 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-28T16:48:32.855 INFO:tasks.ceph.osd.1:Started 2024-03-28T16:48:32.855 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-28T16:48:32.855 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-28T16:48:32.857 INFO:tasks.ceph.osd.2:Started 2024-03-28T16:48:32.857 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-28T16:48:32.858 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-28T16:48:32.860 INFO:tasks.ceph.osd.3:Started 2024-03-28T16:48:32.860 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T16:48:32.978 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f454c453640 1 Processor -- start 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f454c453640 1 -- start start 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f454c453640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f454406c3f0 0x7f454412fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f454c453640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4544072390 con 0x7f454412b000 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f454c453640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454409c1e0 con 0x7f454406c3f0 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f45499ee640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f454406c3f0 0x7f454412fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f454a1ef640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f454412b000 0x7f454412b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45012/0 (socket says 172.21.15.119:45012) 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.974+0000 7f454a1ef640 1 -- 172.21.15.119:0/654602664 learned_addr learned my addr 172.21.15.119:0/654602664 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f45499ee640 1 -- 172.21.15.119:0/654602664 >> v1:172.21.15.119:6789/0 conn(0x7f454412b000 legacy=0x7f454412b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f45491ed640 1 -- 172.21.15.119:0/654602664 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400915628 0 0) 0x7f4544072390 con 0x7f454412b000 2024-03-28T16:48:32.979 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f45499ee640 1 -- 172.21.15.119:0/654602664 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454406d030 con 0x7f454406c3f0 2024-03-28T16:48:32.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f45499ee640 1 --2- 172.21.15.119:0/654602664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f454412fe30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4534009740 tx=0x7f453401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ab2d292398bd7c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f45491ed640 1 -- 172.21.15.119:0/654602664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45340ab530 con 0x7f454406c3f0 2024-03-28T16:48:32.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f45491ed640 1 -- 172.21.15.119:0/654602664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45340a7050 con 0x7f454406c3f0 2024-03-28T16:48:32.981 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f45491ed640 1 -- 172.21.15.119:0/654602664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45340ab7e0 con 0x7f454406c3f0 2024-03-28T16:48:32.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f454c453640 1 -- 172.21.15.119:0/654602664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 msgr2=0x7f454412fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:32.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f454c453640 1 --2- 172.21.15.119:0/654602664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f454412fe30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4534009740 tx=0x7f453401c370 comp rx=0 tx=0).stop 2024-03-28T16:48:32.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f454c453640 1 -- 172.21.15.119:0/654602664 shutdown_connections 2024-03-28T16:48:32.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f454c453640 1 --2- 172.21.15.119:0/654602664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f454412fe30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:32.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f454c453640 1 -- 172.21.15.119:0/654602664 >> 172.21.15.119:0/654602664 conn(0x7f454409a2f0 msgr2=0x7f454409a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:32.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f454c453640 1 -- 172.21.15.119:0/654602664 shutdown_connections 2024-03-28T16:48:32.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.978+0000 7f454c453640 1 -- 172.21.15.119:0/654602664 wait complete. 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454c453640 1 Processor -- start 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454c453640 1 -- start start 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454c453640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f45441222d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454c453640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454409a9f0 con 0x7f454406c3f0 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454a1ef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f45441222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454a1ef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f45441222d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45058/0 (socket says 172.21.15.119:45058) 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454a1ef640 1 -- 172.21.15.119:0/1137712219 learned_addr learned my addr 172.21.15.119:0/1137712219 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454a1ef640 1 -- 172.21.15.119:0/1137712219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4534009420 con 0x7f454406c3f0 2024-03-28T16:48:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454a1ef640 1 --2- 172.21.15.119:0/1137712219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f45441222d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f454000c420 tx=0x7f454000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f453affd640 1 -- 172.21.15.119:0/1137712219 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45400074d0 con 0x7f454406c3f0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f453affd640 1 -- 172.21.15.119:0/1137712219 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540007630 con 0x7f454406c3f0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454412dfe0 con 0x7f454406c3f0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f453affd640 1 -- 172.21.15.119:0/1137712219 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4540004960 con 0x7f454406c3f0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4544071890 con 0x7f454406c3f0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f453affd640 1 -- 172.21.15.119:0/1137712219 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4540004ac0 con 0x7f454406c3f0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f453affd640 1 -- 172.21.15.119:0/1137712219 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f454001c8c0 con 0x7f454406c3f0 2024-03-28T16:48:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.982+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4544067e00 con 0x7f454406c3f0 2024-03-28T16:48:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:32.986+0000 7f453affd640 1 -- 172.21.15.119:0/1137712219 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4540018070 con 0x7f454406c3f0 2024-03-28T16:48:33.043 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4544123c60 con 0x7f454406c3f0 2024-03-28T16:48:33.044 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f453affd640 1 -- 172.21.15.119:0/1137712219 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4153 (secure 0 0 0) 0x7f454001cba0 con 0x7f454406c3f0 2024-03-28T16:48:33.044 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:33.044 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":6,"fsid":"5a56263c-53d8-4d15-86cb-f8a4a2a85230","created":"2024-03-28T16:48:31.676831+0000","modified":"2024-03-28T16:48:32.834854+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T16:48:32.834854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"75d33f3f-95d7-438e-8617-4161e6766019","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"29d55d30-92be-4309-a398-2412a4b9674f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"60322449-dbf6-4117-9bc4-10ac42982aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df1546de-4221-40e6-b2b6-cdc2d7f86a3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T16:48:33.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 msgr2=0x7f45441222d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:33.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 --2- 172.21.15.119:0/1137712219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f45441222d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f454000c420 tx=0x7f454000c8e0 comp rx=0 tx=0).stop 2024-03-28T16:48:33.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 shutdown_connections 2024-03-28T16:48:33.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 --2- 172.21.15.119:0/1137712219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f454406c3f0 0x7f45441222d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:33.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 >> 172.21.15.119:0/1137712219 conn(0x7f454409a2f0 msgr2=0x7f4544094a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:33.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 shutdown_connections 2024-03-28T16:48:33.045 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:33.042+0000 7f454c453640 1 -- 172.21.15.119:0/1137712219 wait complete. 2024-03-28T16:48:33.061 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-28T16:48:33.062 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-28T16:48:33.284 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.283+0000 7fa011d8e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T16:48:33.337 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.335+0000 7fa011d8e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T16:48:33.389 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.387+0000 7fa011d8e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T16:48:33.456 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-03-28T16:48:33.454+0000 7fa2242017c0 -1 Falling back to public interface 2024-03-28T16:48:33.465 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T16:48:33.465 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.463+0000 7fa011d8e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T16:48:33.468 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-03-28T16:48:33.467+0000 7fa68d9777c0 -1 Falling back to public interface 2024-03-28T16:48:33.468 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-03-28T16:48:33.467+0000 7f030c42b7c0 -1 Falling back to public interface 2024-03-28T16:48:33.470 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-03-28T16:48:33.471+0000 7f408f6d47c0 -1 Falling back to public interface 2024-03-28T16:48:33.520 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.519+0000 7fa011d8e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.578+0000 7f02f2ded640 1 Processor -- start 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.578+0000 7f02f2ded640 1 -- start start 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.578+0000 7f02f2ded640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02ec128430 0x7f02ec128950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.578+0000 7f02f2ded640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ec0961b0 con 0x7f02ec128e90 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.578+0000 7f02f2ded640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ec094b00 con 0x7f02ec128430 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f0b89640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f02ec128430 0x7f02ec128950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02ebfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f02ec128e90 0x7f02ec12b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45016/0 (socket says 172.21.15.119:45016) 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02ebfff640 1 -- 172.21.15.119:0/187829010 learned_addr learned my addr 172.21.15.119:0/187829010 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:33.583 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f0b89640 1 -- 172.21.15.119:0/187829010 >> v1:172.21.15.119:6789/0 conn(0x7f02ec128e90 legacy=0x7f02ec12b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:33.584 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f0b89640 1 -- 172.21.15.119:0/187829010 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec0724c0 con 0x7f02ec128430 2024-03-28T16:48:33.585 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f0b89640 1 --2- 172.21.15.119:0/187829010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec128950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009770 tx=0x7f02dc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7bc53f31ba05eb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:33.585 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02eb7fe640 1 -- 172.21.15.119:0/187829010 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02dc0ab530 con 0x7f02ec128430 2024-03-28T16:48:33.585 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02eb7fe640 1 -- 172.21.15.119:0/187829010 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc0a7050 con 0x7f02ec128430 2024-03-28T16:48:33.585 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02eb7fe640 1 -- 172.21.15.119:0/187829010 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02dc0ab7e0 con 0x7f02ec128430 2024-03-28T16:48:33.586 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f2ded640 1 -- 172.21.15.119:0/187829010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 msgr2=0x7f02ec128950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:33.586 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f2ded640 1 --2- 172.21.15.119:0/187829010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec128950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009770 tx=0x7f02dc01c400 comp rx=0 tx=0).stop 2024-03-28T16:48:33.586 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f2ded640 1 -- 172.21.15.119:0/187829010 shutdown_connections 2024-03-28T16:48:33.586 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f2ded640 1 --2- 172.21.15.119:0/187829010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec128950 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:33.586 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f2ded640 1 -- 172.21.15.119:0/187829010 >> 172.21.15.119:0/187829010 conn(0x7f02ec09a000 msgr2=0x7f02ec09a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:33.587 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f2ded640 1 -- 172.21.15.119:0/187829010 shutdown_connections 2024-03-28T16:48:33.587 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.582+0000 7f02f2ded640 1 -- 172.21.15.119:0/187829010 wait complete. 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f2ded640 1 Processor -- start 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f2ded640 1 -- start start 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f2ded640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec1c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f2ded640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec1255a0 con 0x7f02ec128430 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f0b89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec1c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f0b89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec1c3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45074/0 (socket says 172.21.15.119:45074) 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f0b89640 1 -- 172.21.15.119:0/2802082584 learned_addr learned my addr 172.21.15.119:0/2802082584 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f0b89640 1 -- 172.21.15.119:0/2802082584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc009420 con 0x7f02ec128430 2024-03-28T16:48:33.589 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f0b89640 1 --2- 172.21.15.119:0/2802082584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec1c3600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0ae040 tx=0x7f02dc0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02e97fa640 1 -- 172.21.15.119:0/2802082584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02dc0b4540 con 0x7f02ec128430 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02e97fa640 1 -- 172.21.15.119:0/2802082584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc0b46a0 con 0x7f02ec128430 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec095ea0 con 0x7f02ec128430 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02e97fa640 1 -- 172.21.15.119:0/2802082584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02dc0b4980 con 0x7f02ec128430 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec125ad0 con 0x7f02ec128430 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02e97fa640 1 -- 172.21.15.119:0/2802082584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f02dc0b2040 con 0x7f02ec128430 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02e97fa640 1 -- 172.21.15.119:0/2802082584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f02dc0bdcb0 con 0x7f02ec128430 2024-03-28T16:48:33.590 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.586+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec067fa0 con 0x7f02ec128430 2024-03-28T16:48:33.591 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.590+0000 7f02e97fa640 1 -- 172.21.15.119:0/2802082584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f02dc0dcdd0 con 0x7f02ec128430 2024-03-28T16:48:33.596 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.595+0000 7fa011d8e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T16:48:33.647 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02ec062860 con 0x7f02ec128430 2024-03-28T16:48:33.648 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02e97fa640 1 -- 172.21.15.119:0/2802082584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4153 (secure 0 0 0) 0x7f02dc0cb030 con 0x7f02ec128430 2024-03-28T16:48:33.649 INFO:teuthology.misc.health.smithi119.stdout: 2024-03-28T16:48:33.649 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":6,"fsid":"5a56263c-53d8-4d15-86cb-f8a4a2a85230","created":"2024-03-28T16:48:31.676831+0000","modified":"2024-03-28T16:48:32.834854+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T16:48:32.834854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"75d33f3f-95d7-438e-8617-4161e6766019","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"29d55d30-92be-4309-a398-2412a4b9674f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"60322449-dbf6-4117-9bc4-10ac42982aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df1546de-4221-40e6-b2b6-cdc2d7f86a3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T16:48:33.649 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.647+0000 7fa011d8e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T16:48:33.650 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 msgr2=0x7f02ec1c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:33.650 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 --2- 172.21.15.119:0/2802082584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec1c3600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0ae040 tx=0x7f02dc0191a0 comp rx=0 tx=0).stop 2024-03-28T16:48:33.650 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 shutdown_connections 2024-03-28T16:48:33.650 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 --2- 172.21.15.119:0/2802082584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ec128430 0x7f02ec1c3600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:33.650 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 >> 172.21.15.119:0/2802082584 conn(0x7f02ec09a000 msgr2=0x7f02ec096930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:33.650 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 shutdown_connections 2024-03-28T16:48:33.651 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:33.646+0000 7f02f2ded640 1 -- 172.21.15.119:0/2802082584 wait complete. 2024-03-28T16:48:33.665 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-03-28T16:48:33.797 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.795+0000 7fa011d8e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T16:48:33.858 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:33.855+0000 7fa011d8e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T16:48:34.501 INFO:tasks.ceph.mgr.x.smithi170.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-28T16:48:34.501 INFO:tasks.ceph.mgr.x.smithi170.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-28T16:48:34.501 INFO:tasks.ceph.mgr.x.smithi170.stderr: from numpy import show_config as show_numpy_config 2024-03-28T16:48:34.581 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:34.579+0000 7fa011d8e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T16:48:34.761 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:34.759+0000 7fa011d8e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T16:48:34.929 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:34.927+0000 7fa011d8e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T16:48:34.999 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:34.999+0000 7fa011d8e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T16:48:35.145 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:35.143+0000 7fa011d8e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T16:48:35.772 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:35.771+0000 7fa011d8e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T16:48:35.846 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:35.847+0000 7fa011d8e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T16:48:36.025 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:36.023+0000 7fa011d8e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T16:48:36.097 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:36.095+0000 7fa011d8e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T16:48:36.388 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:36.387+0000 7fa011d8e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T16:48:36.450 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:36.451+0000 7fa011d8e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T16:48:37.152 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-03-28T16:48:37.150+0000 7fa2242017c0 -1 osd.0 0 log_to_monitors true 2024-03-28T16:48:37.251 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-03-28T16:48:37.251+0000 7f408f6d47c0 -1 osd.3 0 log_to_monitors true 2024-03-28T16:48:37.344 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-03-28T16:48:37.343+0000 7fa68d9777c0 -1 osd.2 0 log_to_monitors true 2024-03-28T16:48:37.345 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-03-28T16:48:37.343+0000 7f030c42b7c0 -1 osd.1 0 log_to_monitors true 2024-03-28T16:48:37.487 INFO:tasks.ceph.mgr.x.smithi170.stderr:2024-03-28T16:48:37.487+0000 7f9fb91d9640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-03-28T16:48:39.534 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-03-28T16:48:39.530+0000 7fa21f987640 -1 osd.0 0 waiting for initial osdmap 2024-03-28T16:48:39.556 INFO:tasks.ceph.osd.0.smithi119.stderr:2024-03-28T16:48:39.554+0000 7fa217fce640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T16:48:39.788 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-03-28T16:48:39.787+0000 7fa6890fd640 -1 osd.2 0 waiting for initial osdmap 2024-03-28T16:48:39.788 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-03-28T16:48:39.787+0000 7f0307bb1640 -1 osd.1 0 waiting for initial osdmap 2024-03-28T16:48:39.810 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-03-28T16:48:39.811+0000 7f03001f8640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T16:48:39.811 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-03-28T16:48:39.811+0000 7fa681744640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T16:48:39.821 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-03-28T16:48:39.819+0000 7f408ae5a640 -1 osd.3 0 waiting for initial osdmap 2024-03-28T16:48:39.850 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-03-28T16:48:39.851+0000 7f40834a1640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T16:48:40.068 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T16:48:40.190 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.186+0000 7f9f3dab8640 1 Processor -- start 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.186+0000 7f9f3dab8640 1 -- start start 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.186+0000 7f9f3dab8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f38128430 0x7f9f38128950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.186+0000 7f9f3dab8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f380961b0 con 0x7f9f38128e90 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.186+0000 7f9f3dab8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f38094b00 con 0x7f9f38128430 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f377fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f38128430 0x7f9f38128950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f36ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9f38128e90 0x7f9f3812b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45020/0 (socket says 172.21.15.119:45020) 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f36ffd640 1 -- 172.21.15.119:0/2569156799 learned_addr learned my addr 172.21.15.119:0/2569156799 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f377fe640 1 -- 172.21.15.119:0/2569156799 >> v1:172.21.15.119:6789/0 conn(0x7f9f38128e90 legacy=0x7f9f3812b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f377fe640 1 -- 172.21.15.119:0/2569156799 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f380724c0 con 0x7f9f38128430 2024-03-28T16:48:40.191 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f367fc640 1 -- 172.21.15.119:0/2569156799 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443621643 0 0) 0x7f9f380961b0 con 0x7f9f38128e90 2024-03-28T16:48:40.193 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f377fe640 1 --2- 172.21.15.119:0/2569156799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f38128950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009dc0 tx=0x7f9f3001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc67bcb38ba6079 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:40.193 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f367fc640 1 -- 172.21.15.119:0/2569156799 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f300a65d0 con 0x7f9f38128430 2024-03-28T16:48:40.193 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f367fc640 1 -- 172.21.15.119:0/2569156799 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f300a6730 con 0x7f9f38128430 2024-03-28T16:48:40.193 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f367fc640 1 -- 172.21.15.119:0/2569156799 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f300a69f0 con 0x7f9f38128430 2024-03-28T16:48:40.194 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f3dab8640 1 -- 172.21.15.119:0/2569156799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 msgr2=0x7f9f38128950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:40.194 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f3dab8640 1 --2- 172.21.15.119:0/2569156799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f38128950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009dc0 tx=0x7f9f3001c930 comp rx=0 tx=0).stop 2024-03-28T16:48:40.194 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f3dab8640 1 -- 172.21.15.119:0/2569156799 shutdown_connections 2024-03-28T16:48:40.194 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f3dab8640 1 --2- 172.21.15.119:0/2569156799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f38128950 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:40.194 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f3dab8640 1 -- 172.21.15.119:0/2569156799 >> 172.21.15.119:0/2569156799 conn(0x7f9f3809a000 msgr2=0x7f9f3809a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:40.194 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f3dab8640 1 -- 172.21.15.119:0/2569156799 shutdown_connections 2024-03-28T16:48:40.194 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.190+0000 7f9f3dab8640 1 -- 172.21.15.119:0/2569156799 wait complete. 2024-03-28T16:48:40.196 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f3dab8640 1 Processor -- start 2024-03-28T16:48:40.196 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f3dab8640 1 -- start start 2024-03-28T16:48:40.196 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f3dab8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f3806c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:40.196 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f3dab8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3806e0f0 con 0x7f9f38128430 2024-03-28T16:48:40.196 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f3806c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:40.196 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f377fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f3806c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45086/0 (socket says 172.21.15.119:45086) 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f377fe640 1 -- 172.21.15.119:0/1468062787 learned_addr learned my addr 172.21.15.119:0/1468062787 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f377fe640 1 -- 172.21.15.119:0/1468062787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f30009a70 con 0x7f9f38128430 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f377fe640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f3806c0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f3001a560 tx=0x7f9f300196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f1ffff640 1 -- 172.21.15.119:0/1468062787 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f300a7840 con 0x7f9f38128430 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f1ffff640 1 -- 172.21.15.119:0/1468062787 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f300a79a0 con 0x7f9f38128430 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3806d9c0 con 0x7f9f38128430 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f1ffff640 1 -- 172.21.15.119:0/1468062787 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f300a7c60 con 0x7f9f38128430 2024-03-28T16:48:40.197 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3806def0 con 0x7f9f38128430 2024-03-28T16:48:40.198 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f38067fa0 con 0x7f9f38128430 2024-03-28T16:48:40.198 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f1ffff640 1 -- 172.21.15.119:0/1468062787 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 43479+0+0 (secure 0 0 0) 0x7f9f300bb430 con 0x7f9f38128430 2024-03-28T16:48:40.198 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f1ffff640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f24035980 0x7f9f24037e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:40.198 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f1ffff640 1 -- 172.21.15.119:0/1468062787 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1623+0+0 (secure 0 0 0) 0x7f9f300b7070 con 0x7f9f38128430 2024-03-28T16:48:40.198 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f36ffd640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f24035980 0x7f9f24037e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:40.198 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.194+0000 7f9f36ffd640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f24035980 0x7f9f24037e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c007fd0 tx=0x7f9f2c007920 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:40.201 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.198+0000 7f9f1ffff640 1 -- 172.21.15.119:0/1468062787 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f300a7dc0 con 0x7f9f38128430 2024-03-28T16:48:40.359 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.358+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f38123660 con 0x7f9f38128430 2024-03-28T16:48:40.360 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.358+0000 7f9f1ffff640 1 -- 172.21.15.119:0/1468062787 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4153 (secure 0 0 0) 0x7f9f38123660 con 0x7f9f38128430 2024-03-28T16:48:40.361 INFO:teuthology.misc.health.smithi119.stdout: 2024-03-28T16:48:40.361 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":8,"fsid":"5a56263c-53d8-4d15-86cb-f8a4a2a85230","created":"2024-03-28T16:48:31.676831+0000","modified":"2024-03-28T16:48:38.542383+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T16:48:32.834854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"75d33f3f-95d7-438e-8617-4161e6766019","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"29d55d30-92be-4309-a398-2412a4b9674f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"60322449-dbf6-4117-9bc4-10ac42982aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df1546de-4221-40e6-b2b6-cdc2d7f86a3c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f24035980 msgr2=0x7f9f24037e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f24035980 0x7f9f24037e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c007fd0 tx=0x7f9f2c007920 comp rx=0 tx=0).stop 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 msgr2=0x7f9f3806c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f3806c0e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f3001a560 tx=0x7f9f300196d0 comp rx=0 tx=0).stop 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 shutdown_connections 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f24035980 0x7f9f24037e40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 --2- 172.21.15.119:0/1468062787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38128430 0x7f9f3806c0e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:40.363 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 >> 172.21.15.119:0/1468062787 conn(0x7f9f3809a000 msgr2=0x7f9f38127120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:40.364 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 shutdown_connections 2024-03-28T16:48:40.364 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:40.362+0000 7f9f3dab8640 1 -- 172.21.15.119:0/1468062787 wait complete. 2024-03-28T16:48:40.379 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-03-28T16:48:46.783 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T16:48:46.903 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f600021c640 1 Processor -- start 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f600021c640 1 -- start start 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f600021c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ff812df90 0x7f5ff812e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f600021c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff809d5e0 con 0x7f5ff812e8b0 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f600021c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff809b660 con 0x7f5ff812df90 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f5ffdfb8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ff812df90 0x7f5ff812e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f5ffd7b7640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5ff812e8b0 0x7f5ff8099a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41550/0 (socket says 172.21.15.119:41550) 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f5ffdfb8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ff812df90 0x7f5ff812e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42468/0 (socket says 172.21.15.119:42468) 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f5ffd7b7640 1 -- 172.21.15.119:0/407346943 learned_addr learned my addr 172.21.15.119:0/407346943 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f5ffdfb8640 1 -- 172.21.15.119:0/407346943 >> v1:172.21.15.119:6789/0 conn(0x7f5ff812e8b0 legacy=0x7f5ff8099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:46.904 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.902+0000 7f5ffdfb8640 1 -- 172.21.15.119:0/407346943 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff809bb80 con 0x7f5ff812df90 2024-03-28T16:48:46.907 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffdfb8640 1 --2- 172.21.15.119:0/407346943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff812e370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009740 tx=0x7f5ff401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d17ac7da61cc654 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:46.907 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffcfb6640 1 -- 172.21.15.119:0/407346943 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ff40ab530 con 0x7f5ff812df90 2024-03-28T16:48:46.907 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffcfb6640 1 -- 172.21.15.119:0/407346943 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff40a7050 con 0x7f5ff812df90 2024-03-28T16:48:46.907 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffcfb6640 1 -- 172.21.15.119:0/407346943 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ff40ab7e0 con 0x7f5ff812df90 2024-03-28T16:48:46.908 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- 172.21.15.119:0/407346943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 msgr2=0x7f5ff812e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:46.908 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 --2- 172.21.15.119:0/407346943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff812e370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009740 tx=0x7f5ff401c370 comp rx=0 tx=0).stop 2024-03-28T16:48:46.908 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- 172.21.15.119:0/407346943 shutdown_connections 2024-03-28T16:48:46.908 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 --2- 172.21.15.119:0/407346943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff812e370 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:46.908 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- 172.21.15.119:0/407346943 >> 172.21.15.119:0/407346943 conn(0x7f5ff806c4e0 msgr2=0x7f5ff806c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:46.908 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- 172.21.15.119:0/407346943 shutdown_connections 2024-03-28T16:48:46.909 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- 172.21.15.119:0/407346943 wait complete. 2024-03-28T16:48:46.910 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 Processor -- start 2024-03-28T16:48:46.910 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- start start 2024-03-28T16:48:46.910 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff811c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:46.910 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8133b20 con 0x7f5ff812df90 2024-03-28T16:48:46.910 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffdfb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff811c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:46.910 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffdfb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff811c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42476/0 (socket says 172.21.15.119:42476) 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffdfb8640 1 -- 172.21.15.119:0/1191045142 learned_addr learned my addr 172.21.15.119:0/1191045142 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffdfb8640 1 -- 172.21.15.119:0/1191045142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff4009420 con 0x7f5ff812df90 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5ffdfb8640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff811c970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40ae040 tx=0x7f5ff401abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5fee7fc640 1 -- 172.21.15.119:0/1191045142 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ff40b7450 con 0x7f5ff812df90 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5fee7fc640 1 -- 172.21.15.119:0/1191045142 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff40b75b0 con 0x7f5ff812df90 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff811fb70 con 0x7f5ff812df90 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f5fee7fc640 1 -- 172.21.15.119:0/1191045142 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ff40b7890 con 0x7f5ff812df90 2024-03-28T16:48:46.911 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.906+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff811f490 con 0x7f5ff812df90 2024-03-28T16:48:46.912 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.910+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8062860 con 0x7f5ff812df90 2024-03-28T16:48:46.912 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.910+0000 7f5fee7fc640 1 -- 172.21.15.119:0/1191045142 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f5ff40b79f0 con 0x7f5ff812df90 2024-03-28T16:48:46.912 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.910+0000 7f5fee7fc640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5fdc035a10 0x7f5fdc037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:46.912 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.910+0000 7f5fee7fc640 1 -- 172.21.15.119:0/1191045142 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f5ff40e2460 con 0x7f5ff812df90 2024-03-28T16:48:46.912 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.910+0000 7f5ffd7b7640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5fdc035a10 0x7f5fdc037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:46.912 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.910+0000 7f5ffd7b7640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5fdc035a10 0x7f5fdc037ed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8001610 tx=0x7f5fe8007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:46.916 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:46.914+0000 7f5fee7fc640 1 -- 172.21.15.119:0/1191045142 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ff40c74e0 con 0x7f5ff812df90 2024-03-28T16:48:47.072 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.070+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ff811e2d0 con 0x7f5ff812df90 2024-03-28T16:48:47.073 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.070+0000 7f5fee7fc640 1 -- 172.21.15.119:0/1191045142 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7963 (secure 0 0 0) 0x7f5ff40bb020 con 0x7f5ff812df90 2024-03-28T16:48:47.073 INFO:teuthology.misc.health.smithi119.stdout: 2024-03-28T16:48:47.073 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":13,"fsid":"5a56263c-53d8-4d15-86cb-f8a4a2a85230","created":"2024-03-28T16:48:31.676831+0000","modified":"2024-03-28T16:48:44.588450+0000","last_up_change":"2024-03-28T16:48:40.552150+0000","last_in_change":"2024-03-28T16:48:32.834854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T16:48:42.560893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75d33f3f-95d7-438e-8617-4161e6766019","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6801","nonce":2028815515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6803","nonce":2028815515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6807","nonce":2028815515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6805","nonce":2028815515}]},"public_addr":"172.21.15.119:6801/2028815515","cluster_addr":"172.21.15.119:6803/2028815515","heartbeat_back_addr":"172.21.15.119:6807/2028815515","heartbeat_front_addr":"172.21.15.119:6805/2028815515","state":["exists","up"]},{"osd":1,"uuid":"29d55d30-92be-4309-a398-2412a4b9674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6807","nonce":11349688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6811","nonce":11349688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6815","nonce":11349688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6813","nonce":11349688}]},"public_addr":"172.21.15.170:6807/11349688","cluster_addr":"172.21.15.170:6811/11349688","heartbeat_back_addr":"172.21.15.170:6815/11349688","heartbeat_front_addr":"172.21.15.170:6813/11349688","state":["exists","up"]},{"osd":2,"uuid":"60322449-dbf6-4117-9bc4-10ac42982aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6801","nonce":447751618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6803","nonce":447751618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6810","nonce":447751618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6806","nonce":447751618}]},"public_addr":"172.21.15.170:6801/447751618","cluster_addr":"172.21.15.170:6803/447751618","heartbeat_back_addr":"172.21.15.170:6810/447751618","heartbeat_front_addr":"172.21.15.170:6806/447751618","state":["exists","up"]},{"osd":3,"uuid":"df1546de-4221-40e6-b2b6-cdc2d7f86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6817","nonce":2102412426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6819","nonce":2102412426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6823","nonce":2102412426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6821","nonce":2102412426}]},"public_addr":"172.21.15.170:6817/2102412426","cluster_addr":"172.21.15.170:6819/2102412426","heartbeat_back_addr":"172.21.15.170:6823/2102412426","heartbeat_front_addr":"172.21.15.170:6821/2102412426","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.161553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.340062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.387074+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.214018+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T16:48:47.075 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5fdc035a10 msgr2=0x7f5fdc037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5fdc035a10 0x7f5fdc037ed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8001610 tx=0x7f5fe8007ad0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 msgr2=0x7f5ff811c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff811c970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40ae040 tx=0x7f5ff401abb0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 shutdown_connections 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5fdc035a10 0x7f5fdc037ed0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 --2- 172.21.15.119:0/1191045142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff812df90 0x7f5ff811c970 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 >> 172.21.15.119:0/1191045142 conn(0x7f5ff806c4e0 msgr2=0x7f5ff8132430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 shutdown_connections 2024-03-28T16:48:47.076 INFO:teuthology.misc.health.smithi119.stderr:2024-03-28T16:48:47.074+0000 7f600021c640 1 -- 172.21.15.119:0/1191045142 wait complete. 2024-03-28T16:48:47.090 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-03-28T16:48:47.091 INFO:tasks.ceph:Creating RBD pool 2024-03-28T16:48:47.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd pool create rbd 8 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.206+0000 7f3c054d3640 1 Processor -- start 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.206+0000 7f3c054d3640 1 -- start start 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c00095620 0x7f3c00130e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c00094fb0 con 0x7f3c0012c080 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c000949e0 con 0x7f3c00095620 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfe7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c00095620 0x7f3c00130e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfe7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c00095620 0x7f3c00130e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42490/0 (socket says 172.21.15.119:42490) 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfe7fc640 1 -- 172.21.15.119:0/2595712392 learned_addr learned my addr 172.21.15.119:0/2595712392 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfeffd640 1 --1- 172.21.15.119:0/2595712392 >> v1:172.21.15.119:6789/0 conn(0x7f3c0012c080 0x7f3c0012c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41556/0 (socket says 172.21.15.119:41556) 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfe7fc640 1 -- 172.21.15.119:0/2595712392 >> v1:172.21.15.119:6789/0 conn(0x7f3c0012c080 legacy=0x7f3c0012c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfe7fc640 1 -- 172.21.15.119:0/2595712392 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c000951e0 con 0x7f3c00095620 2024-03-28T16:48:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfe7fc640 1 --2- 172.21.15.119:0/2595712392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c00130e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00097f0 tx=0x7f3bf001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca1cca0727eaa49 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfdffb640 1 -- 172.21.15.119:0/2595712392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c00095fa0 con 0x7f3c00095620 2024-03-28T16:48:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfdffb640 1 -- 172.21.15.119:0/2595712392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf00ab400 con 0x7f3c00095620 2024-03-28T16:48:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3bfdffb640 1 -- 172.21.15.119:0/2595712392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bf00ab6c0 con 0x7f3c00095620 2024-03-28T16:48:47.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 -- 172.21.15.119:0/2595712392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 msgr2=0x7f3c00130e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 --2- 172.21.15.119:0/2595712392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c00130e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00097f0 tx=0x7f3bf001c4c0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 -- 172.21.15.119:0/2595712392 shutdown_connections 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 --2- 172.21.15.119:0/2595712392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c00130e40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 -- 172.21.15.119:0/2595712392 >> 172.21.15.119:0/2595712392 conn(0x7f3c0009a050 msgr2=0x7f3c0009a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 -- 172.21.15.119:0/2595712392 shutdown_connections 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.210+0000 7f3c054d3640 1 -- 172.21.15.119:0/2595712392 wait complete. 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3c054d3640 1 Processor -- start 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3c054d3640 1 -- start start 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3c054d3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c001c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3c054d3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c00123660 con 0x7f3c00095620 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c001c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c001c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42496/0 (socket says 172.21.15.119:42496) 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3bfeffd640 1 -- 172.21.15.119:0/1656976373 learned_addr learned my addr 172.21.15.119:0/1656976373 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:47.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3bfeffd640 1 -- 172.21.15.119:0/1656976373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00094d0 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3bfeffd640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c001c34f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c480 tx=0x7f3bf400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3be77fe640 1 -- 172.21.15.119:0/1656976373 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bf40073b0 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3be77fe640 1 -- 172.21.15.119:0/1656976373 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf4007510 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c00072140 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3be77fe640 1 -- 172.21.15.119:0/1656976373 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bf40047f0 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c00123e50 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3be77fe640 1 -- 172.21.15.119:0/1656976373 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3bf4004950 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c00124100 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3be77fe640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3bec035a10 0x7f3bec037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3be77fe640 1 -- 172.21.15.119:0/1656976373 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f3bf40470b0 con 0x7f3c00095620 2024-03-28T16:48:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3bfe7fc640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3bec035a10 0x7f3bec037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:47.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.214+0000 7f3bfe7fc640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3bec035a10 0x7f3bec037ed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00af000 tx=0x7f3bf0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:47.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.218+0000 7f3be77fe640 1 -- 172.21.15.119:0/1656976373 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bf401ad70 con 0x7f3c00095620 2024-03-28T16:48:47.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.374+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f3c00062860 con 0x7f3c00095620 2024-03-28T16:48:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.598+0000 7f3be77fe640 1 -- 172.21.15.119:0/1656976373 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f3bf4018030 con 0x7f3c00095620 2024-03-28T16:48:47.602 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'rbd' created 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3bec035a10 msgr2=0x7f3bec037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3bec035a10 0x7f3bec037ed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00af000 tx=0x7f3bf0008040 comp rx=0 tx=0).stop 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 msgr2=0x7f3c001c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c001c34f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c480 tx=0x7f3bf400c940 comp rx=0 tx=0).stop 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 shutdown_connections 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3bec035a10 0x7f3bec037ed0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 --2- 172.21.15.119:0/1656976373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c00095620 0x7f3c001c34f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 >> 172.21.15.119:0/1656976373 conn(0x7f3c0009a050 msgr2=0x7f3c00097380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 shutdown_connections 2024-03-28T16:48:47.607 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.606+0000 7f3c054d3640 1 -- 172.21.15.119:0/1656976373 wait complete. 2024-03-28T16:48:47.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-03-28T16:48:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.750+0000 7f355399c640 1 Processor -- start 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.750+0000 7f355399c640 1 -- start start 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.750+0000 7f355399c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f354c130ff0 0x7f354c1333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.750+0000 7f355399c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f354c09a530 con 0x7f354c0967e0 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f355399c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354c09a030 con 0x7f354c130ff0 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3550f37640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f354c130ff0 0x7f354c1333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3551738640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f354c0967e0 0x7f354c096bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41562/0 (socket says 172.21.15.119:41562) 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3551738640 1 -- 172.21.15.119:0/217156817 learned_addr learned my addr 172.21.15.119:0/217156817 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3550f37640 1 -- 172.21.15.119:0/217156817 >> v1:172.21.15.119:6789/0 conn(0x7f354c0967e0 legacy=0x7f354c096bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3550f37640 1 -- 172.21.15.119:0/217156817 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0a1250 con 0x7f354c130ff0 2024-03-28T16:48:47.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3550f37640 1 --2- 172.21.15.119:0/217156817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c130ff0 0x7f354c1333e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3548009740 tx=0x7f354801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eae1de7f7acbe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:47.757 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3543fff640 1 -- 172.21.15.119:0/217156817 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35480ab530 con 0x7f354c130ff0 2024-03-28T16:48:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3543fff640 1 -- 172.21.15.119:0/217156817 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35480a7050 con 0x7f354c130ff0 2024-03-28T16:48:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f3543fff640 1 -- 172.21.15.119:0/217156817 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35480ab7e0 con 0x7f354c130ff0 2024-03-28T16:48:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f355399c640 1 -- 172.21.15.119:0/217156817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c130ff0 msgr2=0x7f354c1333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f355399c640 1 --2- 172.21.15.119:0/217156817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c130ff0 0x7f354c1333e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3548009740 tx=0x7f354801c370 comp rx=0 tx=0).stop 2024-03-28T16:48:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f355399c640 1 -- 172.21.15.119:0/217156817 shutdown_connections 2024-03-28T16:48:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f355399c640 1 --2- 172.21.15.119:0/217156817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c130ff0 0x7f354c1333e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.754+0000 7f355399c640 1 -- 172.21.15.119:0/217156817 >> 172.21.15.119:0/217156817 conn(0x7f354c0a0920 msgr2=0x7f354c09ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 -- 172.21.15.119:0/217156817 shutdown_connections 2024-03-28T16:48:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 -- 172.21.15.119:0/217156817 wait complete. 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 Processor -- start 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 -- start start 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c0967e0 0x7f354c1c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c0716d0 con 0x7f354c0967e0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3551738640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c0967e0 0x7f354c1c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3551738640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c0967e0 0x7f354c1c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42518/0 (socket says 172.21.15.119:42518) 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3551738640 1 -- 172.21.15.119:0/4028002806 learned_addr learned my addr 172.21.15.119:0/4028002806 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3551738640 1 -- 172.21.15.119:0/4028002806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009420 con 0x7f354c0967e0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3551738640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c0967e0 0x7f354c1c5570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f353c00c480 tx=0x7f353c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3541ffb640 1 -- 172.21.15.119:0/4028002806 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f353c005c20 con 0x7f354c0967e0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3541ffb640 1 -- 172.21.15.119:0/4028002806 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353c005d80 con 0x7f354c0967e0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c071290 con 0x7f354c0967e0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3541ffb640 1 -- 172.21.15.119:0/4028002806 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f353c0071a0 con 0x7f354c0967e0 2024-03-28T16:48:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c071f20 con 0x7f354c0967e0 2024-03-28T16:48:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c062860 con 0x7f354c0967e0 2024-03-28T16:48:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3541ffb640 1 -- 172.21.15.119:0/4028002806 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f353c007300 con 0x7f354c0967e0 2024-03-28T16:48:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3541ffb640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3530035a10 0x7f3530037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.758+0000 7f3541ffb640 1 -- 172.21.15.119:0/4028002806 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3471+0+0 (secure 0 0 0) 0x7f353c047210 con 0x7f354c0967e0 2024-03-28T16:48:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.762+0000 7f3550f37640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3530035a10 0x7f3530037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.762+0000 7f3550f37640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3530035a10 0x7f3530037ed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f35480ad000 tx=0x7f3548008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:47.766 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.766+0000 7f3541ffb640 1 -- 172.21.15.119:0/4028002806 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f353c01b3b0 con 0x7f354c0967e0 2024-03-28T16:48:47.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:47.910+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f354c09a7f0 con 0x7f354c0967e0 2024-03-28T16:48:48.608 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.606+0000 7f3541ffb640 1 -- 172.21.15.119:0/4028002806 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f353c017070 con 0x7f354c0967e0 2024-03-28T16:48:48.608 INFO:teuthology.orchestra.run.smithi119.stderr:enabled application 'rbd' on pool 'rbd' 2024-03-28T16:48:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3530035a10 msgr2=0x7f3530037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3530035a10 0x7f3530037ed0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f35480ad000 tx=0x7f3548008040 comp rx=0 tx=0).stop 2024-03-28T16:48:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c0967e0 msgr2=0x7f354c1c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c0967e0 0x7f354c1c5570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f353c00c480 tx=0x7f353c00c940 comp rx=0 tx=0).stop 2024-03-28T16:48:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 shutdown_connections 2024-03-28T16:48:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3530035a10 0x7f3530037ed0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 --2- 172.21.15.119:0/4028002806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f354c0967e0 0x7f354c1c5570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:48.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 >> 172.21.15.119:0/4028002806 conn(0x7f354c0a0920 msgr2=0x7f354c09ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:48.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 shutdown_connections 2024-03-28T16:48:48.612 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.610+0000 7f355399c640 1 -- 172.21.15.119:0/4028002806 wait complete. 2024-03-28T16:48:48.632 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-28T16:48:48.632 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-28T16:48:48.632 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-28T16:48:48.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-28T16:48:48.633 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-28T16:48:48.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.742+0000 7fa975fec640 1 Processor -- start 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.742+0000 7fa975fec640 1 -- start start 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.742+0000 7fa975fec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa970128d10 0x7fa9701290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.742+0000 7fa975fec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa970095ee0 con 0x7fa97012bb20 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.742+0000 7fa975fec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa970095b30 con 0x7fa970128d10 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa970128d10 0x7fa9701290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa970128d10 0x7fa9701290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42522/0 (socket says 172.21.15.119:42522) 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96f7fe640 1 -- 172.21.15.119:0/447946361 learned_addr learned my addr 172.21.15.119:0/447946361 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96f7fe640 1 -- 172.21.15.119:0/447946361 >> v1:172.21.15.119:6789/0 conn(0x7fa97012bb20 legacy=0x7fa97012bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:48.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96f7fe640 1 -- 172.21.15.119:0/447946361 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970096320 con 0x7fa970128d10 2024-03-28T16:48:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96f7fe640 1 --2- 172.21.15.119:0/447946361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701290f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097f0 tx=0x7fa96401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51adace17bd7081 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96e7fc640 1 -- 172.21.15.119:0/447946361 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9640ab530 con 0x7fa970128d10 2024-03-28T16:48:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96e7fc640 1 -- 172.21.15.119:0/447946361 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9640a7050 con 0x7fa970128d10 2024-03-28T16:48:48.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa96e7fc640 1 -- 172.21.15.119:0/447946361 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9640ab7e0 con 0x7fa970128d10 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 -- 172.21.15.119:0/447946361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 msgr2=0x7fa9701290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 --2- 172.21.15.119:0/447946361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701290f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097f0 tx=0x7fa96401c4c0 comp rx=0 tx=0).stop 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 -- 172.21.15.119:0/447946361 shutdown_connections 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 --2- 172.21.15.119:0/447946361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701290f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 -- 172.21.15.119:0/447946361 >> 172.21.15.119:0/447946361 conn(0x7fa97009a2f0 msgr2=0x7fa97009a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 -- 172.21.15.119:0/447946361 shutdown_connections 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 -- 172.21.15.119:0/447946361 wait complete. 2024-03-28T16:48:48.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.746+0000 7fa975fec640 1 Processor -- start 2024-03-28T16:48:48.751 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa975fec640 1 -- start start 2024-03-28T16:48:48.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa975fec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:48.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa975fec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970071970 con 0x7fa970128d10 2024-03-28T16:48:48.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:48.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42528/0 (socket says 172.21.15.119:42528) 2024-03-28T16:48:48.752 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa96f7fe640 1 -- 172.21.15.119:0/2448690590 learned_addr learned my addr 172.21.15.119:0/2448690590 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa96f7fe640 1 -- 172.21.15.119:0/2448690590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9640094d0 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa96f7fe640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701c3ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9640190d0 tx=0x7fa964019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa957fff640 1 -- 172.21.15.119:0/2448690590 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9640b4780 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa957fff640 1 -- 172.21.15.119:0/2448690590 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9640b48e0 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97012cc70 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa957fff640 1 -- 172.21.15.119:0/2448690590 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9640b4bc0 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa970071650 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa970067e00 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa957fff640 1 -- 172.21.15.119:0/2448690590 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa9640b2070 con 0x7fa970128d10 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa957fff640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa95c035a10 0x7fa95c037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:48.753 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa957fff640 1 -- 172.21.15.119:0/2448690590 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fa9640e2120 con 0x7fa970128d10 2024-03-28T16:48:48.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa96effd640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa95c035a10 0x7fa95c037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:48.754 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.750+0000 7fa96effd640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa95c035a10 0x7fa95c037ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa960000f70 tx=0x7fa960007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:48.756 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.754+0000 7fa957fff640 1 -- 172.21.15.119:0/2448690590 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9640c44e0 con 0x7fa970128d10 2024-03-28T16:48:48.945 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.942+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa97012cff0 con 0x7fa970128d10 2024-03-28T16:48:48.949 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.946+0000 7fa957fff640 1 -- 172.21.15.119:0/2448690590 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+83330 (secure 0 0 0) 0x7fa9640b8020 con 0x7fa970128d10 2024-03-28T16:48:48.951 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:48.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa95c035a10 msgr2=0x7fa95c037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:48.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa95c035a10 0x7fa95c037ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa960000f70 tx=0x7fa960007a40 comp rx=0 tx=0).stop 2024-03-28T16:48:48.952 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 msgr2=0x7fa9701c3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:48.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701c3ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9640190d0 tx=0x7fa964019260 comp rx=0 tx=0).stop 2024-03-28T16:48:48.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 shutdown_connections 2024-03-28T16:48:48.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa95c035a10 0x7fa95c037ed0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:48.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 --2- 172.21.15.119:0/2448690590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa970128d10 0x7fa9701c3ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:48.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 >> 172.21.15.119:0/2448690590 conn(0x7fa97009a2f0 msgr2=0x7fa970097170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:48.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 shutdown_connections 2024-03-28T16:48:48.953 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:48.950+0000 7fa975fec640 1 -- 172.21.15.119:0/2448690590 wait complete. 2024-03-28T16:48:48.966 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":317743614},{"type":"v1","addr":"172.21.15.170:6825","nonce":317743614}]},"active_addr":"172.21.15.170:6825/317743614","active_change":"2024-03-28T16:48:36.474725+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":2340746102}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":2256404857}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":2901273096}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":15437204}]}]} 2024-03-28T16:48:48.968 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-28T16:48:48.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-28T16:48:48.969 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T16:48:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.082+0000 7f2e48f8d640 1 Processor -- start 2024-03-28T16:48:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.082+0000 7f2e48f8d640 1 -- start start 2024-03-28T16:48:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e44093a10 0x7f2e44093df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e44095170 con 0x7f2e44094330 2024-03-28T16:48:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e44072550 con 0x7f2e44093a10 2024-03-28T16:48:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e44093a10 0x7f2e44093df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e44093a10 0x7f2e44093df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42530/0 (socket says 172.21.15.119:42530) 2024-03-28T16:48:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e41d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2e44094330 0x7f2e4412e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41578/0 (socket says 172.21.15.119:41578) 2024-03-28T16:48:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4259c640 1 -- 172.21.15.119:0/12185852 learned_addr learned my addr 172.21.15.119:0/12185852 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4259c640 1 -- 172.21.15.119:0/12185852 >> v1:172.21.15.119:6789/0 conn(0x7f2e44094330 legacy=0x7f2e4412e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4259c640 1 -- 172.21.15.119:0/12185852 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44094bb0 con 0x7f2e44093a10 2024-03-28T16:48:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4259c640 1 --2- 172.21.15.119:0/12185852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e44093df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097f0 tx=0x7f2e2c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8efb081bc9ea888 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4159a640 1 -- 172.21.15.119:0/12185852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c0ab530 con 0x7f2e44093a10 2024-03-28T16:48:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4159a640 1 -- 172.21.15.119:0/12185852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c0a7050 con 0x7f2e44093a10 2024-03-28T16:48:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e4159a640 1 -- 172.21.15.119:0/12185852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c0ab7e0 con 0x7f2e44093a10 2024-03-28T16:48:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 -- 172.21.15.119:0/12185852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 msgr2=0x7f2e44093df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 --2- 172.21.15.119:0/12185852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e44093df0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097f0 tx=0x7f2e2c01c430 comp rx=0 tx=0).stop 2024-03-28T16:48:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 -- 172.21.15.119:0/12185852 shutdown_connections 2024-03-28T16:48:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 --2- 172.21.15.119:0/12185852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e44093df0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.086+0000 7f2e48f8d640 1 -- 172.21.15.119:0/12185852 >> 172.21.15.119:0/12185852 conn(0x7f2e4409a2f0 msgr2=0x7f2e4409a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 -- 172.21.15.119:0/12185852 shutdown_connections 2024-03-28T16:48:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 -- 172.21.15.119:0/12185852 wait complete. 2024-03-28T16:48:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 Processor -- start 2024-03-28T16:48:49.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 -- start start 2024-03-28T16:48:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e4412d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44070c00 con 0x7f2e44093a10 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e4259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e4412d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e4259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e4412d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42546/0 (socket says 172.21.15.119:42546) 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e4259c640 1 -- 172.21.15.119:0/3240445746 learned_addr learned my addr 172.21.15.119:0/3240445746 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e4259c640 1 -- 172.21.15.119:0/3240445746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0094d0 con 0x7f2e44093a10 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e4259c640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e4412d3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c01a060 tx=0x7f2e2c01ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e3affd640 1 -- 172.21.15.119:0/3240445746 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c0b6420 con 0x7f2e44093a10 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e3affd640 1 -- 172.21.15.119:0/3240445746 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c0b6580 con 0x7f2e44093a10 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e3affd640 1 -- 172.21.15.119:0/3240445746 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e2c0b6820 con 0x7f2e44093a10 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e44071930 con 0x7f2e44093a10 2024-03-28T16:48:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e44070540 con 0x7f2e44093a10 2024-03-28T16:48:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e3affd640 1 -- 172.21.15.119:0/3240445746 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2e2c0b2040 con 0x7f2e44093a10 2024-03-28T16:48:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e44067e00 con 0x7f2e44093a10 2024-03-28T16:48:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e3affd640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2e24035a10 0x7f2e24037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e3affd640 1 -- 172.21.15.119:0/3240445746 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f2e2c0e21e0 con 0x7f2e44093a10 2024-03-28T16:48:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.090+0000 7f2e41d9b640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2e24035a10 0x7f2e24037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.094+0000 7f2e41d9b640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2e24035a10 0x7f2e24037ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c001d00 tx=0x7f2e3c007dd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.098 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.094+0000 7f2e3affd640 1 -- 172.21.15.119:0/3240445746 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e2c0b6980 con 0x7f2e44093a10 2024-03-28T16:48:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.258+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e44062860 con 0x7f2e44093a10 2024-03-28T16:48:49.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.258+0000 7f2e3affd640 1 -- 172.21.15.119:0/3240445746 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+9744 (secure 0 0 0) 0x7f2e2c0b0070 con 0x7f2e44093a10 2024-03-28T16:48:49.262 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:49.262 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":15,"fsid":"5a56263c-53d8-4d15-86cb-f8a4a2a85230","created":"2024-03-28T16:48:31.676831+0000","modified":"2024-03-28T16:48:48.605581+0000","last_up_change":"2024-03-28T16:48:40.552150+0000","last_in_change":"2024-03-28T16:48:32.834854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T16:48:42.560893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-28T16:48:47.418826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75d33f3f-95d7-438e-8617-4161e6766019","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6801","nonce":2028815515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6803","nonce":2028815515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6807","nonce":2028815515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6805","nonce":2028815515}]},"public_addr":"172.21.15.119:6801/2028815515","cluster_addr":"172.21.15.119:6803/2028815515","heartbeat_back_addr":"172.21.15.119:6807/2028815515","heartbeat_front_addr":"172.21.15.119:6805/2028815515","state":["exists","up"]},{"osd":1,"uuid":"29d55d30-92be-4309-a398-2412a4b9674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6807","nonce":11349688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6811","nonce":11349688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6815","nonce":11349688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6813","nonce":11349688}]},"public_addr":"172.21.15.170:6807/11349688","cluster_addr":"172.21.15.170:6811/11349688","heartbeat_back_addr":"172.21.15.170:6815/11349688","heartbeat_front_addr":"172.21.15.170:6813/11349688","state":["exists","up"]},{"osd":2,"uuid":"60322449-dbf6-4117-9bc4-10ac42982aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6801","nonce":447751618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6803","nonce":447751618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6810","nonce":447751618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6806","nonce":447751618}]},"public_addr":"172.21.15.170:6801/447751618","cluster_addr":"172.21.15.170:6803/447751618","heartbeat_back_addr":"172.21.15.170:6810/447751618","heartbeat_front_addr":"172.21.15.170:6806/447751618","state":["exists","up"]},{"osd":3,"uuid":"df1546de-4221-40e6-b2b6-cdc2d7f86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6817","nonce":2102412426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6819","nonce":2102412426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6823","nonce":2102412426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6821","nonce":2102412426}]},"public_addr":"172.21.15.170:6817/2102412426","cluster_addr":"172.21.15.170:6819/2102412426","heartbeat_back_addr":"172.21.15.170:6823/2102412426","heartbeat_front_addr":"172.21.15.170:6821/2102412426","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.161553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.340062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.387074+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.214018+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T16:48:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2e24035a10 msgr2=0x7f2e24037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2e24035a10 0x7f2e24037ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c001d00 tx=0x7f2e3c007dd0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 msgr2=0x7f2e4412d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e4412d3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c01a060 tx=0x7f2e2c01ac70 comp rx=0 tx=0).stop 2024-03-28T16:48:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 shutdown_connections 2024-03-28T16:48:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2e24035a10 0x7f2e24037ed0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 --2- 172.21.15.119:0/3240445746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e44093a10 0x7f2e4412d3d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 >> 172.21.15.119:0/3240445746 conn(0x7f2e4409a2f0 msgr2=0x7f2e4412f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 shutdown_connections 2024-03-28T16:48:49.265 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.262+0000 7f2e48f8d640 1 -- 172.21.15.119:0/3240445746 wait complete. 2024-03-28T16:48:49.281 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-28T16:48:49.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T16:48:49.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.394+0000 7f7c89d1c640 1 Processor -- start 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.394+0000 7f7c89d1c640 1 -- start start 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c89d1c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c84128d10 0x7f7c841290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c89d1c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c84095ee0 con 0x7f7c8412bb20 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c89d1c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c84095b30 con 0x7f7c84128d10 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c837fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c84128d10 0x7f7c841290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c82ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7c8412bb20 0x7f7c8412bf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41582/0 (socket says 172.21.15.119:41582) 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c82ffd640 1 -- 172.21.15.119:0/431558201 learned_addr learned my addr 172.21.15.119:0/431558201 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c837fe640 1 -- 172.21.15.119:0/431558201 >> v1:172.21.15.119:6789/0 conn(0x7f7c8412bb20 legacy=0x7f7c8412bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c837fe640 1 -- 172.21.15.119:0/431558201 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84096320 con 0x7f7c84128d10 2024-03-28T16:48:49.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c837fe640 1 --2- 172.21.15.119:0/431558201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c841290f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009740 tx=0x7f7c6c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864e61bb6ead8802 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c827fc640 1 -- 172.21.15.119:0/431558201 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c6c0ab530 con 0x7f7c84128d10 2024-03-28T16:48:49.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c827fc640 1 -- 172.21.15.119:0/431558201 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c0a7050 con 0x7f7c84128d10 2024-03-28T16:48:49.402 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.398+0000 7f7c827fc640 1 -- 172.21.15.119:0/431558201 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c6c0ab7e0 con 0x7f7c84128d10 2024-03-28T16:48:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/431558201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 msgr2=0x7f7c841290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 --2- 172.21.15.119:0/431558201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c841290f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009740 tx=0x7f7c6c01c400 comp rx=0 tx=0).stop 2024-03-28T16:48:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/431558201 shutdown_connections 2024-03-28T16:48:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 --2- 172.21.15.119:0/431558201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c841290f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/431558201 >> 172.21.15.119:0/431558201 conn(0x7f7c8409a2f0 msgr2=0x7f7c8409a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/431558201 shutdown_connections 2024-03-28T16:48:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/431558201 wait complete. 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 Processor -- start 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- start start 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c8412cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8412fac0 con 0x7f7c84128d10 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c8412cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c8412cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42568/0 (socket says 172.21.15.119:42568) 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c837fe640 1 -- 172.21.15.119:0/3676633769 learned_addr learned my addr 172.21.15.119:0/3676633769 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c837fe640 1 -- 172.21.15.119:0/3676633769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c009420 con 0x7f7c84128d10 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c837fe640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c8412cb50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c01a030 tx=0x7f7c6c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c7bfff640 1 -- 172.21.15.119:0/3676633769 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c6c0b4750 con 0x7f7c84128d10 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c7bfff640 1 -- 172.21.15.119:0/3676633769 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c0a4e90 con 0x7f7c84128d10 2024-03-28T16:48:49.405 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c7bfff640 1 -- 172.21.15.119:0/3676633769 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c6c0b49e0 con 0x7f7c84128d10 2024-03-28T16:48:49.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8412dd40 con 0x7f7c84128d10 2024-03-28T16:48:49.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8412f150 con 0x7f7c84128d10 2024-03-28T16:48:49.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c7bfff640 1 -- 172.21.15.119:0/3676633769 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7c6c0b2040 con 0x7f7c84128d10 2024-03-28T16:48:49.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c7bfff640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7c640359c0 0x7f7c64037e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.406 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c840983a0 con 0x7f7c84128d10 2024-03-28T16:48:49.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.402+0000 7f7c7bfff640 1 -- 172.21.15.119:0/3676633769 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f7c6c0e2230 con 0x7f7c84128d10 2024-03-28T16:48:49.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.406+0000 7f7c82ffd640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7c640359c0 0x7f7c64037e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.407 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.406+0000 7f7c82ffd640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7c640359c0 0x7f7c64037e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c74001610 tx=0x7f7c74007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.410+0000 7f7c7bfff640 1 -- 172.21.15.119:0/3676633769 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c6c0b4b40 con 0x7f7c84128d10 2024-03-28T16:48:49.571 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.570+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c84067e00 con 0x7f7c84128d10 2024-03-28T16:48:49.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.570+0000 7f7c7bfff640 1 -- 172.21.15.119:0/3676633769 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+9744 (secure 0 0 0) 0x7f7c6c0a8260 con 0x7f7c84128d10 2024-03-28T16:48:49.573 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:49.573 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":15,"fsid":"5a56263c-53d8-4d15-86cb-f8a4a2a85230","created":"2024-03-28T16:48:31.676831+0000","modified":"2024-03-28T16:48:48.605581+0000","last_up_change":"2024-03-28T16:48:40.552150+0000","last_in_change":"2024-03-28T16:48:32.834854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T16:48:42.560893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-28T16:48:47.418826+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75d33f3f-95d7-438e-8617-4161e6766019","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6801","nonce":2028815515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6803","nonce":2028815515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6807","nonce":2028815515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6805","nonce":2028815515}]},"public_addr":"172.21.15.119:6801/2028815515","cluster_addr":"172.21.15.119:6803/2028815515","heartbeat_back_addr":"172.21.15.119:6807/2028815515","heartbeat_front_addr":"172.21.15.119:6805/2028815515","state":["exists","up"]},{"osd":1,"uuid":"29d55d30-92be-4309-a398-2412a4b9674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6807","nonce":11349688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6811","nonce":11349688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6815","nonce":11349688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6813","nonce":11349688}]},"public_addr":"172.21.15.170:6807/11349688","cluster_addr":"172.21.15.170:6811/11349688","heartbeat_back_addr":"172.21.15.170:6815/11349688","heartbeat_front_addr":"172.21.15.170:6813/11349688","state":["exists","up"]},{"osd":2,"uuid":"60322449-dbf6-4117-9bc4-10ac42982aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6801","nonce":447751618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6803","nonce":447751618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6810","nonce":447751618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6806","nonce":447751618}]},"public_addr":"172.21.15.170:6801/447751618","cluster_addr":"172.21.15.170:6803/447751618","heartbeat_back_addr":"172.21.15.170:6810/447751618","heartbeat_front_addr":"172.21.15.170:6806/447751618","state":["exists","up"]},{"osd":3,"uuid":"df1546de-4221-40e6-b2b6-cdc2d7f86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6817","nonce":2102412426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6819","nonce":2102412426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6823","nonce":2102412426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6821","nonce":2102412426}]},"public_addr":"172.21.15.170:6817/2102412426","cluster_addr":"172.21.15.170:6819/2102412426","heartbeat_back_addr":"172.21.15.170:6823/2102412426","heartbeat_front_addr":"172.21.15.170:6821/2102412426","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.161553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.340062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.387074+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.214018+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7c640359c0 msgr2=0x7f7c64037e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7c640359c0 0x7f7c64037e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7c74001610 tx=0x7f7c74007ad0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 msgr2=0x7f7c8412cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c8412cb50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c01a030 tx=0x7f7c6c0191a0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 shutdown_connections 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7c640359c0 0x7f7c64037e80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 --2- 172.21.15.119:0/3676633769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c84128d10 0x7f7c8412cb50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 >> 172.21.15.119:0/3676633769 conn(0x7f7c8409a2f0 msgr2=0x7f7c84130f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 shutdown_connections 2024-03-28T16:48:49.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.574+0000 7f7c89d1c640 1 -- 172.21.15.119:0/3676633769 wait complete. 2024-03-28T16:48:49.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-28T16:48:49.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-28T16:48:49.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-28T16:48:49.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-28T16:48:49.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.698+0000 7f8c58308640 1 Processor -- start 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.698+0000 7f8c58308640 1 -- start start 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c58308640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c5012a900 0x7f8c5012ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c58308640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5012a700 con 0x7f8c5012b220 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c58308640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c50095310 con 0x7f8c5012a900 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c558a3640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8c5012b220 0x7f8c5006c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41590/0 (socket says 172.21.15.119:41590) 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c560a4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c5012a900 0x7f8c5012ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c558a3640 1 -- 172.21.15.119:0/27361463 learned_addr learned my addr 172.21.15.119:0/27361463 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f486c228640 1 Processor -- start 2024-03-28T16:48:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c560a4640 1 -- 172.21.15.119:0/27361463 >> v1:172.21.15.119:6789/0 conn(0x7f8c5012b220 legacy=0x7f8c5006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c560a4640 1 -- 172.21.15.119:0/27361463 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500948e0 con 0x7f8c5012a900 2024-03-28T16:48:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f486c228640 1 -- start start 2024-03-28T16:48:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4d1839640 1 Processor -- start 2024-03-28T16:48:49.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f969e6ce640 1 Processor -- start 2024-03-28T16:48:49.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f969e6ce640 1 -- start start 2024-03-28T16:48:49.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4d1839640 1 -- start start 2024-03-28T16:48:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f486c228640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f486409c4c0 0x7f4864130f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f486c228640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486409ce40 con 0x7f486409a3f0 2024-03-28T16:48:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f486c228640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486409b7a0 con 0x7f486409c4c0 2024-03-28T16:48:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f4869fc4640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f486409a3f0 0x7f486409a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41604/0 (socket says 172.21.15.119:41604) 2024-03-28T16:48:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f4869fc4640 1 -- 172.21.15.119:0/3521922483 learned_addr learned my addr 172.21.15.119:0/3521922483 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f48697c3640 1 --2- 172.21.15.119:0/3521922483 >> v2:172.21.15.119:3300/0 conn(0x7f486409c4c0 0x7f4864130f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f969e6ce640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f969806d3c0 0x7f9698130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f969e6ce640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96981284f0 con 0x7f9698128990 2024-03-28T16:48:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f969e6ce640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9698129480 con 0x7f969806d3c0 2024-03-28T16:48:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f48697c3640 1 -- 172.21.15.119:0/3521922483 >> v1:172.21.15.119:6789/0 conn(0x7f486409a3f0 legacy=0x7f486409a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f4868fc2640 1 -- 172.21.15.119:0/3521922483 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305281898 0 0) 0x7f486409ce40 con 0x7f486409a3f0 2024-03-28T16:48:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f48697c3640 1 -- 172.21.15.119:0/3521922483 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486409da30 con 0x7f486409c4c0 2024-03-28T16:48:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f9697fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9698128990 0x7f9698128d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41606/0 (socket says 172.21.15.119:41606) 2024-03-28T16:48:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f9697fff640 1 -- 172.21.15.119:0/3544151592 learned_addr learned my addr 172.21.15.119:0/3544151592 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:49.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4d1839640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff4cc12e0e0 0x7ff4cc12e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f96977fe640 1 --2- 172.21.15.119:0/3544151592 >> v2:172.21.15.119:3300/0 conn(0x7f969806d3c0 0x7f9698130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4d1839640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4cc12dc20 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4d1839640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4cc098b80 con 0x7ff4cc12e0e0 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4ca7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff4cc06d1e0 0x7ff4cc06d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41616/0 (socket says 172.21.15.119:41616) 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4ca7fc640 1 -- 172.21.15.119:0/90503256 learned_addr learned my addr 172.21.15.119:0/90503256 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4caffd640 1 --2- 172.21.15.119:0/90503256 >> v2:172.21.15.119:3300/0 conn(0x7ff4cc12e0e0 0x7ff4cc12e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f9696ffd640 1 -- 172.21.15.119:0/3544151592 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842659041 0 0) 0x7f96981284f0 con 0x7f9698128990 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f9696ffd640 1 -- 172.21.15.119:0/3544151592 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9680003530 con 0x7f9698128990 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f96977fe640 1 -- 172.21.15.119:0/3544151592 >> v1:172.21.15.119:6789/0 conn(0x7f9698128990 legacy=0x7f9698128d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f96977fe640 1 -- 172.21.15.119:0/3544151592 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698128790 con 0x7f969806d3c0 2024-03-28T16:48:49.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4caffd640 1 -- 172.21.15.119:0/90503256 >> v1:172.21.15.119:6789/0 conn(0x7ff4cc06d1e0 legacy=0x7ff4cc06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7ff4caffd640 1 -- 172.21.15.119:0/90503256 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc0720c0 con 0x7ff4cc12e0e0 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.702+0000 7f8c560a4640 1 --2- 172.21.15.119:0/27361463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c5012ace0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009740 tx=0x7f8c4c01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4262badcb62a23ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c550a2640 1 -- 172.21.15.119:0/27361463 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c4c0ab530 con 0x7f8c5012a900 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c550a2640 1 -- 172.21.15.119:0/27361463 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c0a7050 con 0x7f8c5012a900 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c550a2640 1 -- 172.21.15.119:0/27361463 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c4c0ab7e0 con 0x7f8c5012a900 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- 172.21.15.119:0/27361463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 msgr2=0x7f8c5012ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 --2- 172.21.15.119:0/27361463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c5012ace0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009740 tx=0x7f8c4c01c370 comp rx=0 tx=0).stop 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- 172.21.15.119:0/27361463 shutdown_connections 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 --2- 172.21.15.119:0/27361463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c5012ace0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- 172.21.15.119:0/27361463 >> 172.21.15.119:0/27361463 conn(0x7f8c5009d4e0 msgr2=0x7f8c5009d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- 172.21.15.119:0/27361463 shutdown_connections 2024-03-28T16:48:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- 172.21.15.119:0/27361463 wait complete. 2024-03-28T16:48:49.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f48697c3640 1 --2- 172.21.15.119:0/3521922483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409c4c0 0x7f4864130f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4854009740 tx=0x7f485401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea18a2e6c67ecc63 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f4868fc2640 1 -- 172.21.15.119:0/3521922483 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48540ab530 con 0x7f486409c4c0 2024-03-28T16:48:49.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f4868fc2640 1 -- 172.21.15.119:0/3521922483 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48540a7050 con 0x7f486409c4c0 2024-03-28T16:48:49.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f4868fc2640 1 -- 172.21.15.119:0/3521922483 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48540ab7e0 con 0x7f486409c4c0 2024-03-28T16:48:49.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4caffd640 1 --2- 172.21.15.119:0/90503256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc12e0e0 0x7ff4cc12e4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009740 tx=0x7ff4bc01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f49f6b124a7e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 -- 172.21.15.119:0/3521922483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409c4c0 msgr2=0x7f4864130f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 --2- 172.21.15.119:0/3521922483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409c4c0 0x7f4864130f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4854009740 tx=0x7f485401c370 comp rx=0 tx=0).stop 2024-03-28T16:48:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4c9ffb640 1 -- 172.21.15.119:0/90503256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc0a8050 con 0x7ff4cc12e0e0 2024-03-28T16:48:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4c9ffb640 1 -- 172.21.15.119:0/90503256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc0ac520 con 0x7ff4cc12e0e0 2024-03-28T16:48:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4c9ffb640 1 -- 172.21.15.119:0/90503256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc0ac7e0 con 0x7ff4cc12e0e0 2024-03-28T16:48:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 Processor -- start 2024-03-28T16:48:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 -- 172.21.15.119:0/3521922483 shutdown_connections 2024-03-28T16:48:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 --2- 172.21.15.119:0/3521922483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409c4c0 0x7f4864130f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 -- 172.21.15.119:0/3521922483 >> 172.21.15.119:0/3521922483 conn(0x7f4864070240 msgr2=0x7f4864098a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- start start 2024-03-28T16:48:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f96977fe640 1 --2- 172.21.15.119:0/3544151592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f9698130d70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f968c009740 tx=0x7f968c01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51b478e4f8361a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f9696ffd640 1 -- 172.21.15.119:0/3544151592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f968c0ab530 con 0x7f969806d3c0 2024-03-28T16:48:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f9696ffd640 1 -- 172.21.15.119:0/3544151592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c0a7050 con 0x7f969806d3c0 2024-03-28T16:48:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f9696ffd640 1 -- 172.21.15.119:0/3544151592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f968c0ab7e0 con 0x7f969806d3c0 2024-03-28T16:48:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4d1839640 1 -- 172.21.15.119:0/90503256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc12e0e0 msgr2=0x7ff4cc12e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4d1839640 1 --2- 172.21.15.119:0/90503256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc12e0e0 0x7ff4cc12e4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009740 tx=0x7ff4bc01c370 comp rx=0 tx=0).stop 2024-03-28T16:48:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c501c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5012fa30 con 0x7f8c5012a900 2024-03-28T16:48:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4d1839640 1 -- 172.21.15.119:0/90503256 shutdown_connections 2024-03-28T16:48:49.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4d1839640 1 --2- 172.21.15.119:0/90503256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc12e0e0 0x7ff4cc12e4c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4d1839640 1 -- 172.21.15.119:0/90503256 >> 172.21.15.119:0/90503256 conn(0x7ff4cc09c330 msgr2=0x7ff4cc071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 -- 172.21.15.119:0/3521922483 shutdown_connections 2024-03-28T16:48:49.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 -- 172.21.15.119:0/3521922483 wait complete. 2024-03-28T16:48:49.717 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4d1839640 1 -- 172.21.15.119:0/90503256 shutdown_connections 2024-03-28T16:48:49.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c560a4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c501c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c560a4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c501c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42594/0 (socket says 172.21.15.119:42594) 2024-03-28T16:48:49.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7ff4d1839640 1 -- 172.21.15.119:0/90503256 wait complete. 2024-03-28T16:48:49.718 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c560a4640 1 -- 172.21.15.119:0/2813712886 learned_addr learned my addr 172.21.15.119:0/2813712886 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c560a4640 1 -- 172.21.15.119:0/2813712886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c009420 con 0x7f8c5012a900 2024-03-28T16:48:49.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f969e6ce640 1 -- 172.21.15.119:0/3544151592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 msgr2=0x7f9698130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f969e6ce640 1 --2- 172.21.15.119:0/3544151592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f9698130d70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f968c009740 tx=0x7f968c01c370 comp rx=0 tx=0).stop 2024-03-28T16:48:49.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c560a4640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c501c5740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0ae040 tx=0x7f8c4c019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c4c0b4410 con 0x7f8c5012a900 2024-03-28T16:48:49.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c0b4570 con 0x7f8c5012a900 2024-03-28T16:48:49.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5012e8e0 con 0x7f8c5012a900 2024-03-28T16:48:49.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 Processor -- start 2024-03-28T16:48:49.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.706+0000 7f486c228640 1 -- start start 2024-03-28T16:48:49.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4d1839640 1 Processor -- start 2024-03-28T16:48:49.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4d1839640 1 -- start start 2024-03-28T16:48:49.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f486c228640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409a3f0 0x7f48641bdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f486c228640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4864126770 con 0x7f486409a3f0 2024-03-28T16:48:49.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 -- 172.21.15.119:0/3544151592 shutdown_connections 2024-03-28T16:48:49.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 --2- 172.21.15.119:0/3544151592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f9698130d70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 -- 172.21.15.119:0/3544151592 >> 172.21.15.119:0/3544151592 conn(0x7f969809d1d0 msgr2=0x7f969809d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 -- 172.21.15.119:0/3544151592 shutdown_connections 2024-03-28T16:48:49.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 -- 172.21.15.119:0/3544151592 wait complete. 2024-03-28T16:48:49.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5012f3a0 con 0x7f8c5012a900 2024-03-28T16:48:49.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f4869fc4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409a3f0 0x7f48641bdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f4869fc4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409a3f0 0x7f48641bdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42600/0 (socket says 172.21.15.119:42600) 2024-03-28T16:48:49.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f4869fc4640 1 -- 172.21.15.119:0/2580175610 learned_addr learned my addr 172.21.15.119:0/2580175610 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c4c0b4850 con 0x7f8c5012a900 2024-03-28T16:48:49.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8c4c0b2040 con 0x7f8c5012a900 2024-03-28T16:48:49.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4d1839640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06d1e0 0x7ff4cc1329f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.724 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4d1839640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc132040 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06d1e0 0x7ff4cc1329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06d1e0 0x7ff4cc1329f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42602/0 (socket says 172.21.15.119:42602) 2024-03-28T16:48:49.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4caffd640 1 -- 172.21.15.119:0/2381842148 learned_addr learned my addr 172.21.15.119:0/2381842148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.725 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f4869fc4640 1 -- 172.21.15.119:0/2580175610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009420 con 0x7f486409a3f0 2024-03-28T16:48:49.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c340359f0 0x7f8c34037eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c5012f5f0 con 0x7f8c5012a900 2024-03-28T16:48:49.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f4869fc4640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409a3f0 0x7f48641bdf70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f485c00c840 tx=0x7f485c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4caffd640 1 -- 172.21.15.119:0/2381842148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009420 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f8c4c0a4e40 con 0x7f8c5012a900 2024-03-28T16:48:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4caffd640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06d1e0 0x7ff4cc1329f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0af040 tx=0x7ff4bc01abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f8c3403b000 0x7f8c3403d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 Processor -- start 2024-03-28T16:48:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c3403db10 con 0x7f8c3403b000 2024-03-28T16:48:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c4c0aa3e0 con 0x7f8c5012a900 2024-03-28T16:48:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f485a7fc640 1 -- 172.21.15.119:0/2580175610 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f485c00a660 con 0x7f486409a3f0 2024-03-28T16:48:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f485a7fc640 1 -- 172.21.15.119:0/2580175610 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c00a7e0 con 0x7f486409a3f0 2024-03-28T16:48:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f485a7fc640 1 -- 172.21.15.119:0/2580175610 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f485c00aac0 con 0x7f486409a3f0 2024-03-28T16:48:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c558a3640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c340359f0 0x7f8c34037eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 -- start start 2024-03-28T16:48:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c568a5640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f8c3403b000 0x7f8c3403d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48641166a0 con 0x7f486409a3f0 2024-03-28T16:48:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48641300b0 con 0x7f486409a3f0 2024-03-28T16:48:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c568a5640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f8c3403b000 0x7f8c3403d440 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c558a3640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c340359f0 0x7f8c34037eb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001610 tx=0x7f8c40007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc0b8790 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc0b88f0 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc0b8bd0 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4d1839640 1 -- 172.21.15.119:0/2381842148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc06f200 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4d1839640 1 -- 172.21.15.119:0/2381842148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc132560 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8c3403db10 con 0x7f8c3403b000 2024-03-28T16:48:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f485a7fc640 1 -- 172.21.15.119:0/2580175610 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f485c00ac20 con 0x7f486409a3f0 2024-03-28T16:48:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff4a4000f90 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f485a7fc640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4848035a10 0x7f4848037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f485a7fc640 1 -- 172.21.15.119:0/2580175610 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f485c047740 con 0x7f486409a3f0 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f96981c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f969e6ce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969812b820 con 0x7f969806d3c0 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f9697fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f96981c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f9697fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f96981c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42608/0 (socket says 172.21.15.119:42608) 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.710+0000 7f9697fff640 1 -- 172.21.15.119:0/578835759 learned_addr learned my addr 172.21.15.119:0/578835759 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9697fff640 1 -- 172.21.15.119:0/578835759 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c009420 con 0x7f969806d3c0 2024-03-28T16:48:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f486c228640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f484c001660 0x7f484c003b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f484c006b10 con 0x7f484c001660 2024-03-28T16:48:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f48697c3640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4848035a10 0x7f4848037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9697fff640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f96981c56e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9688001b40 tx=0x7f968800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f486a7c5640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f484c001660 0x7f484c003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f968800aea0 con 0x7f969806d3c0 2024-03-28T16:48:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96880138e0 con 0x7f969806d3c0 2024-03-28T16:48:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9688013be0 con 0x7f969806d3c0 2024-03-28T16:48:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff4bc0b8d30 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4b3fff640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff4b8035ae0 0x7ff4b8037fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7ff4bc0e2cd0 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4b3fff640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff4b803b2e0 0x7ff4b803d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4b803ddf0 con 0x7ff4b803b2e0 2024-03-28T16:48:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4bc0e3000 con 0x7ff4cc06d1e0 2024-03-28T16:48:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969812b580 con 0x7f969806d3c0 2024-03-28T16:48:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969812bf30 con 0x7f969806d3c0 2024-03-28T16:48:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4cb7fe640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff4b803b2e0 0x7ff4b803d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4ca7fc640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff4b8035ae0 0x7ff4b8037fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f48697c3640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4848035a10 0x7f4848037ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f48540ad000 tx=0x7f48540a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f486a7c5640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f484c001660 0x7f484c003b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.737 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9698099110 con 0x7f969806d3c0 2024-03-28T16:48:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9688013d40 con 0x7f969806d3c0 2024-03-28T16:48:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4cb7fe640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff4b803b2e0 0x7ff4b803d720 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7ff4ca7fc640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff4b8035ae0 0x7ff4b8037fa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0001610 tx=0x7ff4c0007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9684035a10 0x7f9684037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f968804fe20 con 0x7f969806d3c0 2024-03-28T16:48:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f968403b190 0x7f968403d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f968403dca0 con 0x7f968403b190 2024-03-28T16:48:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96880553f0 con 0x7f969806d3c0 2024-03-28T16:48:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f96977fe640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9684035a10 0x7f9684037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.739 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f969cc6b640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f968403b190 0x7f968403d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f969cc6b640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f968403b190 0x7f968403d5d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f96977fe640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9684035a10 0x7f9684037ed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f968c0ad000 tx=0x7f968c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.714+0000 7f485a7fc640 1 -- 172.21.15.119:0/2580175610 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f484c006b10 con 0x7f484c001660 2024-03-28T16:48:49.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.718+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff4b803ddf0 con 0x7ff4b803b2e0 2024-03-28T16:48:49.740 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.722+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f968403dca0 con 0x7f968403b190 2024-03-28T16:48:49.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c5012fff0 con 0x7f8c3403b000 2024-03-28T16:48:49.741 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c46ffd640 1 -- 172.21.15.119:0/2813712886 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8c5012fff0 con 0x7f8c3403b000 2024-03-28T16:48:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f8c3403b000 msgr2=0x7f8c3403d440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f8c3403b000 0x7f8c3403d440 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c340359f0 msgr2=0x7f8c34037eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c340359f0 0x7f8c34037eb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c40001610 tx=0x7f8c40007ad0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 msgr2=0x7f8c501c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c501c5740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0ae040 tx=0x7f8c4c019110 comp rx=0 tx=0).stop 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 shutdown_connections 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f8c3403b000 0x7f8c3403d440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c340359f0 0x7f8c34037eb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 --2- 172.21.15.119:0/2813712886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c5012a900 0x7f8c501c5740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 >> 172.21.15.119:0/2813712886 conn(0x7f8c5009d4e0 msgr2=0x7f8c501221a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 shutdown_connections 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.726+0000 7f8c58308640 1 -- 172.21.15.119:0/2813712886 wait complete. 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff4a4002be0 con 0x7ff4b803b2e0 2024-03-28T16:48:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b3fff640 1 -- 172.21.15.119:0/2381842148 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff4a4002be0 con 0x7ff4b803b2e0 2024-03-28T16:48:49.744 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff4b803b2e0 msgr2=0x7ff4b803d720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff4b803b2e0 0x7ff4b803d720 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff4b8035ae0 msgr2=0x7ff4b8037fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff4b8035ae0 0x7ff4b8037fa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0001610 tx=0x7ff4c0007ad0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06d1e0 msgr2=0x7ff4cc1329f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06d1e0 0x7ff4cc1329f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0af040 tx=0x7ff4bc01abb0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 shutdown_connections 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff4b803b2e0 0x7ff4b803d720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff4b8035ae0 0x7ff4b8037fa0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 --2- 172.21.15.119:0/2381842148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4cc06d1e0 0x7ff4cc1329f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 >> 172.21.15.119:0/2381842148 conn(0x7ff4cc09c330 msgr2=0x7ff4cc12c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 shutdown_connections 2024-03-28T16:48:49.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.730+0000 7ff4b1ffb640 1 -- 172.21.15.119:0/2381842148 wait complete. 2024-03-28T16:48:49.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f484c005bf0 con 0x7f484c001660 2024-03-28T16:48:49.746 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f485a7fc640 1 -- 172.21.15.119:0/2580175610 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f484c005bf0 con 0x7f484c001660 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f484c001660 msgr2=0x7f484c003b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f484c001660 0x7f484c003b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4848035a10 msgr2=0x7f4848037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4848035a10 0x7f4848037ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f48540ad000 tx=0x7f48540a6000 comp rx=0 tx=0).stop 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409a3f0 msgr2=0x7f48641bdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409a3f0 0x7f48641bdf70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f485c00c840 tx=0x7f485c00cd00 comp rx=0 tx=0).stop 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 shutdown_connections 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f484c001660 0x7f484c003b20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4848035a10 0x7f4848037ed0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 --2- 172.21.15.119:0/2580175610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f486409a3f0 0x7f48641bdf70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 >> 172.21.15.119:0/2580175610 conn(0x7f4864070240 msgr2=0x7f4864128bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 shutdown_connections 2024-03-28T16:48:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f486c228640 1 -- 172.21.15.119:0/2580175610 wait complete. 2024-03-28T16:48:49.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9698098f60 con 0x7f968403b190 2024-03-28T16:48:49.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.734+0000 7f9694ff9640 1 -- 172.21.15.119:0/578835759 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9698098f60 con 0x7f968403b190 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f968403b190 msgr2=0x7f968403d5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f968403b190 0x7f968403d5d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9684035a10 msgr2=0x7f9684037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9684035a10 0x7f9684037ed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f968c0ad000 tx=0x7f968c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 msgr2=0x7f96981c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f96981c56e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9688001b40 tx=0x7f968800a430 comp rx=0 tx=0).stop 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 shutdown_connections 2024-03-28T16:48:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f968403b190 0x7f968403d5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9684035a10 0x7f9684037ed0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 --2- 172.21.15.119:0/578835759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969806d3c0 0x7f96981c56e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 >> 172.21.15.119:0/578835759 conn(0x7f969809d1d0 msgr2=0x7f9698098650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 shutdown_connections 2024-03-28T16:48:49.750 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.738+0000 7f969e6ce640 1 -- 172.21.15.119:0/578835759 wait complete. 2024-03-28T16:48:49.754 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:49.754 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T16:48:49.755 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:49.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T16:48:49.758 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:49.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T16:48:49.761 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:49.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T16:48:49.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f345c226640 1 Processor -- start 2024-03-28T16:48:49.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f345c226640 1 -- start start 2024-03-28T16:48:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6ce16b640 1 Processor -- start 2024-03-28T16:48:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6ce16b640 1 -- start start 2024-03-28T16:48:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f345c226640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3454096410 0x7f34540967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f345c226640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345406f950 con 0x7f345406dcc0 2024-03-28T16:48:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f345c226640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345406f390 con 0x7f3454096410 2024-03-28T16:48:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6ce16b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa6c8095640 0x7fa6c806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f34597c1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f345406dcc0 0x7f345406e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41628/0 (socket says 172.21.15.119:41628) 2024-03-28T16:48:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f34597c1640 1 -- 172.21.15.119:0/1002852381 learned_addr learned my addr 172.21.15.119:0/1002852381 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6c6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa6c8095640 0x7fa6c806cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6c6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa6c8095640 0x7fa6c806cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42628/0 (socket says 172.21.15.119:42628) 2024-03-28T16:48:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6c6ffd640 1 -- 172.21.15.119:0/3526394887 learned_addr learned my addr 172.21.15.119:0/3526394887 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f3459fc2640 1 --2- 172.21.15.119:0/1002852381 >> v2:172.21.15.119:3300/0 conn(0x7f3454096410 0x7f34540967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6ce16b640 1 -- 172.21.15.119:0/3526394887 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c8096290 con 0x7fa6c806c0d0 2024-03-28T16:48:49.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6ce16b640 1 -- 172.21.15.119:0/3526394887 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c809df10 con 0x7fa6c8095640 2024-03-28T16:48:49.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6c6ffd640 1 -- 172.21.15.119:0/3526394887 >> v1:172.21.15.119:6789/0 conn(0x7fa6c806c0d0 legacy=0x7fa6c806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7fa6c6ffd640 1 -- 172.21.15.119:0/3526394887 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c809a470 con 0x7fa6c8095640 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f3459fc2640 1 -- 172.21.15.119:0/1002852381 >> v1:172.21.15.119:6789/0 conn(0x7f345406dcc0 legacy=0x7f345406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.858+0000 7f3459fc2640 1 -- 172.21.15.119:0/1002852381 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345409d7d0 con 0x7f3454096410 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3459fc2640 1 --2- 172.21.15.119:0/1002852381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3454096410 0x7f34540967f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3450009740 tx=0x7f345001e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbabf3f73d900055 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3458fc0640 1 -- 172.21.15.119:0/1002852381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34500ad530 con 0x7f3454096410 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3458fc0640 1 -- 172.21.15.119:0/1002852381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34500a9050 con 0x7f3454096410 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3458fc0640 1 -- 172.21.15.119:0/1002852381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34500ad7e0 con 0x7f3454096410 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 -- 172.21.15.119:0/1002852381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3454096410 msgr2=0x7f34540967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 --2- 172.21.15.119:0/1002852381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3454096410 0x7f34540967f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3450009740 tx=0x7f345001e410 comp rx=0 tx=0).stop 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 -- 172.21.15.119:0/1002852381 shutdown_connections 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 --2- 172.21.15.119:0/1002852381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3454096410 0x7f34540967f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 -- 172.21.15.119:0/1002852381 >> 172.21.15.119:0/1002852381 conn(0x7f345409c8e0 msgr2=0x7f345409ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6c6ffd640 1 --2- 172.21.15.119:0/3526394887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c8095640 0x7fa6c806cbc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0097f0 tx=0x7fa6bc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ea3683379c2a78 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 -- 172.21.15.119:0/1002852381 shutdown_connections 2024-03-28T16:48:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 -- 172.21.15.119:0/1002852381 wait complete. 2024-03-28T16:48:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6c67fc640 1 -- 172.21.15.119:0/3526394887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc0ab530 con 0x7fa6c8095640 2024-03-28T16:48:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6c67fc640 1 -- 172.21.15.119:0/3526394887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc0a7050 con 0x7fa6c8095640 2024-03-28T16:48:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6c67fc640 1 -- 172.21.15.119:0/3526394887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc0ab7e0 con 0x7fa6c8095640 2024-03-28T16:48:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 Processor -- start 2024-03-28T16:48:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 -- start start 2024-03-28T16:48:49.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6ce16b640 1 -- 172.21.15.119:0/3526394887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c8095640 msgr2=0x7fa6c806cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6ce16b640 1 --2- 172.21.15.119:0/3526394887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c8095640 0x7fa6c806cbc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0097f0 tx=0x7fa6bc01c430 comp rx=0 tx=0).stop 2024-03-28T16:48:49.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6ce16b640 1 -- 172.21.15.119:0/3526394887 shutdown_connections 2024-03-28T16:48:49.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6ce16b640 1 --2- 172.21.15.119:0/3526394887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c8095640 0x7fa6c806cbc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6ce16b640 1 -- 172.21.15.119:0/3526394887 >> 172.21.15.119:0/3526394887 conn(0x7fa6c809d600 msgr2=0x7fa6c8099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f345406dcc0 0x7f34541c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f345c226640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34541269e0 con 0x7f345406dcc0 2024-03-28T16:48:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6ce16b640 1 -- 172.21.15.119:0/3526394887 shutdown_connections 2024-03-28T16:48:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7fa6ce16b640 1 -- 172.21.15.119:0/3526394887 wait complete. 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3459fc2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f345406dcc0 0x7f34541c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3459fc2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f345406dcc0 0x7f34541c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42642/0 (socket says 172.21.15.119:42642) 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3459fc2640 1 -- 172.21.15.119:0/4135901660 learned_addr learned my addr 172.21.15.119:0/4135901660 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3459fc2640 1 -- 172.21.15.119:0/4135901660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3450009420 con 0x7f345406dcc0 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f3459fc2640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f345406dcc0 0x7f34541c6030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f345001b020 tx=0x7f345001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.862+0000 7f344a7fc640 1 -- 172.21.15.119:0/4135901660 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34500bc070 con 0x7f345406dcc0 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f344a7fc640 1 -- 172.21.15.119:0/4135901660 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34500ade80 con 0x7f345406dcc0 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f345c226640 1 -- 172.21.15.119:0/4135901660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3454096f40 con 0x7f345406dcc0 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f344a7fc640 1 -- 172.21.15.119:0/4135901660 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34500b85e0 con 0x7f345406dcc0 2024-03-28T16:48:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f345c226640 1 -- 172.21.15.119:0/4135901660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34541260d0 con 0x7f345406dcc0 2024-03-28T16:48:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6ce16b640 1 Processor -- start 2024-03-28T16:48:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6ce16b640 1 -- start start 2024-03-28T16:48:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6ce16b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c806c0d0 0x7fa6c8122290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6ce16b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c8124120 con 0x7fa6c806c0d0 2024-03-28T16:48:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6c77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c806c0d0 0x7fa6c8122290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6c77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c806c0d0 0x7fa6c8122290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42648/0 (socket says 172.21.15.119:42648) 2024-03-28T16:48:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6c77fe640 1 -- 172.21.15.119:0/1050797112 learned_addr learned my addr 172.21.15.119:0/1050797112 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3430005090 con 0x7f345406dcc0 2024-03-28T16:48:49.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6c77fe640 1 -- 172.21.15.119:0/1050797112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0094d0 con 0x7fa6c806c0d0 2024-03-28T16:48:49.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6c77fe640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c806c0d0 0x7fa6c8122290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c480 tx=0x7fa6b800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.873 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f344a7fc640 1 -- 172.21.15.119:0/4135901660 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f34500b4070 con 0x7f345406dcc0 2024-03-28T16:48:49.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6affff640 1 -- 172.21.15.119:0/1050797112 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6b8005150 con 0x7fa6c806c0d0 2024-03-28T16:48:49.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6affff640 1 -- 172.21.15.119:0/1050797112 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b80052b0 con 0x7fa6c806c0d0 2024-03-28T16:48:49.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f344a7fc640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3438035a10 0x7f3438037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f344a7fc640 1 -- 172.21.15.119:0/4135901660 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f34500bf320 con 0x7f345406dcc0 2024-03-28T16:48:49.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f34597c1640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3438035a10 0x7f3438037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6affff640 1 -- 172.21.15.119:0/1050797112 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6b8007030 con 0x7fa6c806c0d0 2024-03-28T16:48:49.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c8123e50 con 0x7fa6c806c0d0 2024-03-28T16:48:49.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c8123390 con 0x7fa6c806c0d0 2024-03-28T16:48:49.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.866+0000 7f34597c1640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3438035a10 0x7f3438037ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3444001800 tx=0x7f3444007a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.870+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8062860 con 0x7fa6c806c0d0 2024-03-28T16:48:49.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.870+0000 7f344a7fc640 1 -- 172.21.15.119:0/4135901660 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34500e4030 con 0x7f345406dcc0 2024-03-28T16:48:49.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.870+0000 7fa6affff640 1 -- 172.21.15.119:0/1050797112 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa6b8007190 con 0x7fa6c806c0d0 2024-03-28T16:48:49.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.870+0000 7fa6affff640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6b40359f0 0x7fa6b4037eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.870+0000 7fa6affff640 1 -- 172.21.15.119:0/1050797112 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fa6b80474b0 con 0x7fa6c806c0d0 2024-03-28T16:48:49.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.870+0000 7fa6c6ffd640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6b40359f0 0x7fa6b4037eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.874+0000 7fa6c6ffd640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6b40359f0 0x7fa6b4037eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa6c804f4a0 tx=0x7fa6bc0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.874+0000 7fa6affff640 1 -- 172.21.15.119:0/1050797112 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6b8007440 con 0x7fa6c806c0d0 2024-03-28T16:48:49.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f505bf33640 1 Processor -- start 2024-03-28T16:48:49.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0cd6f640 1 Processor -- start 2024-03-28T16:48:49.908 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0cd6f640 1 -- start start 2024-03-28T16:48:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f505bf33640 1 -- start start 2024-03-28T16:48:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f505bf33640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f505412db20 0x7f505412ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f505bf33640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5054072990 con 0x7f505412acc0 2024-03-28T16:48:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f505bf33640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505412b7b0 con 0x7f505412db20 2024-03-28T16:48:49.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0cd6f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a0806cbb0 0x7f1a0806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0cd6f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0809d7d0 con 0x7f1a0806d4d0 2024-03-28T16:48:49.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0cd6f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a08070da0 con 0x7f1a0806cbb0 2024-03-28T16:48:49.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f5059ccf640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f505412acc0 0x7f505412b0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41636/0 (socket says 172.21.15.119:41636) 2024-03-28T16:48:49.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f50594ce640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f505412db20 0x7f505412ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.910 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f5059ccf640 1 -- 172.21.15.119:0/3852292222 learned_addr learned my addr 172.21.15.119:0/3852292222 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a0806cbb0 0x7f1a0806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a0806cbb0 0x7f1a0806cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42656/0 (socket says 172.21.15.119:42656) 2024-03-28T16:48:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0659c640 1 -- 172.21.15.119:0/2976440975 learned_addr learned my addr 172.21.15.119:0/2976440975 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f5058ccd640 1 -- 172.21.15.119:0/3852292222 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001092 0 0) 0x7f5054072990 con 0x7f505412acc0 2024-03-28T16:48:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f5058ccd640 1 -- 172.21.15.119:0/3852292222 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5040003530 con 0x7f505412acc0 2024-03-28T16:48:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0659c640 1 -- 172.21.15.119:0/2976440975 >> v1:172.21.15.119:6789/0 conn(0x7f1a0806d4d0 legacy=0x7f1a080960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f1a0659c640 1 -- 172.21.15.119:0/2976440975 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08072990 con 0x7f1a0806cbb0 2024-03-28T16:48:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f50594ce640 1 -- 172.21.15.119:0/3852292222 >> v1:172.21.15.119:6789/0 conn(0x7f505412acc0 legacy=0x7f505412b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.906+0000 7f50594ce640 1 -- 172.21.15.119:0/3852292222 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505412a7b0 con 0x7f505412db20 2024-03-28T16:48:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f50594ce640 1 --2- 172.21.15.119:0/3852292222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412db20 0x7f505412ff10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f504400b9e0 tx=0x7f504401ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3839147d75fda6c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0659c640 1 --2- 172.21.15.119:0/2976440975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806cf90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a000097f0 tx=0x7f1a0001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1864b12495b85f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5058ccd640 1 -- 172.21.15.119:0/3852292222 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50440a78d0 con 0x7f505412db20 2024-03-28T16:48:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0559a640 1 -- 172.21.15.119:0/2976440975 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a000ab530 con 0x7f1a0806cbb0 2024-03-28T16:48:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5058ccd640 1 -- 172.21.15.119:0/3852292222 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50440a7a30 con 0x7f505412db20 2024-03-28T16:48:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5058ccd640 1 -- 172.21.15.119:0/3852292222 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50440a7cf0 con 0x7f505412db20 2024-03-28T16:48:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0559a640 1 -- 172.21.15.119:0/2976440975 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a000a7050 con 0x7f1a0806cbb0 2024-03-28T16:48:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0559a640 1 -- 172.21.15.119:0/2976440975 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a000ab7e0 con 0x7f1a0806cbb0 2024-03-28T16:48:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- 172.21.15.119:0/3852292222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412db20 msgr2=0x7f505412ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 --2- 172.21.15.119:0/3852292222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412db20 0x7f505412ff10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f504400b9e0 tx=0x7f504401ea40 comp rx=0 tx=0).stop 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2976440975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 msgr2=0x7f1a0806cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 --2- 172.21.15.119:0/2976440975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806cf90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a000097f0 tx=0x7f1a0001c430 comp rx=0 tx=0).stop 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2976440975 shutdown_connections 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 --2- 172.21.15.119:0/2976440975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806cf90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2976440975 >> 172.21.15.119:0/2976440975 conn(0x7f1a0809c8e0 msgr2=0x7f1a0809ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- 172.21.15.119:0/3852292222 shutdown_connections 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 --2- 172.21.15.119:0/3852292222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412db20 0x7f505412ff10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- 172.21.15.119:0/3852292222 >> 172.21.15.119:0/3852292222 conn(0x7f505409d600 msgr2=0x7f5054099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2976440975 shutdown_connections 2024-03-28T16:48:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2976440975 wait complete. 2024-03-28T16:48:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- 172.21.15.119:0/3852292222 shutdown_connections 2024-03-28T16:48:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- 172.21.15.119:0/3852292222 wait complete. 2024-03-28T16:48:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 Processor -- start 2024-03-28T16:48:49.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 Processor -- start 2024-03-28T16:48:49.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- start start 2024-03-28T16:48:49.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412acc0 0x7f50541c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5054127990 con 0x7f505412acc0 2024-03-28T16:48:49.917 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 -- start start 2024-03-28T16:48:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0cd6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0806f6b0 con 0x7f1a0806cbb0 2024-03-28T16:48:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5059ccf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412acc0 0x7f50541c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5059ccf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412acc0 0x7f50541c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42672/0 (socket says 172.21.15.119:42672) 2024-03-28T16:48:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5059ccf640 1 -- 172.21.15.119:0/605744847 learned_addr learned my addr 172.21.15.119:0/605744847 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42682/0 (socket says 172.21.15.119:42682) 2024-03-28T16:48:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0659c640 1 -- 172.21.15.119:0/2529578089 learned_addr learned my addr 172.21.15.119:0/2529578089 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5059ccf640 1 -- 172.21.15.119:0/605744847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504400b6c0 con 0x7f505412acc0 2024-03-28T16:48:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f5059ccf640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412acc0 0x7f50541c54b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f505000c450 tx=0x7f505000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.919 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f504a7fc640 1 -- 172.21.15.119:0/605744847 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5050005c20 con 0x7f505412acc0 2024-03-28T16:48:49.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f1a0659c640 1 -- 172.21.15.119:0/2529578089 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000094d0 con 0x7f1a0806cbb0 2024-03-28T16:48:49.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f504a7fc640 1 -- 172.21.15.119:0/605744847 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050005d80 con 0x7f505412acc0 2024-03-28T16:48:49.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50541267c0 con 0x7f505412acc0 2024-03-28T16:48:49.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f504a7fc640 1 -- 172.21.15.119:0/605744847 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50500071a0 con 0x7f505412acc0 2024-03-28T16:48:49.920 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.910+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5054126bb0 con 0x7f505412acc0 2024-03-28T16:48:49.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f1a0659c640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806dcc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a0001a060 tx=0x7f1a0001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f19feffd640 1 -- 172.21.15.119:0/2529578089 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a000b6420 con 0x7f1a0806cbb0 2024-03-28T16:48:49.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a08094080 con 0x7f1a0806cbb0 2024-03-28T16:48:49.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a08128a30 con 0x7f1a0806cbb0 2024-03-28T16:48:49.921 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f19feffd640 1 -- 172.21.15.119:0/2529578089 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a000b6580 con 0x7f1a0806cbb0 2024-03-28T16:48:49.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f19feffd640 1 -- 172.21.15.119:0/2529578089 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a000b6420 con 0x7f1a0806cbb0 2024-03-28T16:48:49.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5054062860 con 0x7f505412acc0 2024-03-28T16:48:49.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f504a7fc640 1 -- 172.21.15.119:0/605744847 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f5050007300 con 0x7f505412acc0 2024-03-28T16:48:49.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f504a7fc640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5038035ab0 0x7f5038037f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.922 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f504a7fc640 1 -- 172.21.15.119:0/605744847 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f50500472b0 con 0x7f505412acc0 2024-03-28T16:48:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec005090 con 0x7f1a0806cbb0 2024-03-28T16:48:49.923 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f50594ce640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5038035ab0 0x7f5038037f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f19feffd640 1 -- 172.21.15.119:0/2529578089 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f1a000b2040 con 0x7f1a0806cbb0 2024-03-28T16:48:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f19feffd640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f19e8035a60 0x7f19e8037f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f50594ce640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5038035ab0 0x7f5038037f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f504400b9e0 tx=0x7f50440a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.924 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f19feffd640 1 -- 172.21.15.119:0/2529578089 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f1a000b4db0 con 0x7f1a0806cbb0 2024-03-28T16:48:49.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f1a05d9b640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f19e8035a60 0x7f19e8037f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:49.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.914+0000 7f1a05d9b640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f19e8035a60 0x7f19e8037f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f19f0001610 tx=0x7f19f00079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:49.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.918+0000 7f504a7fc640 1 -- 172.21.15.119:0/605744847 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5050010c20 con 0x7f505412acc0 2024-03-28T16:48:49.925 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:49.922+0000 7f19feffd640 1 -- 172.21.15.119:0/2529578089 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a000b6db0 con 0x7f1a0806cbb0 2024-03-28T16:48:50.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5054127790 con 0x7f505412acc0 2024-03-28T16:48:50.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f504a7fc640 1 -- 172.21.15.119:0/605744847 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5050017020 con 0x7f505412acc0 2024-03-28T16:48:50.075 INFO:teuthology.orchestra.run.smithi119.stdout:38654705667 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5038035ab0 msgr2=0x7f5038037f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5038035ab0 0x7f5038037f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f504400b9e0 tx=0x7f50440a9070 comp rx=0 tx=0).stop 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412acc0 msgr2=0x7f50541c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412acc0 0x7f50541c54b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f505000c450 tx=0x7f505000c910 comp rx=0 tx=0).stop 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 shutdown_connections 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5038035ab0 0x7f5038037f70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 --2- 172.21.15.119:0/605744847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505412acc0 0x7f50541c54b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 >> 172.21.15.119:0/605744847 conn(0x7f505409d600 msgr2=0x7f5054129d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:50.077 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 shutdown_connections 2024-03-28T16:48:50.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f505bf33640 1 -- 172.21.15.119:0/605744847 wait complete. 2024-03-28T16:48:50.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f19ec005610 con 0x7f1a0806cbb0 2024-03-28T16:48:50.078 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.074+0000 7f19feffd640 1 -- 172.21.15.119:0/2529578089 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a000b6db0 con 0x7f1a0806cbb0 2024-03-28T16:48:50.078 INFO:teuthology.orchestra.run.smithi119.stdout:38654705667 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f19e8035a60 msgr2=0x7f19e8037f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f19e8035a60 0x7f19e8037f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f19f0001610 tx=0x7f19f00079e0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 msgr2=0x7f1a0806dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806dcc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a0001a060 tx=0x7f1a0001ac70 comp rx=0 tx=0).stop 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 shutdown_connections 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f19e8035a60 0x7f19e8037f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 --2- 172.21.15.119:0/2529578089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a0806cbb0 0x7f1a0806dcc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 >> 172.21.15.119:0/2529578089 conn(0x7f1a0809c8e0 msgr2=0x7f1a08098170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 shutdown_connections 2024-03-28T16:48:50.082 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.078+0000 7f1a0cd6f640 1 -- 172.21.15.119:0/2529578089 wait complete. 2024-03-28T16:48:50.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.3 2024-03-28T16:48:50.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.1 2024-03-28T16:48:50.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.142+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3430005340 con 0x7f345406dcc0 2024-03-28T16:48:50.146 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f344a7fc640 1 -- 172.21.15.119:0/4135901660 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34500b2030 con 0x7f345406dcc0 2024-03-28T16:48:50.146 INFO:teuthology.orchestra.run.smithi119.stdout:38654705667 2024-03-28T16:48:50.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3438035a10 msgr2=0x7f3438037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3438035a10 0x7f3438037ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3444001800 tx=0x7f3444007a10 comp rx=0 tx=0).stop 2024-03-28T16:48:50.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f345406dcc0 msgr2=0x7f34541c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f345406dcc0 0x7f34541c6030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f345001b020 tx=0x7f345001cc50 comp rx=0 tx=0).stop 2024-03-28T16:48:50.149 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 shutdown_connections 2024-03-28T16:48:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3438035a10 0x7f3438037ed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 --2- 172.21.15.119:0/4135901660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f345406dcc0 0x7f34541c6030 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 >> 172.21.15.119:0/4135901660 conn(0x7f345409c8e0 msgr2=0x7f345408c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 shutdown_connections 2024-03-28T16:48:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.146+0000 7f3437fff640 1 -- 172.21.15.119:0/4135901660 wait complete. 2024-03-28T16:48:50.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.150+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa6c8125250 con 0x7fa6c806c0d0 2024-03-28T16:48:50.151 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.150+0000 7fa6affff640 1 -- 172.21.15.119:0/1050797112 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6b801a3a0 con 0x7fa6c806c0d0 2024-03-28T16:48:50.151 INFO:teuthology.orchestra.run.smithi119.stdout:38654705667 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6b40359f0 msgr2=0x7fa6b4037eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6b40359f0 0x7fa6b4037eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa6c804f4a0 tx=0x7fa6bc0a6000 comp rx=0 tx=0).stop 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c806c0d0 msgr2=0x7fa6c8122290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c806c0d0 0x7fa6c8122290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800c480 tx=0x7fa6b800c940 comp rx=0 tx=0).stop 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 shutdown_connections 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6b40359f0 0x7fa6b4037eb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 --2- 172.21.15.119:0/1050797112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c806c0d0 0x7fa6c8122290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 >> 172.21.15.119:0/1050797112 conn(0x7fa6c809d600 msgr2=0x7fa6c8090730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 shutdown_connections 2024-03-28T16:48:50.156 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:50.154+0000 7fa6ce16b640 1 -- 172.21.15.119:0/1050797112 wait complete. 2024-03-28T16:48:50.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2024-03-28T16:48:50.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.0 2024-03-28T16:48:51.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T16:48:51.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T16:48:51.173 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T16:48:51.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T16:48:51.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f134caef640 1 Processor -- start 2024-03-28T16:48:51.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f134caef640 1 -- start start 2024-03-28T16:48:51.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f134caef640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f134812ee50 0x7f1348131240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f134caef640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134806c0d0 con 0x7f134806c490 2024-03-28T16:48:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f134caef640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1348093fb0 con 0x7f134812ee50 2024-03-28T16:48:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f1345d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f134812ee50 0x7f1348131240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f1345d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f134812ee50 0x7f1348131240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42686/0 (socket says 172.21.15.119:42686) 2024-03-28T16:48:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f1345d9b640 1 -- 172.21.15.119:0/2019822728 learned_addr learned my addr 172.21.15.119:0/2019822728 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f1345d9b640 1 -- 172.21.15.119:0/2019822728 >> v1:172.21.15.119:6789/0 conn(0x7f134806c490 legacy=0x7f134806c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.218+0000 7f1345d9b640 1 -- 172.21.15.119:0/2019822728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134806cf80 con 0x7f134812ee50 2024-03-28T16:48:51.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f1345d9b640 1 --2- 172.21.15.119:0/2019822728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134812ee50 0x7f1348131240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f133c0097f0 tx=0x7f133c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da8aef9c44d96e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134559a640 1 -- 172.21.15.119:0/2019822728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f133c0ac560 con 0x7f134812ee50 2024-03-28T16:48:51.224 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134559a640 1 -- 172.21.15.119:0/2019822728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c0ac6c0 con 0x7f134812ee50 2024-03-28T16:48:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134559a640 1 -- 172.21.15.119:0/2019822728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f133c0ac980 con 0x7f134812ee50 2024-03-28T16:48:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 -- 172.21.15.119:0/2019822728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134812ee50 msgr2=0x7f1348131240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 --2- 172.21.15.119:0/2019822728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134812ee50 0x7f1348131240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f133c0097f0 tx=0x7f133c01c430 comp rx=0 tx=0).stop 2024-03-28T16:48:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 -- 172.21.15.119:0/2019822728 shutdown_connections 2024-03-28T16:48:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 --2- 172.21.15.119:0/2019822728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134812ee50 0x7f1348131240 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 -- 172.21.15.119:0/2019822728 >> 172.21.15.119:0/2019822728 conn(0x7f134809b1d0 msgr2=0x7f134809b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 -- 172.21.15.119:0/2019822728 shutdown_connections 2024-03-28T16:48:51.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 -- 172.21.15.119:0/2019822728 wait complete. 2024-03-28T16:48:51.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 Processor -- start 2024-03-28T16:48:51.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 -- start start 2024-03-28T16:48:51.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134806c490 0x7f1348127a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.230 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134caef640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348129300 con 0x7f134806c490 2024-03-28T16:48:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134806c490 0x7f1348127a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134806c490 0x7f1348127a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42702/0 (socket says 172.21.15.119:42702) 2024-03-28T16:48:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.222+0000 7f134659c640 1 -- 172.21.15.119:0/1789307286 learned_addr learned my addr 172.21.15.119:0/1789307286 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f134659c640 1 -- 172.21.15.119:0/1789307286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0094d0 con 0x7f134806c490 2024-03-28T16:48:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f134659c640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134806c490 0x7f1348127a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f133800c480 tx=0x7f133800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.231 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1336ffd640 1 -- 172.21.15.119:0/1789307286 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1338005150 con 0x7f134806c490 2024-03-28T16:48:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1336ffd640 1 -- 172.21.15.119:0/1789307286 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13380052b0 con 0x7f134806c490 2024-03-28T16:48:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1348092de0 con 0x7f134806c490 2024-03-28T16:48:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1336ffd640 1 -- 172.21.15.119:0/1789307286 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1338007030 con 0x7f134806c490 2024-03-28T16:48:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1348128b50 con 0x7f134806c490 2024-03-28T16:48:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1348092fb0 con 0x7f134806c490 2024-03-28T16:48:51.232 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1336ffd640 1 -- 172.21.15.119:0/1789307286 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f1338007190 con 0x7f134806c490 2024-03-28T16:48:51.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1336ffd640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1328035a10 0x7f1328037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1336ffd640 1 -- 172.21.15.119:0/1789307286 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f133804f350 con 0x7f134806c490 2024-03-28T16:48:51.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1345d9b640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1328035a10 0x7f1328037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.226+0000 7f1345d9b640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1328035a10 0x7f1328037ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f133c0ae000 tx=0x7f133c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.237 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.234+0000 7f1336ffd640 1 -- 172.21.15.119:0/1789307286 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f133804c060 con 0x7f134806c490 2024-03-28T16:48:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f091a768640 1 Processor -- start 2024-03-28T16:48:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f091a768640 1 -- start start 2024-03-28T16:48:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f091a768640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f091406c0d0 0x7f091406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f091a768640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0914072550 con 0x7f091406c9f0 2024-03-28T16:48:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f091a768640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0914068650 con 0x7f091406c0d0 2024-03-28T16:48:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f09137fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f091406c9f0 0x7f09141321a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41666/0 (socket says 172.21.15.119:41666) 2024-03-28T16:48:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f09137fe640 1 -- 172.21.15.119:0/1757365739 learned_addr learned my addr 172.21.15.119:0/1757365739 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f0913fff640 1 --2- 172.21.15.119:0/1757365739 >> v2:172.21.15.119:3300/0 conn(0x7f091406c0d0 0x7f091406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f0913fff640 1 -- 172.21.15.119:0/1757365739 >> v1:172.21.15.119:6789/0 conn(0x7f091406c9f0 legacy=0x7f09141321a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.270+0000 7f0913fff640 1 -- 172.21.15.119:0/1757365739 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914093f80 con 0x7f091406c0d0 2024-03-28T16:48:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f0913fff640 1 --2- 172.21.15.119:0/1757365739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f091406c4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0904009740 tx=0x7f090401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97e64c3ac1ceca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f0912ffd640 1 -- 172.21.15.119:0/1757365739 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09040ab530 con 0x7f091406c0d0 2024-03-28T16:48:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f0912ffd640 1 -- 172.21.15.119:0/1757365739 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09040a7050 con 0x7f091406c0d0 2024-03-28T16:48:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f0912ffd640 1 -- 172.21.15.119:0/1757365739 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09040ab7e0 con 0x7f091406c0d0 2024-03-28T16:48:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 -- 172.21.15.119:0/1757365739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 msgr2=0x7f091406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 --2- 172.21.15.119:0/1757365739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f091406c4b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0904009740 tx=0x7f090401c400 comp rx=0 tx=0).stop 2024-03-28T16:48:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 -- 172.21.15.119:0/1757365739 shutdown_connections 2024-03-28T16:48:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 --2- 172.21.15.119:0/1757365739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f091406c4b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 -- 172.21.15.119:0/1757365739 >> 172.21.15.119:0/1757365739 conn(0x7f091409d600 msgr2=0x7f0914099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 -- 172.21.15.119:0/1757365739 shutdown_connections 2024-03-28T16:48:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 -- 172.21.15.119:0/1757365739 wait complete. 2024-03-28T16:48:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 Processor -- start 2024-03-28T16:48:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 -- start start 2024-03-28T16:48:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f09141308c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f091a768640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914123240 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f0913fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f09141308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f0913fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f09141308c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42712/0 (socket says 172.21.15.119:42712) 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.274+0000 7f0913fff640 1 -- 172.21.15.119:0/3979573160 learned_addr learned my addr 172.21.15.119:0/3979573160 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0913fff640 1 -- 172.21.15.119:0/3979573160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904009420 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0913fff640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f09141308c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f09040ae040 tx=0x7f09040191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0910ff9640 1 -- 172.21.15.119:0/3979573160 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09040abe50 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0910ff9640 1 -- 172.21.15.119:0/3979573160 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09040b4410 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f091a768640 1 -- 172.21.15.119:0/3979573160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0914124200 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0910ff9640 1 -- 172.21.15.119:0/3979573160 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09040b46f0 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f091a768640 1 -- 172.21.15.119:0/3979573160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0914122c00 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0910ff9640 1 -- 172.21.15.119:0/3979573160 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f09040b2040 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0914062860 con 0x7f091406c0d0 2024-03-28T16:48:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0910ff9640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0900035a10 0x7f0900037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f0910ff9640 1 -- 172.21.15.119:0/3979573160 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f09040e2040 con 0x7f091406c0d0 2024-03-28T16:48:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f09137fe640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0900035a10 0x7f0900037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.278+0000 7f09137fe640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0900035a10 0x7f0900037ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0908001610 tx=0x7f0908007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.286+0000 7f0910ff9640 1 -- 172.21.15.119:0/3979573160 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09040bb950 con 0x7f091406c0d0 2024-03-28T16:48:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f98811fc640 1 Processor -- start 2024-03-28T16:48:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f98811fc640 1 -- start start 2024-03-28T16:48:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f98811fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f987c099560 0x7f987c130fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f98811fc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987c072150 con 0x7f987c098a70 2024-03-28T16:48:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f98811fc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987c09b060 con 0x7f987c099560 2024-03-28T16:48:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f987a59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f987c099560 0x7f987c130fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f987a59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f987c099560 0x7f987c130fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42720/0 (socket says 172.21.15.119:42720) 2024-03-28T16:48:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.298+0000 7f987a59c640 1 -- 172.21.15.119:0/357842733 learned_addr learned my addr 172.21.15.119:0/357842733 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f987a59c640 1 -- 172.21.15.119:0/357842733 >> v1:172.21.15.119:6789/0 conn(0x7f987c098a70 legacy=0x7f987c098e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.304 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f987a59c640 1 -- 172.21.15.119:0/357842733 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c09a910 con 0x7f987c099560 2024-03-28T16:48:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f987a59c640 1 --2- 172.21.15.119:0/357842733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c099560 0x7f987c130fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98700097f0 tx=0x7f987001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2709a96e3a7eef23 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f9879d9b640 1 -- 172.21.15.119:0/357842733 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98700ab520 con 0x7f987c099560 2024-03-28T16:48:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f9879d9b640 1 -- 172.21.15.119:0/357842733 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98700ab680 con 0x7f987c099560 2024-03-28T16:48:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f9879d9b640 1 -- 172.21.15.119:0/357842733 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98700ab940 con 0x7f987c099560 2024-03-28T16:48:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f98811fc640 1 -- 172.21.15.119:0/357842733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c099560 msgr2=0x7f987c130fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f98811fc640 1 --2- 172.21.15.119:0/357842733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c099560 0x7f987c130fd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98700097f0 tx=0x7f987001c430 comp rx=0 tx=0).stop 2024-03-28T16:48:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f98811fc640 1 -- 172.21.15.119:0/357842733 shutdown_connections 2024-03-28T16:48:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f98811fc640 1 --2- 172.21.15.119:0/357842733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c099560 0x7f987c130fd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f98811fc640 1 -- 172.21.15.119:0/357842733 >> 172.21.15.119:0/357842733 conn(0x7f987c09c330 msgr2=0x7f987c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f98811fc640 1 -- 172.21.15.119:0/357842733 shutdown_connections 2024-03-28T16:48:51.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.302+0000 7f98811fc640 1 -- 172.21.15.119:0/357842733 wait complete. 2024-03-28T16:48:51.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98811fc640 1 Processor -- start 2024-03-28T16:48:51.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98811fc640 1 -- start start 2024-03-28T16:48:51.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98811fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c098a70 0x7f987c11a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98811fc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c11db40 con 0x7f987c098a70 2024-03-28T16:48:51.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f987ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c098a70 0x7f987c11a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.310 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f987ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c098a70 0x7f987c11a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42736/0 (socket says 172.21.15.119:42736) 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f987ad9d640 1 -- 172.21.15.119:0/2163725131 learned_addr learned my addr 172.21.15.119:0/2163725131 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f987ad9d640 1 -- 172.21.15.119:0/2163725131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700094d0 con 0x7f987c098a70 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f987ad9d640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c098a70 0x7f987c11a550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c3a0 tx=0x7f986c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98637fe640 1 -- 172.21.15.119:0/2163725131 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f986c005150 con 0x7f987c098a70 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98637fe640 1 -- 172.21.15.119:0/2163725131 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c0052b0 con 0x7f987c098a70 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987c11b750 con 0x7f987c098a70 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98637fe640 1 -- 172.21.15.119:0/2163725131 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f986c0067e0 con 0x7f987c098a70 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c11d700 con 0x7f987c098a70 2024-03-28T16:48:51.311 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98637fe640 1 -- 172.21.15.119:0/2163725131 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f986c006940 con 0x7f987c098a70 2024-03-28T16:48:51.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91d68b0640 1 Processor -- start 2024-03-28T16:48:51.312 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91d68b0640 1 -- start start 2024-03-28T16:48:51.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c11dc90 con 0x7f987c098a70 2024-03-28T16:48:51.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98637fe640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9868035a60 0x7f9868037f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.313 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f98637fe640 1 -- 172.21.15.119:0/2163725131 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f986c0470d0 con 0x7f987c098a70 2024-03-28T16:48:51.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91d68b0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91d0071a60 0x7f91d006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.314 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91d68b0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d006e8f0 con 0x7f91d0066b60 2024-03-28T16:48:51.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91cffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91d0071a60 0x7f91d006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91cffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91d0071a60 0x7f91d006cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42740/0 (socket says 172.21.15.119:42740) 2024-03-28T16:48:51.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91cffff640 1 -- 172.21.15.119:0/1470332256 learned_addr learned my addr 172.21.15.119:0/1470332256 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.315 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f987a59c640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9868035a60 0x7f9868037f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91d68b0640 1 -- 172.21.15.119:0/1470332256 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d006dc70 con 0x7f91d0071a60 2024-03-28T16:48:51.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91cffff640 1 -- 172.21.15.119:0/1470332256 >> v1:172.21.15.119:6789/0 conn(0x7f91d0066b60 legacy=0x7f91d006d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.316 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.306+0000 7f91cffff640 1 -- 172.21.15.119:0/1470332256 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d006ec00 con 0x7f91d0071a60 2024-03-28T16:48:51.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.310+0000 7f987a59c640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9868035a60 0x7f9868037f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f98700097f0 tx=0x7f9870008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.317 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.310+0000 7f91cffff640 1 --2- 172.21.15.119:0/1470332256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0071a60 0x7f91d006cbb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f91c00097f0 tx=0x7f91c001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e4271273f31123 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.318 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f98637fe640 1 -- 172.21.15.119:0/2163725131 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f986c006bf0 con 0x7f987c098a70 2024-03-28T16:48:51.320 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91ceffd640 1 -- 172.21.15.119:0/1470332256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c00ab530 con 0x7f91d0071a60 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91ceffd640 1 -- 172.21.15.119:0/1470332256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c00a7050 con 0x7f91d0071a60 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91ceffd640 1 -- 172.21.15.119:0/1470332256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c00ab7e0 con 0x7f91d0071a60 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- 172.21.15.119:0/1470332256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0071a60 msgr2=0x7f91d006cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 --2- 172.21.15.119:0/1470332256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0071a60 0x7f91d006cbb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f91c00097f0 tx=0x7f91c001c430 comp rx=0 tx=0).stop 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- 172.21.15.119:0/1470332256 shutdown_connections 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 --2- 172.21.15.119:0/1470332256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0071a60 0x7f91d006cbb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- 172.21.15.119:0/1470332256 >> 172.21.15.119:0/1470332256 conn(0x7f91d009c8e0 msgr2=0x7f91d009ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- 172.21.15.119:0/1470332256 shutdown_connections 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- 172.21.15.119:0/1470332256 wait complete. 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 Processor -- start 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- start start 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0066b60 0x7f91d0122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0125ef0 con 0x7f91d0066b60 2024-03-28T16:48:51.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0066b60 0x7f91d0122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0066b60 0x7f91d0122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42742/0 (socket says 172.21.15.119:42742) 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91cffff640 1 -- 172.21.15.119:0/3821339775 learned_addr learned my addr 172.21.15.119:0/3821339775 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91cffff640 1 -- 172.21.15.119:0/3821339775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00094d0 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91cffff640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0066b60 0x7f91d0122900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c001a060 tx=0x7f91c00191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91ccff9640 1 -- 172.21.15.119:0/3821339775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c00b4440 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91ccff9640 1 -- 172.21.15.119:0/3821339775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c00b45a0 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91ccff9640 1 -- 172.21.15.119:0/3821339775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91c00b4880 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d0070810 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.314+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d0124e80 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.318+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d0126030 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.318+0000 7f91ccff9640 1 -- 172.21.15.119:0/3821339775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f91c00b2070 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.318+0000 7f91ccff9640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f91bc035a10 0x7f91bc037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.318+0000 7f91ccff9640 1 -- 172.21.15.119:0/3821339775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f91c00e1d90 con 0x7f91d0066b60 2024-03-28T16:48:51.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.318+0000 7f91cf7fe640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f91bc035a10 0x7f91bc037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.323 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.318+0000 7f91cf7fe640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f91bc035a10 0x7f91bc037ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91c4000f70 tx=0x7f91c4007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.328 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.326+0000 7f91ccff9640 1 -- 172.21.15.119:0/3821339775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91c00c46c0 con 0x7f91d0066b60 2024-03-28T16:48:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1348062860 con 0x7f134806c490 2024-03-28T16:48:51.416 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f1336ffd640 1 -- 172.21.15.119:0/1789307286 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f133804c020 con 0x7f134806c490 2024-03-28T16:48:51.416 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1328035a10 msgr2=0x7f1328037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1328035a10 0x7f1328037ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f133c0ae000 tx=0x7f133c008040 comp rx=0 tx=0).stop 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134806c490 msgr2=0x7f1348127a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134806c490 0x7f1348127a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f133800c480 tx=0x7f133800c940 comp rx=0 tx=0).stop 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 shutdown_connections 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1328035a10 0x7f1328037ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 --2- 172.21.15.119:0/1789307286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f134806c490 0x7f1348127a50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 >> 172.21.15.119:0/1789307286 conn(0x7f134809b1d0 msgr2=0x7f1348095ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.418 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 shutdown_connections 2024-03-28T16:48:51.419 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.414+0000 7f134caef640 1 -- 172.21.15.119:0/1789307286 wait complete. 2024-03-28T16:48:51.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.3 2024-03-28T16:48:51.433 DEBUG:teuthology.parallel:result is None 2024-03-28T16:48:51.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f09140008e0 con 0x7f091406c0d0 2024-03-28T16:48:51.451 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f0910ff9640 1 -- 172.21.15.119:0/3979573160 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09040b0070 con 0x7f091406c0d0 2024-03-28T16:48:51.451 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0900035a10 msgr2=0x7f0900037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0900035a10 0x7f0900037ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0908001610 tx=0x7f0908007ad0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 msgr2=0x7f09141308c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f09141308c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f09040ae040 tx=0x7f09040191a0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 shutdown_connections 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0900035a10 0x7f0900037ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 --2- 172.21.15.119:0/3979573160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f091406c0d0 0x7f09141308c0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f09040ae040 tx=0x7f09040191a0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 >> 172.21.15.119:0/3979573160 conn(0x7f091409d600 msgr2=0x7f09141323d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 shutdown_connections 2024-03-28T16:48:51.453 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.450+0000 7f08fa7fc640 1 -- 172.21.15.119:0/3979573160 wait complete. 2024-03-28T16:48:51.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.1 2024-03-28T16:48:51.467 DEBUG:teuthology.parallel:result is None 2024-03-28T16:48:51.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f91d0062860 con 0x7f91d0066b60 2024-03-28T16:48:51.483 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91ccff9640 1 -- 172.21.15.119:0/3821339775 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91c00b8070 con 0x7f91d0066b60 2024-03-28T16:48:51.483 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f91bc035a10 msgr2=0x7f91bc037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f91bc035a10 0x7f91bc037ed0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91c4000f70 tx=0x7f91c4007a40 comp rx=0 tx=0).stop 2024-03-28T16:48:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0066b60 msgr2=0x7f91d0122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0066b60 0x7f91d0122900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91c001a060 tx=0x7f91c00191d0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 shutdown_connections 2024-03-28T16:48:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f91bc035a10 0x7f91bc037ed0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 --2- 172.21.15.119:0/3821339775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d0066b60 0x7f91d0122900 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 >> 172.21.15.119:0/3821339775 conn(0x7f91d009c8e0 msgr2=0x7f91d0090420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 shutdown_connections 2024-03-28T16:48:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.482+0000 7f91d68b0640 1 -- 172.21.15.119:0/3821339775 wait complete. 2024-03-28T16:48:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f987c062860 con 0x7f987c098a70 2024-03-28T16:48:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98637fe640 1 -- 172.21.15.119:0/2163725131 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f986c0102e0 con 0x7f987c098a70 2024-03-28T16:48:51.487 INFO:teuthology.orchestra.run.smithi119.stdout:38654705668 2024-03-28T16:48:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9868035a60 msgr2=0x7f9868037f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9868035a60 0x7f9868037f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f98700097f0 tx=0x7f9870008040 comp rx=0 tx=0).stop 2024-03-28T16:48:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c098a70 msgr2=0x7f987c11a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c098a70 0x7f987c11a550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c3a0 tx=0x7f986c00c860 comp rx=0 tx=0).stop 2024-03-28T16:48:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 shutdown_connections 2024-03-28T16:48:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9868035a60 0x7f9868037f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 --2- 172.21.15.119:0/2163725131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f987c098a70 0x7f987c11a550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 >> 172.21.15.119:0/2163725131 conn(0x7f987c09c330 msgr2=0x7f987c070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 shutdown_connections 2024-03-28T16:48:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.486+0000 7f98811fc640 1 -- 172.21.15.119:0/2163725131 wait complete. 2024-03-28T16:48:51.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.2 2024-03-28T16:48:51.502 DEBUG:teuthology.parallel:result is None 2024-03-28T16:48:51.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2024-03-28T16:48:51.505 DEBUG:teuthology.parallel:result is None 2024-03-28T16:48:51.505 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-28T16:48:51.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:48:51.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff6848ed640 1 Processor -- start 2024-03-28T16:48:51.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff6848ed640 1 -- start start 2024-03-28T16:48:51.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff6848ed640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff68009d7e0 0x7ff6801332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.653 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff6848ed640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff680099d80 con 0x7ff680098910 2024-03-28T16:48:51.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff6848ed640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68009a340 con 0x7ff68009d7e0 2024-03-28T16:48:51.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff67dd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff68009d7e0 0x7ff6801332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff67e59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff680098910 0x7ff680098cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41692/0 (socket says 172.21.15.119:41692) 2024-03-28T16:48:51.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff67e59c640 1 -- 172.21.15.119:0/1770894813 learned_addr learned my addr 172.21.15.119:0/1770894813 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:51.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff67dd9b640 1 -- 172.21.15.119:0/1770894813 >> v1:172.21.15.119:6789/0 conn(0x7ff680098910 legacy=0x7ff680098cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.654 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff67dd9b640 1 -- 172.21.15.119:0/1770894813 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68009a5e0 con 0x7ff68009d7e0 2024-03-28T16:48:51.655 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.650+0000 7ff67d59a640 1 -- 172.21.15.119:0/1770894813 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365125166 0 0) 0x7ff680099d80 con 0x7ff680098910 2024-03-28T16:48:51.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff67dd9b640 1 --2- 172.21.15.119:0/1770894813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff68009d7e0 0x7ff6801332f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff668009740 tx=0x7ff66801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a23b576543560c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff67d59a640 1 -- 172.21.15.119:0/1770894813 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6680ab530 con 0x7ff68009d7e0 2024-03-28T16:48:51.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff67d59a640 1 -- 172.21.15.119:0/1770894813 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6680a7050 con 0x7ff68009d7e0 2024-03-28T16:48:51.656 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff67d59a640 1 -- 172.21.15.119:0/1770894813 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6680ab7e0 con 0x7ff68009d7e0 2024-03-28T16:48:51.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 -- 172.21.15.119:0/1770894813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff68009d7e0 msgr2=0x7ff6801332f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 --2- 172.21.15.119:0/1770894813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff68009d7e0 0x7ff6801332f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff668009740 tx=0x7ff66801c370 comp rx=0 tx=0).stop 2024-03-28T16:48:51.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 -- 172.21.15.119:0/1770894813 shutdown_connections 2024-03-28T16:48:51.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 --2- 172.21.15.119:0/1770894813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff68009d7e0 0x7ff6801332f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 -- 172.21.15.119:0/1770894813 >> 172.21.15.119:0/1770894813 conn(0x7ff6800a0a00 msgr2=0x7ff68009cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 -- 172.21.15.119:0/1770894813 shutdown_connections 2024-03-28T16:48:51.657 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 -- 172.21.15.119:0/1770894813 wait complete. 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 Processor -- start 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 -- start start 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff680098910 0x7ff6801c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.654+0000 7ff6848ed640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff680132b70 con 0x7ff680098910 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff67e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff680098910 0x7ff6801c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff67e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff680098910 0x7ff6801c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42748/0 (socket says 172.21.15.119:42748) 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff67e59c640 1 -- 172.21.15.119:0/780316144 learned_addr learned my addr 172.21.15.119:0/780316144 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff67e59c640 1 -- 172.21.15.119:0/780316144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668009420 con 0x7ff680098910 2024-03-28T16:48:51.659 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff67e59c640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff680098910 0x7ff6801c5940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c840 tx=0x7ff67800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff676ffd640 1 -- 172.21.15.119:0/780316144 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff67800a660 con 0x7ff680098910 2024-03-28T16:48:51.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff676ffd640 1 -- 172.21.15.119:0/780316144 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800a7e0 con 0x7ff680098910 2024-03-28T16:48:51.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6801316c0 con 0x7ff680098910 2024-03-28T16:48:51.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff676ffd640 1 -- 172.21.15.119:0/780316144 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff67800aac0 con 0x7ff680098910 2024-03-28T16:48:51.660 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff680131b60 con 0x7ff680098910 2024-03-28T16:48:51.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6801329c0 con 0x7ff680098910 2024-03-28T16:48:51.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff676ffd640 1 -- 172.21.15.119:0/780316144 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff67800ac20 con 0x7ff680098910 2024-03-28T16:48:51.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff676ffd640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff660035a10 0x7ff660037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff676ffd640 1 -- 172.21.15.119:0/780316144 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7ff678047760 con 0x7ff680098910 2024-03-28T16:48:51.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff67dd9b640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff660035a10 0x7ff660037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.661 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.658+0000 7ff67dd9b640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff660035a10 0x7ff660037ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ad000 tx=0x7ff668008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.662+0000 7ff676ffd640 1 -- 172.21.15.119:0/780316144 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff67801a6b0 con 0x7ff680098910 2024-03-28T16:48:51.805 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.802+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6800975f0 con 0x7ff660035a10 2024-03-28T16:48:51.809 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.806+0000 7ff676ffd640 1 -- 172.21.15.119:0/780316144 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34663 (secure 0 0 0) 0x7ff6800975f0 con 0x7ff660035a10 2024-03-28T16:48:51.810 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:51.810 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff660035a10 msgr2=0x7ff660037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff660035a10 0x7ff660037ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ad000 tx=0x7ff668008040 comp rx=0 tx=0).stop 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff680098910 msgr2=0x7ff6801c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff680098910 0x7ff6801c5940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c840 tx=0x7ff67800cd00 comp rx=0 tx=0).stop 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 shutdown_connections 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff660035a10 0x7ff660037ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 --2- 172.21.15.119:0/780316144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff680098910 0x7ff6801c5940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 >> 172.21.15.119:0/780316144 conn(0x7ff6800a0a00 msgr2=0x7ff68012db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 shutdown_connections 2024-03-28T16:48:51.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.810+0000 7ff6848ed640 1 -- 172.21.15.119:0/780316144 wait complete. 2024-03-28T16:48:51.826 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-03-28T16:48:50.483775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":122412,"kb_used_data":1664,"kb_used_omap":6,"kb_used_meta":120505,"kb_avail":377364948,"statfs":{"total":386547056640,"available":386421706752,"internally_reserved":0,"allocated":1703936,"data_stored":1336536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":123397910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001131"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.262597+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:48:49.262597+0000","last_peered":"2024-03-28T16:48:49.262597+0000","last_clean":"2024-03-28T16:48:49.262597+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:48:49.262597+0000","last_undegraded":"2024-03-28T16:48:49.262597+0000","last_fullsized":"2024-03-28T16:48:49.262597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:25.262378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.614252+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:48:49.614252+0000","last_peered":"2024-03-28T16:48:49.614252+0000","last_clean":"2024-03-28T16:48:49.614252+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:48:49.614252+0000","last_undegraded":"2024-03-28T16:48:49.614252+0000","last_fullsized":"2024-03-28T16:48:49.614252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626860+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:48:48.626860+0000","last_peered":"2024-03-28T16:48:48.626860+0000","last_clean":"2024-03-28T16:48:48.626860+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:48:48.626860+0000","last_undegraded":"2024-03-28T16:48:48.626860+0000","last_fullsized":"2024-03-28T16:48:48.626860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.614272+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:48:49.614272+0000","last_peered":"2024-03-28T16:48:49.614272+0000","last_clean":"2024-03-28T16:48:49.614272+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:48:49.614272+0000","last_undegraded":"2024-03-28T16:48:49.614272+0000","last_fullsized":"2024-03-28T16:48:49.614272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626335+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:48:48.626335+0000","last_peered":"2024-03-28T16:48:48.626335+0000","last_clean":"2024-03-28T16:48:48.626335+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:48:48.626335+0000","last_undegraded":"2024-03-28T16:48:48.626335+0000","last_fullsized":"2024-03-28T16:48:48.626335+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:40.767997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.622657+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:48:48.622657+0000","last_peered":"2024-03-28T16:48:48.622657+0000","last_clean":"2024-03-28T16:48:48.622657+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:48:48.622657+0000","last_undegraded":"2024-03-28T16:48:48.622657+0000","last_fullsized":"2024-03-28T16:48:48.622657+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:01:15.487399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626389+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:48:48.626389+0000","last_peered":"2024-03-28T16:48:48.626389+0000","last_clean":"2024-03-28T16:48:48.626389+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:48:48.626389+0000","last_undegraded":"2024-03-28T16:48:48.626389+0000","last_fullsized":"2024-03-28T16:48:48.626389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:01:15.487399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626893+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:48:48.626893+0000","last_peered":"2024-03-28T16:48:48.626893+0000","last_clean":"2024-03-28T16:48:48.626893+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:48:48.626893+0000","last_undegraded":"2024-03-28T16:48:48.626893+0000","last_fullsized":"2024-03-28T16:48:48.626893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":168,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.653092+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:48:49.653092+0000","last_peered":"2024-03-28T16:48:49.653092+0000","last_clean":"2024-03-28T16:48:49.653092+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:48:49.653092+0000","last_undegraded":"2024-03-28T16:48:49.653092+0000","last_fullsized":"2024-03-28T16:48:49.653092+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:35.744620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31308,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":131072,"data_stored":39961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":630329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":716800,"data_stored":626285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":39961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:48:51.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:48:51.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 Processor -- start 2024-03-28T16:48:51.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- start start 2024-03-28T16:48:51.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f67f40938f0 0x7f67f4093cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f40958f0 con 0x7f67f4092560 2024-03-28T16:48:51.931 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f409c160 con 0x7f67f40938f0 2024-03-28T16:48:51.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f67f40938f0 0x7f67f4093cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67ebfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f67f4092560 0x7f67f4092a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41702/0 (socket says 172.21.15.119:41702) 2024-03-28T16:48:51.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67ebfff640 1 -- 172.21.15.119:0/3129154619 learned_addr learned my addr 172.21.15.119:0/3129154619 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:48:51.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f259c640 1 -- 172.21.15.119:0/3129154619 >> v1:172.21.15.119:6789/0 conn(0x7f67f4092560 legacy=0x7f67f4092a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.932 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f259c640 1 -- 172.21.15.119:0/3129154619 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f406c250 con 0x7f67f40938f0 2024-03-28T16:48:51.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f259c640 1 --2- 172.21.15.119:0/3129154619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f40938f0 0x7f67f4093cd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009740 tx=0x7f67e401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0676aabe983c37 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f1d9b640 1 -- 172.21.15.119:0/3129154619 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e40a8050 con 0x7f67f40938f0 2024-03-28T16:48:51.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f1d9b640 1 -- 172.21.15.119:0/3129154619 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e40ac520 con 0x7f67f40938f0 2024-03-28T16:48:51.933 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f1d9b640 1 -- 172.21.15.119:0/3129154619 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e40ac7e0 con 0x7f67f40938f0 2024-03-28T16:48:51.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- 172.21.15.119:0/3129154619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f40938f0 msgr2=0x7f67f4093cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:51.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 --2- 172.21.15.119:0/3129154619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f40938f0 0x7f67f4093cd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009740 tx=0x7f67e401c370 comp rx=0 tx=0).stop 2024-03-28T16:48:51.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- 172.21.15.119:0/3129154619 shutdown_connections 2024-03-28T16:48:51.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 --2- 172.21.15.119:0/3129154619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f40938f0 0x7f67f4093cd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:51.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- 172.21.15.119:0/3129154619 >> 172.21.15.119:0/3129154619 conn(0x7f67f409a2f0 msgr2=0x7f67f409a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:51.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- 172.21.15.119:0/3129154619 shutdown_connections 2024-03-28T16:48:51.934 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.930+0000 7f67f8d56640 1 -- 172.21.15.119:0/3129154619 wait complete. 2024-03-28T16:48:51.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f8d56640 1 Processor -- start 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f8d56640 1 -- start start 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f8d56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f4092560 0x7f67f41c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f8d56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f408b830 con 0x7f67f4092560 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f4092560 0x7f67f41c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f4092560 0x7f67f41c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42760/0 (socket says 172.21.15.119:42760) 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f259c640 1 -- 172.21.15.119:0/3765594558 learned_addr learned my addr 172.21.15.119:0/3765594558 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f259c640 1 -- 172.21.15.119:0/3765594558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4009420 con 0x7f67f4092560 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f259c640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f4092560 0x7f67f41c2510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f67e401aa20 tx=0x7f67e401abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67eaffd640 1 -- 172.21.15.119:0/3765594558 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e40b8790 con 0x7f67f4092560 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67eaffd640 1 -- 172.21.15.119:0/3765594558 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e40b88f0 con 0x7f67f4092560 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f408a570 con 0x7f67f4092560 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67eaffd640 1 -- 172.21.15.119:0/3765594558 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e40b8bb0 con 0x7f67f4092560 2024-03-28T16:48:51.937 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f408b170 con 0x7f67f4092560 2024-03-28T16:48:51.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67eaffd640 1 -- 172.21.15.119:0/3765594558 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f67e40b8d10 con 0x7f67f4092560 2024-03-28T16:48:51.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f4091c10 con 0x7f67f4092560 2024-03-28T16:48:51.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67eaffd640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f67d40359c0 0x7f67d4037e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:51.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67eaffd640 1 -- 172.21.15.119:0/3765594558 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f67e40e27f0 con 0x7f67f4092560 2024-03-28T16:48:51.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67ebfff640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f67d40359c0 0x7f67d4037e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:51.938 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.934+0000 7f67ebfff640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f67d40359c0 0x7f67d4037e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000f70 tx=0x7f67dc007a90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:51.941 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:51.938+0000 7f67eaffd640 1 -- 172.21.15.119:0/3765594558 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67e40b4300 con 0x7f67f4092560 2024-03-28T16:48:52.085 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.082+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67f4067e00 con 0x7f67d40359c0 2024-03-28T16:48:52.088 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.086+0000 7f67eaffd640 1 -- 172.21.15.119:0/3765594558 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34663 (secure 0 0 0) 0x7f67f4067e00 con 0x7f67d40359c0 2024-03-28T16:48:52.089 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:52.089 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f67d40359c0 msgr2=0x7f67d4037e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f67d40359c0 0x7f67d4037e80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000f70 tx=0x7f67dc007a90 comp rx=0 tx=0).stop 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f4092560 msgr2=0x7f67f41c2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f4092560 0x7f67f41c2510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f67e401aa20 tx=0x7f67e401abb0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 shutdown_connections 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f67d40359c0 0x7f67d4037e80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 --2- 172.21.15.119:0/3765594558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67f4092560 0x7f67f41c2510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 >> 172.21.15.119:0/3765594558 conn(0x7f67f409a2f0 msgr2=0x7f67f4090580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 shutdown_connections 2024-03-28T16:48:52.091 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.090+0000 7f67f8d56640 1 -- 172.21.15.119:0/3765594558 wait complete. 2024-03-28T16:48:52.105 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-03-28T16:48:50.483775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":122412,"kb_used_data":1664,"kb_used_omap":6,"kb_used_meta":120505,"kb_avail":377364948,"statfs":{"total":386547056640,"available":386421706752,"internally_reserved":0,"allocated":1703936,"data_stored":1336536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":123397910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001131"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.262597+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:48:49.262597+0000","last_peered":"2024-03-28T16:48:49.262597+0000","last_clean":"2024-03-28T16:48:49.262597+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:48:49.262597+0000","last_undegraded":"2024-03-28T16:48:49.262597+0000","last_fullsized":"2024-03-28T16:48:49.262597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:25.262378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.614252+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:48:49.614252+0000","last_peered":"2024-03-28T16:48:49.614252+0000","last_clean":"2024-03-28T16:48:49.614252+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:48:49.614252+0000","last_undegraded":"2024-03-28T16:48:49.614252+0000","last_fullsized":"2024-03-28T16:48:49.614252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626860+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:48:48.626860+0000","last_peered":"2024-03-28T16:48:48.626860+0000","last_clean":"2024-03-28T16:48:48.626860+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:48:48.626860+0000","last_undegraded":"2024-03-28T16:48:48.626860+0000","last_fullsized":"2024-03-28T16:48:48.626860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.614272+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:48:49.614272+0000","last_peered":"2024-03-28T16:48:49.614272+0000","last_clean":"2024-03-28T16:48:49.614272+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:48:49.614272+0000","last_undegraded":"2024-03-28T16:48:49.614272+0000","last_fullsized":"2024-03-28T16:48:49.614272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626335+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:48:48.626335+0000","last_peered":"2024-03-28T16:48:48.626335+0000","last_clean":"2024-03-28T16:48:48.626335+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:48:48.626335+0000","last_undegraded":"2024-03-28T16:48:48.626335+0000","last_fullsized":"2024-03-28T16:48:48.626335+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:40.767997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.622657+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:48:48.622657+0000","last_peered":"2024-03-28T16:48:48.622657+0000","last_clean":"2024-03-28T16:48:48.622657+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:48:48.622657+0000","last_undegraded":"2024-03-28T16:48:48.622657+0000","last_fullsized":"2024-03-28T16:48:48.622657+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:01:15.487399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626389+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:48:48.626389+0000","last_peered":"2024-03-28T16:48:48.626389+0000","last_clean":"2024-03-28T16:48:48.626389+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:48:48.626389+0000","last_undegraded":"2024-03-28T16:48:48.626389+0000","last_fullsized":"2024-03-28T16:48:48.626389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:01:15.487399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-03-28T16:48:48.626893+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:48:48.626893+0000","last_peered":"2024-03-28T16:48:48.626893+0000","last_clean":"2024-03-28T16:48:48.626893+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:48:48.626893+0000","last_undegraded":"2024-03-28T16:48:48.626893+0000","last_fullsized":"2024-03-28T16:48:48.626893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":168,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-28T16:48:49.653092+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:48:49.653092+0000","last_peered":"2024-03-28T16:48:49.653092+0000","last_clean":"2024-03-28T16:48:49.653092+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:48:49.653092+0000","last_undegraded":"2024-03-28T16:48:49.653092+0000","last_fullsized":"2024-03-28T16:48:49.653092+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:35.744620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31308,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340532,"statfs":{"total":96636764160,"available":96604704768,"internally_reserved":0,"allocated":131072,"data_stored":39961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":724992,"data_stored":630329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":716800,"data_stored":626285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":131072,"data_stored":39961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:48:52.106 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-28T16:48:52.106 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-28T16:48:52.106 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-28T16:48:52.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb70e863640 1 Processor -- start 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb70e863640 1 -- start start 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb70e863640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb708093a10 0x7fb708093df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb70e863640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb708095170 con 0x7fb708094330 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb70e863640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb708072550 con 0x7fb708093a10 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb707fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb708093a10 0x7fb708093df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb707fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb708093a10 0x7fb708093df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44022/0 (socket says 172.21.15.119:44022) 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb707fff640 1 -- 172.21.15.119:0/1680350091 learned_addr learned my addr 172.21.15.119:0/1680350091 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb707fff640 1 -- 172.21.15.119:0/1680350091 >> v1:172.21.15.119:6789/0 conn(0x7fb708094330 legacy=0x7fb70812e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.209 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.206+0000 7fb707fff640 1 -- 172.21.15.119:0/1680350091 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708094bb0 con 0x7fb708093a10 2024-03-28T16:48:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb707fff640 1 --2- 172.21.15.119:0/1680350091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb708093df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7000097f0 tx=0x7fb70001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af968f9bcd051865 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb706ffd640 1 -- 172.21.15.119:0/1680350091 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7000ab530 con 0x7fb708093a10 2024-03-28T16:48:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb706ffd640 1 -- 172.21.15.119:0/1680350091 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7000a7050 con 0x7fb708093a10 2024-03-28T16:48:52.211 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb706ffd640 1 -- 172.21.15.119:0/1680350091 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7000ab7e0 con 0x7fb708093a10 2024-03-28T16:48:52.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- 172.21.15.119:0/1680350091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 msgr2=0x7fb708093df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 --2- 172.21.15.119:0/1680350091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb708093df0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7000097f0 tx=0x7fb70001c430 comp rx=0 tx=0).stop 2024-03-28T16:48:52.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- 172.21.15.119:0/1680350091 shutdown_connections 2024-03-28T16:48:52.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 --2- 172.21.15.119:0/1680350091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb708093df0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- 172.21.15.119:0/1680350091 >> 172.21.15.119:0/1680350091 conn(0x7fb70809a2f0 msgr2=0x7fb70809a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:52.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- 172.21.15.119:0/1680350091 shutdown_connections 2024-03-28T16:48:52.212 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- 172.21.15.119:0/1680350091 wait complete. 2024-03-28T16:48:52.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 Processor -- start 2024-03-28T16:48:52.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- start start 2024-03-28T16:48:52.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb70812d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:52.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb708070c00 con 0x7fb708093a10 2024-03-28T16:48:52.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb707fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb70812d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:52.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb707fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb70812d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44038/0 (socket says 172.21.15.119:44038) 2024-03-28T16:48:52.213 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb707fff640 1 -- 172.21.15.119:0/877495816 learned_addr learned my addr 172.21.15.119:0/877495816 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb707fff640 1 -- 172.21.15.119:0/877495816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7000094d0 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb707fff640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb70812d3d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb70001a060 tx=0x7fb70001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb704ff9640 1 -- 172.21.15.119:0/877495816 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7000b6420 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb704ff9640 1 -- 172.21.15.119:0/877495816 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7000b6580 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb708071930 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.210+0000 7fb704ff9640 1 -- 172.21.15.119:0/877495816 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7000b6820 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.214+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb708070540 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.214+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb708067e00 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.214+0000 7fb704ff9640 1 -- 172.21.15.119:0/877495816 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb7000b2040 con 0x7fb708093a10 2024-03-28T16:48:52.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.214+0000 7fb704ff9640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb6e8035a10 0x7fb6e8037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:52.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.214+0000 7fb704ff9640 1 -- 172.21.15.119:0/877495816 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fb7000e21e0 con 0x7fb708093a10 2024-03-28T16:48:52.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.214+0000 7fb7077fe640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb6e8035a10 0x7fb6e8037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:52.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.214+0000 7fb7077fe640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb6e8035a10 0x7fb6e8037ed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000f70 tx=0x7fb6f0007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:52.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.218+0000 7fb704ff9640 1 -- 172.21.15.119:0/877495816 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7000b6980 con 0x7fb708093a10 2024-03-28T16:48:52.394 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb708062860 con 0x7fb708093a10 2024-03-28T16:48:52.395 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb704ff9640 1 -- 172.21.15.119:0/877495816 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb7000b0020 con 0x7fb708093a10 2024-03-28T16:48:52.395 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:48:52.395 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T16:48:52.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb6e8035a10 msgr2=0x7fb6e8037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb6e8035a10 0x7fb6e8037ed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000f70 tx=0x7fb6f0007a40 comp rx=0 tx=0).stop 2024-03-28T16:48:52.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 msgr2=0x7fb70812d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.397 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb70812d3d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb70001a060 tx=0x7fb70001ac70 comp rx=0 tx=0).stop 2024-03-28T16:48:52.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 shutdown_connections 2024-03-28T16:48:52.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb6e8035a10 0x7fb6e8037ed0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 --2- 172.21.15.119:0/877495816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb708093a10 0x7fb70812d3d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 >> 172.21.15.119:0/877495816 conn(0x7fb70809a2f0 msgr2=0x7fb70812f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:52.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 shutdown_connections 2024-03-28T16:48:52.398 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:48:52.394+0000 7fb70e863640 1 -- 172.21.15.119:0/877495816 wait complete. 2024-03-28T16:48:52.412 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-28T16:48:52.413 INFO:teuthology.run_tasks:Running task rgw... 2024-03-28T16:48:52.427 DEBUG:tasks.rgw:config is {'client.0': None} 2024-03-28T16:48:52.427 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-03-28T16:48:52.427 INFO:tasks.rgw:Creating data pools 2024-03-28T16:48:52.427 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-03-28T16:48:52.427 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2024-03-28T16:48:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4336e28640 1 Processor -- start 2024-03-28T16:48:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4336e28640 1 -- start start 2024-03-28T16:48:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4336e28640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f433006b590 0x7f433006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4336e28640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433006c3f0 con 0x7f433006beb0 2024-03-28T16:48:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4336e28640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4330114310 con 0x7f433006b590 2024-03-28T16:48:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4334bc4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f433006b590 0x7f433006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f432ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f433006beb0 0x7f4330111cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:50302/0 (socket says 172.21.15.170:50302) 2024-03-28T16:48:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4334bc4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f433006b590 0x7f433006b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49124/0 (socket says 172.21.15.170:49124) 2024-03-28T16:48:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f432ffff640 1 -- 172.21.15.170:0/2203494877 learned_addr learned my addr 172.21.15.170:0/2203494877 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:48:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4334bc4640 1 -- 172.21.15.170:0/2203494877 >> v1:172.21.15.119:6789/0 conn(0x7f433006beb0 legacy=0x7f4330111cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.563+0000 7f4334bc4640 1 -- 172.21.15.170:0/2203494877 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43301123e0 con 0x7f433006b590 2024-03-28T16:48:52.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4334bc4640 1 --2- 172.21.15.170:0/2203494877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f433006b970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4328009d90 tx=0x7f432801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fe2b40539c4786 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:52.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f432f7fe640 1 -- 172.21.15.170:0/2203494877 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43280a5740 con 0x7f433006b590 2024-03-28T16:48:52.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f432f7fe640 1 -- 172.21.15.170:0/2203494877 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43280a58a0 con 0x7f433006b590 2024-03-28T16:48:52.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f432f7fe640 1 -- 172.21.15.170:0/2203494877 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43280a5b60 con 0x7f433006b590 2024-03-28T16:48:52.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 -- 172.21.15.170:0/2203494877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 msgr2=0x7f433006b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:52.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 --2- 172.21.15.170:0/2203494877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f433006b970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4328009d90 tx=0x7f432801c8c0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 -- 172.21.15.170:0/2203494877 shutdown_connections 2024-03-28T16:48:52.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 --2- 172.21.15.170:0/2203494877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f433006b970 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:52.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 -- 172.21.15.170:0/2203494877 >> 172.21.15.170:0/2203494877 conn(0x7f433006f4e0 msgr2=0x7f4330071900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:52.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 -- 172.21.15.170:0/2203494877 shutdown_connections 2024-03-28T16:48:52.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 -- 172.21.15.170:0/2203494877 wait complete. 2024-03-28T16:48:52.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 Processor -- start 2024-03-28T16:48:52.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 -- start start 2024-03-28T16:48:52.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f43301b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:52.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4336e28640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301b37d0 con 0x7f433006b590 2024-03-28T16:48:52.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4334bc4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f43301b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:52.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4334bc4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f43301b3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49136/0 (socket says 172.21.15.170:49136) 2024-03-28T16:48:52.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.567+0000 7f4334bc4640 1 -- 172.21.15.170:0/1163478636 learned_addr learned my addr 172.21.15.170:0/1163478636 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:52.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f4334bc4640 1 -- 172.21.15.170:0/1163478636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4328009a70 con 0x7f433006b590 2024-03-28T16:48:52.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f4334bc4640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f43301b3200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f432801a4f0 tx=0x7f4328019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:52.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432d7fa640 1 -- 172.21.15.170:0/1163478636 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43280adb80 con 0x7f433006b590 2024-03-28T16:48:52.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43301b3970 con 0x7f433006b590 2024-03-28T16:48:52.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432d7fa640 1 -- 172.21.15.170:0/1163478636 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43280adce0 con 0x7f433006b590 2024-03-28T16:48:52.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43301ad090 con 0x7f433006b590 2024-03-28T16:48:52.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432d7fa640 1 -- 172.21.15.170:0/1163478636 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43280c2450 con 0x7f433006b590 2024-03-28T16:48:52.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4330062860 con 0x7f433006b590 2024-03-28T16:48:52.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432d7fa640 1 -- 172.21.15.170:0/1163478636 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f43280b4050 con 0x7f433006b590 2024-03-28T16:48:52.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432d7fa640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f43100359c0 0x7f4310037e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:52.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432d7fa640 1 -- 172.21.15.170:0/1163478636 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f43280e3890 con 0x7f433006b590 2024-03-28T16:48:52.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432ffff640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f43100359c0 0x7f4310037e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:52.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.571+0000 7f432ffff640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f43100359c0 0x7f4310037e80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4318005890 tx=0x7f4318007a90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:52.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.583+0000 7f432d7fa640 1 -- 172.21.15.170:0/1163478636 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43280c13b0 con 0x7f433006b590 2024-03-28T16:48:52.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:52.747+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7f433006beb0 con 0x7f433006b590 2024-03-28T16:48:53.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f432d7fa640 1 -- 172.21.15.170:0/1163478636 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.data' created v17) v1 ==== 167+0+0 (secure 0 0 0) 0x7f43280c7070 con 0x7f433006b590 2024-03-28T16:48:53.626 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'default.rgw.buckets.data' created 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f43100359c0 msgr2=0x7f4310037e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f43100359c0 0x7f4310037e80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4318005890 tx=0x7f4318007a90 comp rx=0 tx=0).stop 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 msgr2=0x7f43301b3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f43301b3200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f432801a4f0 tx=0x7f4328019660 comp rx=0 tx=0).stop 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 shutdown_connections 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f43100359c0 0x7f4310037e80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 --2- 172.21.15.170:0/1163478636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f433006b590 0x7f43301b3200 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:53.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 >> 172.21.15.170:0/1163478636 conn(0x7f433006f4e0 msgr2=0x7f4330120f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:53.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 shutdown_connections 2024-03-28T16:48:53.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.627+0000 7f4336e28640 1 -- 172.21.15.170:0/1163478636 wait complete. 2024-03-28T16:48:53.661 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-03-28T16:48:53.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3648aac640 1 Processor -- start 2024-03-28T16:48:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3648aac640 1 -- start start 2024-03-28T16:48:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3648aac640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36441181b0 0x7f364411a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3648aac640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364411aae0 con 0x7f364410e020 2024-03-28T16:48:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3648aac640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364411bc80 con 0x7f36441181b0 2024-03-28T16:48:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3641d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36441181b0 0x7f364411a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:53.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3641d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36441181b0 0x7f364411a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49152/0 (socket says 172.21.15.170:49152) 2024-03-28T16:48:53.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3641d9b640 1 -- 172.21.15.170:0/2794550886 learned_addr learned my addr 172.21.15.170:0/2794550886 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:53.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3641d9b640 1 -- 172.21.15.170:0/2794550886 >> v1:172.21.15.119:6789/0 conn(0x7f364410e020 legacy=0x7f364410e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:53.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.787+0000 7f3641d9b640 1 -- 172.21.15.170:0/2794550886 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364411bdc0 con 0x7f36441181b0 2024-03-28T16:48:53.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3641d9b640 1 --2- 172.21.15.170:0/2794550886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36441181b0 0x7f364411a5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36380097f0 tx=0x7f363801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470e0c563a60e257 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:53.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364159a640 1 -- 172.21.15.170:0/2794550886 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36380ab580 con 0x7f36441181b0 2024-03-28T16:48:53.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364159a640 1 -- 172.21.15.170:0/2794550886 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36380ab6e0 con 0x7f36441181b0 2024-03-28T16:48:53.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364159a640 1 -- 172.21.15.170:0/2794550886 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36380ab9a0 con 0x7f36441181b0 2024-03-28T16:48:53.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 -- 172.21.15.170:0/2794550886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36441181b0 msgr2=0x7f364411a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:53.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 --2- 172.21.15.170:0/2794550886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36441181b0 0x7f364411a5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36380097f0 tx=0x7f363801c430 comp rx=0 tx=0).stop 2024-03-28T16:48:53.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 -- 172.21.15.170:0/2794550886 shutdown_connections 2024-03-28T16:48:53.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 --2- 172.21.15.170:0/2794550886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36441181b0 0x7f364411a5a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:53.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 -- 172.21.15.170:0/2794550886 >> 172.21.15.170:0/2794550886 conn(0x7f364406e870 msgr2=0x7f3644070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:53.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 -- 172.21.15.170:0/2794550886 shutdown_connections 2024-03-28T16:48:53.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 -- 172.21.15.170:0/2794550886 wait complete. 2024-03-28T16:48:53.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 Processor -- start 2024-03-28T16:48:53.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 -- start start 2024-03-28T16:48:53.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364410e020 0x7f364410f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:53.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f3648aac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364410f600 con 0x7f364410e020 2024-03-28T16:48:53.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364410e020 0x7f364410f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:53.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364410e020 0x7f364410f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49164/0 (socket says 172.21.15.170:49164) 2024-03-28T16:48:53.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364259c640 1 -- 172.21.15.170:0/973402819 learned_addr learned my addr 172.21.15.170:0/973402819 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:53.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364259c640 1 -- 172.21.15.170:0/973402819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380094d0 con 0x7f364410e020 2024-03-28T16:48:53.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.791+0000 7f364259c640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364410e020 0x7f364410f030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f363400c450 tx=0x7f363400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:53.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3632ffd640 1 -- 172.21.15.170:0/973402819 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3634005150 con 0x7f364410e020 2024-03-28T16:48:53.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3632ffd640 1 -- 172.21.15.170:0/973402819 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36340052b0 con 0x7f364410e020 2024-03-28T16:48:53.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364410f800 con 0x7f364410e020 2024-03-28T16:48:53.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3632ffd640 1 -- 172.21.15.170:0/973402819 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3634007030 con 0x7f364410e020 2024-03-28T16:48:53.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364410fc80 con 0x7f364410e020 2024-03-28T16:48:53.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3644062860 con 0x7f364410e020 2024-03-28T16:48:53.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3632ffd640 1 -- 172.21.15.170:0/973402819 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3634007260 con 0x7f364410e020 2024-03-28T16:48:53.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3632ffd640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3624035a10 0x7f3624037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:53.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3632ffd640 1 -- 172.21.15.170:0/973402819 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3845+0+0 (secure 0 0 0) 0x7f36340477f0 con 0x7f364410e020 2024-03-28T16:48:53.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3641d9b640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3624035a10 0x7f3624037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:53.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.795+0000 7f3641d9b640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3624035a10 0x7f3624037ed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36380af000 tx=0x7f3638008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:53.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.807+0000 7f3632ffd640 1 -- 172.21.15.170:0/973402819 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3634007510 con 0x7f364410e020 2024-03-28T16:48:53.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:53.971+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) v1 -- 0x7f36441181b0 con 0x7f364410e020 2024-03-28T16:48:54.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.631+0000 7f3632ffd640 1 -- 172.21.15.170:0/973402819 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v18) v1 ==== 185+0+0 (secure 0 0 0) 0x7f36340102e0 con 0x7f364410e020 2024-03-28T16:48:54.632 INFO:teuthology.orchestra.run.smithi170.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-03-28T16:48:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3624035a10 msgr2=0x7f3624037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:54.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3624035a10 0x7f3624037ed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f36380af000 tx=0x7f3638008040 comp rx=0 tx=0).stop 2024-03-28T16:48:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364410e020 msgr2=0x7f364410f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364410e020 0x7f364410f030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f363400c450 tx=0x7f363400c910 comp rx=0 tx=0).stop 2024-03-28T16:48:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 shutdown_connections 2024-03-28T16:48:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3624035a10 0x7f3624037ed0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 --2- 172.21.15.170:0/973402819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364410e020 0x7f364410f030 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:54.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 >> 172.21.15.170:0/973402819 conn(0x7f364406e870 msgr2=0x7f3644071320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 shutdown_connections 2024-03-28T16:48:54.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.635+0000 7f3648aac640 1 -- 172.21.15.170:0/973402819 wait complete. 2024-03-28T16:48:54.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-03-28T16:48:54.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffae280d640 1 Processor -- start 2024-03-28T16:48:54.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffae280d640 1 -- start start 2024-03-28T16:48:54.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffae280d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffadc115400 0x7ffadc1157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:54.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffae280d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc115e70 con 0x7ffadc10f160 2024-03-28T16:48:54.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffae280d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc10fd20 con 0x7ffadc115400 2024-03-28T16:48:54.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffadbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffadc115400 0x7ffadc1157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:54.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffadb7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffadc10f160 0x7ffadc10f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:50318/0 (socket says 172.21.15.170:50318) 2024-03-28T16:48:54.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffadb7fe640 1 -- 172.21.15.170:0/415004162 learned_addr learned my addr 172.21.15.170:0/415004162 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:48:54.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffadbfff640 1 -- 172.21.15.170:0/415004162 >> v1:172.21.15.119:6789/0 conn(0x7ffadc10f160 legacy=0x7ffadc10f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:54.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.791+0000 7ffadbfff640 1 -- 172.21.15.170:0/415004162 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc10fe90 con 0x7ffadc115400 2024-03-28T16:48:54.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadbfff640 1 --2- 172.21.15.170:0/415004162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc115400 0x7ffadc1157e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009740 tx=0x7ffacc01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7aed2b18e89e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:54.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadaffd640 1 -- 172.21.15.170:0/415004162 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffacc0a8050 con 0x7ffadc115400 2024-03-28T16:48:54.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadaffd640 1 -- 172.21.15.170:0/415004162 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc0ac520 con 0x7ffadc115400 2024-03-28T16:48:54.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadaffd640 1 -- 172.21.15.170:0/415004162 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffacc0ac7e0 con 0x7ffadc115400 2024-03-28T16:48:54.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 -- 172.21.15.170:0/415004162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc115400 msgr2=0x7ffadc1157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:54.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 --2- 172.21.15.170:0/415004162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc115400 0x7ffadc1157e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009740 tx=0x7ffacc01c370 comp rx=0 tx=0).stop 2024-03-28T16:48:54.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 -- 172.21.15.170:0/415004162 shutdown_connections 2024-03-28T16:48:54.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 --2- 172.21.15.170:0/415004162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc115400 0x7ffadc1157e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:54.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 -- 172.21.15.170:0/415004162 >> 172.21.15.170:0/415004162 conn(0x7ffadc070360 msgr2=0x7ffadc072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:54.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 -- 172.21.15.170:0/415004162 shutdown_connections 2024-03-28T16:48:54.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 -- 172.21.15.170:0/415004162 wait complete. 2024-03-28T16:48:54.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 Processor -- start 2024-03-28T16:48:54.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 -- start start 2024-03-28T16:48:54.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10f160 0x7ffadc1ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:54.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffae280d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc1aca00 con 0x7ffadc10f160 2024-03-28T16:48:54.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10f160 0x7ffadc1ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:54.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10f160 0x7ffadc1ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49168/0 (socket says 172.21.15.170:49168) 2024-03-28T16:48:54.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadbfff640 1 -- 172.21.15.170:0/2272592328 learned_addr learned my addr 172.21.15.170:0/2272592328 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:54.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadbfff640 1 -- 172.21.15.170:0/2272592328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc009420 con 0x7ffadc10f160 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffadbfff640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10f160 0x7ffadc1ac430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0af040 tx=0x7ffacc01abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.795+0000 7ffad8ff9640 1 -- 172.21.15.170:0/2272592328 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffacc0ab8d0 con 0x7ffadc10f160 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffad8ff9640 1 -- 172.21.15.170:0/2272592328 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc0aba50 con 0x7ffadc10f160 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc1acba0 con 0x7ffadc10f160 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffad8ff9640 1 -- 172.21.15.170:0/2272592328 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffacc0abd30 con 0x7ffadc10f160 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc1a62c0 con 0x7ffadc10f160 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc062860 con 0x7ffadc10f160 2024-03-28T16:48:54.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffad8ff9640 1 -- 172.21.15.170:0/2272592328 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ffacc0b8450 con 0x7ffadc10f160 2024-03-28T16:48:54.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffad8ff9640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffac80359c0 0x7ffac8037e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:54.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffad8ff9640 1 -- 172.21.15.170:0/2272592328 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3856+0+0 (secure 0 0 0) 0x7ffacc0e3110 con 0x7ffadc10f160 2024-03-28T16:48:54.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffadb7fe640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffac80359c0 0x7ffac8037e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:54.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.799+0000 7ffadb7fe640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffac80359c0 0x7ffac8037e80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffad0001610 tx=0x7ffad0007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:54.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.811+0000 7ffad8ff9640 1 -- 172.21.15.170:0/2272592328 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffacc0c9770 con 0x7ffadc10f160 2024-03-28T16:48:54.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:54.971+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7ffadc067690 con 0x7ffadc10f160 2024-03-28T16:48:55.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.639+0000 7ffad8ff9640 1 -- 172.21.15.170:0/2272592328 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v19) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffacc0b1070 con 0x7ffadc10f160 2024-03-28T16:48:55.641 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'default.rgw.buckets.index' created 2024-03-28T16:48:55.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffac80359c0 msgr2=0x7ffac8037e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:55.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffac80359c0 0x7ffac8037e80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffad0001610 tx=0x7ffad0007ad0 comp rx=0 tx=0).stop 2024-03-28T16:48:55.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10f160 msgr2=0x7ffadc1ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:55.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10f160 0x7ffadc1ac430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0af040 tx=0x7ffacc01abb0 comp rx=0 tx=0).stop 2024-03-28T16:48:55.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 shutdown_connections 2024-03-28T16:48:55.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffac80359c0 0x7ffac8037e80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:55.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 --2- 172.21.15.170:0/2272592328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffadc10f160 0x7ffadc1ac430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:55.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 >> 172.21.15.170:0/2272592328 conn(0x7ffadc070360 msgr2=0x7ffadc11a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:55.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 shutdown_connections 2024-03-28T16:48:55.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.643+0000 7ffae280d640 1 -- 172.21.15.170:0/2272592328 wait complete. 2024-03-28T16:48:55.676 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-03-28T16:48:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a68012640 1 Processor -- start 2024-03-28T16:48:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a68012640 1 -- start start 2024-03-28T16:48:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a68012640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7a60118290 0x7f7a6011a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a68012640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6011abc0 con 0x7f7a6010e020 2024-03-28T16:48:55.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a68012640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6010ec50 con 0x7f7a60118290 2024-03-28T16:48:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a655ad640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7a60118290 0x7f7a6011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a655ad640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7a60118290 0x7f7a6011a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49172/0 (socket says 172.21.15.170:49172) 2024-03-28T16:48:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a655ad640 1 -- 172.21.15.170:0/903184806 learned_addr learned my addr 172.21.15.170:0/903184806 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a655ad640 1 -- 172.21.15.170:0/903184806 >> v1:172.21.15.119:6789/0 conn(0x7f7a6010e020 legacy=0x7f7a6010e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.803+0000 7f7a655ad640 1 -- 172.21.15.170:0/903184806 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6011bd60 con 0x7f7a60118290 2024-03-28T16:48:55.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a655ad640 1 --2- 172.21.15.170:0/903184806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60118290 0x7f7a6011a680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7a500097f0 tx=0x7f7a5001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397243e393c5c87c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:55.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a64dac640 1 -- 172.21.15.170:0/903184806 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a500ab580 con 0x7f7a60118290 2024-03-28T16:48:55.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a64dac640 1 -- 172.21.15.170:0/903184806 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a500ab6e0 con 0x7f7a60118290 2024-03-28T16:48:55.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a64dac640 1 -- 172.21.15.170:0/903184806 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a500ab9a0 con 0x7f7a60118290 2024-03-28T16:48:55.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 -- 172.21.15.170:0/903184806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60118290 msgr2=0x7f7a6011a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 --2- 172.21.15.170:0/903184806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60118290 0x7f7a6011a680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7a500097f0 tx=0x7f7a5001c430 comp rx=0 tx=0).stop 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 -- 172.21.15.170:0/903184806 shutdown_connections 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 --2- 172.21.15.170:0/903184806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60118290 0x7f7a6011a680 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 -- 172.21.15.170:0/903184806 >> 172.21.15.170:0/903184806 conn(0x7f7a6006ee90 msgr2=0x7f7a600712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 -- 172.21.15.170:0/903184806 shutdown_connections 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 -- 172.21.15.170:0/903184806 wait complete. 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 Processor -- start 2024-03-28T16:48:55.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 -- start start 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6010e020 0x7f7a601ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a68012640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a601ac900 con 0x7f7a6010e020 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a65dae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6010e020 0x7f7a601ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a65dae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6010e020 0x7f7a601ac330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49176/0 (socket says 172.21.15.170:49176) 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a65dae640 1 -- 172.21.15.170:0/3235237120 learned_addr learned my addr 172.21.15.170:0/3235237120 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a65dae640 1 -- 172.21.15.170:0/3235237120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500094d0 con 0x7f7a6010e020 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.807+0000 7f7a65dae640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6010e020 0x7f7a601ac330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00c3b0 tx=0x7f7a5c00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a567fc640 1 -- 172.21.15.170:0/3235237120 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c005200 con 0x7f7a6010e020 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a567fc640 1 -- 172.21.15.170:0/3235237120 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c005380 con 0x7f7a6010e020 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a601acaa0 con 0x7f7a6010e020 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a567fc640 1 -- 172.21.15.170:0/3235237120 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5c007020 con 0x7f7a6010e020 2024-03-28T16:48:55.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a601a61c0 con 0x7f7a6010e020 2024-03-28T16:48:55.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a60062860 con 0x7f7a6010e020 2024-03-28T16:48:55.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a567fc640 1 -- 172.21.15.170:0/3235237120 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7a5c007180 con 0x7f7a6010e020 2024-03-28T16:48:55.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a567fc640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7a440359c0 0x7f7a44037e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:48:55.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a567fc640 1 -- 172.21.15.170:0/3235237120 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4220+0+0 (secure 0 0 0) 0x7f7a5c047900 con 0x7f7a6010e020 2024-03-28T16:48:55.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a655ad640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7a440359c0 0x7f7a44037e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:48:55.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.811+0000 7f7a655ad640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7a440359c0 0x7f7a44037e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7a500097f0 tx=0x7f7a50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:48:55.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.823+0000 7f7a567fc640 1 -- 172.21.15.170:0/3235237120 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a5c019c30 con 0x7f7a6010e020 2024-03-28T16:48:55.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:55.983+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) v1 -- 0x7f7a6011b900 con 0x7f7a6010e020 2024-03-28T16:48:56.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.647+0000 7f7a567fc640 1 -- 172.21.15.170:0/3235237120 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v20) v1 ==== 187+0+0 (secure 0 0 0) 0x7f7a5c020070 con 0x7f7a6010e020 2024-03-28T16:48:56.647 INFO:teuthology.orchestra.run.smithi170.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7a440359c0 msgr2=0x7f7a44037e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7a440359c0 0x7f7a44037e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7a500097f0 tx=0x7f7a50008040 comp rx=0 tx=0).stop 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6010e020 msgr2=0x7f7a601ac330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6010e020 0x7f7a601ac330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00c3b0 tx=0x7f7a5c00c870 comp rx=0 tx=0).stop 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 shutdown_connections 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7a440359c0 0x7f7a44037e80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 --2- 172.21.15.170:0/3235237120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a6010e020 0x7f7a601ac330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 >> 172.21.15.170:0/3235237120 conn(0x7f7a6006ee90 msgr2=0x7f7a600700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:48:56.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 shutdown_connections 2024-03-28T16:48:56.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:48:56.651+0000 7f7a68012640 1 -- 172.21.15.170:0/3235237120 wait complete. 2024-03-28T16:48:56.704 DEBUG:tasks.rgw:Pools created 2024-03-28T16:48:56.704 INFO:tasks.rgw:Starting rgw... 2024-03-28T16:48:56.704 INFO:tasks.rgw:rgw client.0 config is {} 2024-03-28T16:48:56.704 INFO:tasks.rgw:Using beast as radosgw frontend 2024-03-28T16:48:56.705 INFO:tasks.rgw.client.0:Restarting daemon 2024-03-28T16:48:56.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-03-28T16:48:56.707 INFO:tasks.rgw.client.0:Started 2024-03-28T16:48:56.707 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi170.front.sepia.ceph.com:80/ 2024-03-28T16:48:56.707 DEBUG:teuthology.orchestra.run.smithi170:> curl http://smithi170.front.sepia.ceph.com:80/ 2024-03-28T16:48:56.738 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-28T16:48:56.739 INFO:teuthology.orchestra.run.smithi170.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-28T16:48:56.739 INFO:teuthology.orchestra.run.smithi170.stderr: Dload Upload Total Spent Left Speed 2024-03-28T16:48:56.739 INFO:teuthology.orchestra.run.smithi170.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-28T16:48:56.739 INFO:teuthology.orchestra.run.smithi170.stderr:curl: (7) Failed to connect to smithi170.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-03-28T16:48:57.739 DEBUG:teuthology.orchestra.run.smithi170:> curl http://smithi170.front.sepia.ceph.com:80/ 2024-03-28T16:48:57.751 INFO:teuthology.orchestra.run.smithi170.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-28T16:48:57.751 INFO:teuthology.orchestra.run.smithi170.stderr: Dload Upload Total Spent Left Speed 2024-03-28T16:48:57.751 INFO:teuthology.orchestra.run.smithi170.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-28T16:48:57.751 INFO:teuthology.orchestra.run.smithi170.stderr:curl: (7) Failed to connect to smithi170.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-03-28T16:48:57.752 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-28T16:48:59.753 DEBUG:teuthology.orchestra.run.smithi170:> curl http://smithi170.front.sepia.ceph.com:80/ 2024-03-28T16:48:59.765 INFO:teuthology.orchestra.run.smithi170.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-28T16:48:59.765 INFO:teuthology.orchestra.run.smithi170.stderr: Dload Upload Total Spent Left Speed 2024-03-28T16:48:59.765 INFO:teuthology.orchestra.run.smithi170.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-28T16:48:59.766 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-28T16:48:59.766 INFO:teuthology.orchestra.run.smithi170.stderr:curl: (7) Failed to connect to smithi170.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-03-28T16:49:03.767 DEBUG:teuthology.orchestra.run.smithi170:> curl http://smithi170.front.sepia.ceph.com:80/ 2024-03-28T16:49:03.779 INFO:teuthology.orchestra.run.smithi170.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-28T16:49:03.779 INFO:teuthology.orchestra.run.smithi170.stderr: Dload Upload Total Spent Left Speed 2024-03-28T16:49:03.779 INFO:teuthology.orchestra.run.smithi170.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-03-28T16:49:03.780 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-03-28T16:49:03.780 INFO:teuthology.orchestra.run.smithi170.stderr:curl: (7) Failed to connect to smithi170.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-03-28T16:49:11.781 DEBUG:teuthology.orchestra.run.smithi170:> curl http://smithi170.front.sepia.ceph.com:80/ 2024-03-28T16:49:11.792 INFO:teuthology.orchestra.run.smithi170.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-03-28T16:49:11.792 INFO:teuthology.orchestra.run.smithi170.stderr: Dload Upload Total Spent Left Speed 2024-03-28T16:49:11.796 INFO:teuthology.orchestra.run.smithi170.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 54605 0 --:--:-- --:--:-- --:--:-- 71333 2024-03-28T16:49:11.797 INFO:teuthology.orchestra.run.smithi170.stdout:anonymous 2024-03-28T16:49:11.797 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-03-28T16:49:11.876 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-03-28T16:49:11.876 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-03-28T16:49:11.876 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-03-28T16:49:11.876 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-03-28T16:49:11.912 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:11.912 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:11.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.911+0000 7fd984f0c980 1 Processor -- start 2024-03-28T16:49:11.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.911+0000 7fd984f0c980 1 -- start start 2024-03-28T16:49:11.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd984f0c980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b731a0e8a0 0x55b731a9cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd984f0c980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b7319e6510 con 0x55b731a9d160 2024-03-28T16:49:11.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd984f0c980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b7319ad0f0 con 0x55b731a0e8a0 2024-03-28T16:49:11.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd982a27640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b731a0e8a0 0x55b731a9cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd982a27640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b731a0e8a0 0x55b731a9cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54730/0 (socket says 172.21.15.170:54730) 2024-03-28T16:49:11.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd982a27640 1 -- 172.21.15.170:0/350550877 learned_addr learned my addr 172.21.15.170:0/350550877 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:11.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd982a27640 1 -- 172.21.15.170:0/350550877 >> v1:172.21.15.119:6789/0 conn(0x55b731a9d160 legacy=0x55b731a9f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:11.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.915+0000 7fd982a27640 1 -- 172.21.15.170:0/350550877 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b731aa1cb0 con 0x55b731a0e8a0 2024-03-28T16:49:11.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd982a27640 1 --2- 172.21.15.170:0/350550877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9cc20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd97400a5c0 tx=0x7fd97401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22da47d6c5dc2e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd981a25640 1 -- 172.21.15.170:0/350550877 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740aa470 con 0x55b731a0e8a0 2024-03-28T16:49:11.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd981a25640 1 -- 172.21.15.170:0/350550877 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740aa5d0 con 0x55b731a0e8a0 2024-03-28T16:49:11.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd981a25640 1 -- 172.21.15.170:0/350550877 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740aa890 con 0x55b731a0e8a0 2024-03-28T16:49:11.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd984f0c980 1 -- 172.21.15.170:0/350550877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 msgr2=0x55b731a9cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:11.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd984f0c980 1 --2- 172.21.15.170:0/350550877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9cc20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd97400a5c0 tx=0x7fd97401b340 comp rx=0 tx=0).stop 2024-03-28T16:49:11.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd984f0c980 1 -- 172.21.15.170:0/350550877 shutdown_connections 2024-03-28T16:49:11.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd984f0c980 1 --2- 172.21.15.170:0/350550877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9cc20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:11.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd984f0c980 1 -- 172.21.15.170:0/350550877 >> 172.21.15.170:0/350550877 conn(0x55b731a94700 msgr2=0x55b731a96b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:11.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd984f0c980 1 -- 172.21.15.170:0/350550877 shutdown_connections 2024-03-28T16:49:11.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.919+0000 7fd984f0c980 1 -- 172.21.15.170:0/350550877 wait complete. 2024-03-28T16:49:11.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd984f0c980 1 Processor -- start 2024-03-28T16:49:11.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd984f0c980 1 -- start start 2024-03-28T16:49:11.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd984f0c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd984f0c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7319aa620 con 0x55b731a0e8a0 2024-03-28T16:49:11.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54736/0 (socket says 172.21.15.170:54736) 2024-03-28T16:49:11.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd982a27640 1 -- 172.21.15.170:0/3448281380 learned_addr learned my addr 172.21.15.170:0/3448281380 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:11.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd982a27640 1 -- 172.21.15.170:0/3448281380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd974005df0 con 0x55b731a0e8a0 2024-03-28T16:49:11.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd982a27640 1 --2- 172.21.15.170:0/3448281380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9a380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ac040 tx=0x7fd974003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646440ddd22dc7bf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.935+0000 7fd870e09640 1 -- 172.21.15.170:0/3448281380 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740aa9b0 con 0x55b731a0e8a0 2024-03-28T16:49:11.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd870e09640 1 -- 172.21.15.170:0/3448281380 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740aab30 con 0x55b731a0e8a0 2024-03-28T16:49:11.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd870e09640 1 -- 172.21.15.170:0/3448281380 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740aae10 con 0x55b731a0e8a0 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- 172.21.15.170:0/3448281380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 msgr2=0x55b731a9a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3448281380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9a380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ac040 tx=0x7fd974003030 comp rx=0 tx=0).stop 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- 172.21.15.170:0/3448281380 shutdown_connections 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3448281380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731a9a380 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- 172.21.15.170:0/3448281380 >> 172.21.15.170:0/3448281380 conn(0x55b731a94700 msgr2=0x55b731b7fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- 172.21.15.170:0/3448281380 shutdown_connections 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- 172.21.15.170:0/3448281380 wait complete. 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 Processor -- start 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- start start 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731b8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7319a5e00 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731b8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731b8ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54752/0 (socket says 172.21.15.170:54752) 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd982a27640 1 -- 172.21.15.170:0/1236004177 learned_addr learned my addr 172.21.15.170:0/1236004177 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd982a27640 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b731aa1cb0 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd982a27640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731b8ce30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd97401b820 tx=0x7fd9740180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd86f606640 1 -- 172.21.15.170:0/1236004177 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740c6710 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b731b8d370 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd86f606640 1 -- 172.21.15.170:0/1236004177 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740c6870 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd86f606640 1 -- 172.21.15.170:0/1236004177 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740c6b10 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.939+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b731b8d7a0 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd86f606640 1 -- 172.21.15.170:0/1236004177 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd9740c6c70 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd86f606640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd868034300 0x7fd8680367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd86f606640 1 -- 172.21.15.170:0/1236004177 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fd9740e4af0 con 0x55b731a0e8a0 2024-03-28T16:49:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd982226640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd868034300 0x7fd8680367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731a9d160 0x55b731b93a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731b93fa0 con 0x55b731a9d160 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd983228640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731a9d160 0x55b731b93a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd982226640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd868034300 0x7fd8680367c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd978000f70 tx=0x7fd978001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd983228640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731a9d160 0x55b731b93a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd983228640 1 -- 172.21.15.170:0/1236004177 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731a9d160 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731b94300 con 0x55b731a9d160 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd983228640 1 -- 172.21.15.170:0/1236004177 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731a9d160 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731b95c90 0x55b731b980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731b98820 con 0x55b731b95c90 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd982a27640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731b95c90 0x55b731b980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.943+0000 7fd982a27640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731b95c90 0x55b731b980b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982a27640 1 -- 172.21.15.170:0/1236004177 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fd9740b0020 con 0x55b731b95c90 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731b94b00 con 0x55b731a9d160 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd983228640 1 -- 172.21.15.170:0/1236004177 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731a9d160 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b731b99c00 0x55b731b9bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731b9c650 con 0x55b731b99c00 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982226640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b731b99c00 0x55b731b9bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982226640 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b731b99c00 0x55b731b9bff0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982226640 1 -- 172.21.15.170:0/1236004177 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fd9780106b0 con 0x55b731b99c00 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 Processor -- start 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 -- start start 2024-03-28T16:49:11.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731baa580 0x55b731baa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b73193ee30 con 0x55b731baa580 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731baa580 0x55b731baa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731baa580 0x55b731baa9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54754/0 (socket says 172.21.15.170:54754) 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982a27640 1 -- 172.21.15.170:0/2425960555 learned_addr learned my addr 172.21.15.170:0/2425960555 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982a27640 1 -- 172.21.15.170:0/2425960555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b731baaee0 con 0x55b731baa580 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd982a27640 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731baa580 0x55b731baa9a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9740b1040 tx=0x7fd9740cf5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd86ce01640 1 -- 172.21.15.170:0/2425960555 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740cfa70 con 0x55b731baa580 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b731bab130 con 0x55b731baa580 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd86ce01640 1 -- 172.21.15.170:0/2425960555 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740ac000 con 0x55b731baa580 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd86ce01640 1 -- 172.21.15.170:0/2425960555 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740ac2a0 con 0x55b731baa580 2024-03-28T16:49:11.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.947+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b731babc80 con 0x55b731baa580 2024-03-28T16:49:11.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd86ce01640 1 -- 172.21.15.170:0/2425960555 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd9740bc5d0 con 0x55b731baa580 2024-03-28T16:49:11.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd86ce01640 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd85c0342b0 0x7fd85c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd86ce01640 1 -- 172.21.15.170:0/2425960555 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fd9740f7af0 con 0x55b731baa580 2024-03-28T16:49:11.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd982226640 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd85c0342b0 0x7fd85c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd984f0c980 1 Processor -- start 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd984f0c980 1 -- start start 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd984f0c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bb7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd984f0c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b731932790 con 0x55b731bb7560 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd982226640 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd85c0342b0 0x7fd85c036770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x55b731b94e60 tx=0x7fd978006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bb7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bb7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54762/0 (socket says 172.21.15.170:54762) 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd982a27640 1 -- 172.21.15.170:0/3501623340 learned_addr learned my addr 172.21.15.170:0/3501623340 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd982a27640 1 -- 172.21.15.170:0/3501623340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b731bb7f30 con 0x55b731bb7560 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd982a27640 1 --2- 172.21.15.170:0/3501623340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bb7960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9740acff0 tx=0x7fd9740ac940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f968d05abc0a7f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd8667fc640 1 -- 172.21.15.170:0/3501623340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740bc880 con 0x55b731bb7560 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd8667fc640 1 -- 172.21.15.170:0/3501623340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740fe070 con 0x55b731bb7560 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd8667fc640 1 -- 172.21.15.170:0/3501623340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740b3a90 con 0x55b731bb7560 2024-03-28T16:49:11.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd984f0c980 1 -- 172.21.15.170:0/3501623340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 msgr2=0x55b731bb7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.951+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3501623340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bb7960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd9740acff0 tx=0x7fd9740ac940 comp rx=0 tx=0).stop 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- 172.21.15.170:0/3501623340 shutdown_connections 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3501623340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bb7960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- 172.21.15.170:0/3501623340 >> 172.21.15.170:0/3501623340 conn(0x55b731bb38a0 msgr2=0x55b731bb1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- 172.21.15.170:0/3501623340 shutdown_connections 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- 172.21.15.170:0/3501623340 wait complete. 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 Processor -- start 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- start start 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bbe370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b731bbe8b0 con 0x55b731bb7560 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bbe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd982a27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bbe370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54778/0 (socket says 172.21.15.170:54778) 2024-03-28T16:49:11.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 learned_addr learned my addr 172.21.15.170:0/3253733518 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b731babc80 con 0x55b731bb7560 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd982a27640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bbe370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ded60 tx=0x7fd9740cdfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd864ff9640 1 -- 172.21.15.170:0/3253733518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740acd50 con 0x55b731bb7560 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b731babc80 con 0x55b731bb7560 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b731babc80 con 0x55b731bb7560 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd864ff9640 1 -- 172.21.15.170:0/3253733518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9740b33f0 con 0x55b731bb7560 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd864ff9640 1 -- 172.21.15.170:0/3253733518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9740b3690 con 0x55b731bb7560 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd864ff9640 1 -- 172.21.15.170:0/3253733518 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd9740e1140 con 0x55b731bb7560 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd864ff9640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd84c0342b0 0x7fd84c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd864ff9640 1 -- 172.21.15.170:0/3253733518 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fd974119ad0 con 0x55b731bb7560 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd982226640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd84c0342b0 0x7fd84c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731bc9620 0x55b731bcbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.955+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcc230 con 0x55b731bc9620 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd983228640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731bc9620 0x55b731bcbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd982226640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd84c0342b0 0x7fd84c036770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55b731b9ca10 tx=0x7fd978011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd983228640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731bc9620 0x55b731bcbae0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 realm 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcc600 con 0x55b731bc9620 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:11.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcd1b0 con 0x55b731bc9620 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731bcdb60 0x55b731bcff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd04d0 con 0x55b731bcdb60 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd982a27640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731bcdb60 0x55b731bcff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd982a27640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731bcdb60 0x55b731bcff50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:11.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 20 searching for the correct realm 2024-03-28T16:49:11.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b731bd2360 0x55b731bd4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd4ef0 con 0x55b731bd2360 2024-03-28T16:49:11.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd982226640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b731bd2360 0x55b731bd4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.959+0000 7fd982226640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b731bd2360 0x55b731bd4820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd982226640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd97800f3d0 0x7fd97800f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd4ef0 con 0x7fd97800f3d0 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd97800f3d0 0x7fd97800f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd97800f3d0 0x7fd97800f810 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.963+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:11.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.967+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:11.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.971+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:11.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:11.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:11.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:11.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:11.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:11.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:11.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:11.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd0cd0 con 0x55b731bc9620 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd0cd0 con 0x55b731bc9620 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd0cd0 con 0x7fd97800f3d0 2024-03-28T16:49:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 zone default found 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 4 Realm: () 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 zonegroup default 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.975+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd1520 con 0x55b731bcdb60 2024-03-28T16:49:11.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:11.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd1920 con 0x55b731bd2360 2024-03-28T16:49:11.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd6dd0 con 0x55b731bd2360 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 20 started sync module instance, tier type = 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd65f0 con 0x7fd97800f3d0 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94245301870928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd69a0 con 0x7fd97800f3d0 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd7b30 con 0x55b731bc9620 2024-03-28T16:49:11.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.979+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94245301870928] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.983+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.983+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94245301875184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd7b10 con 0x55b731bc9620 2024-03-28T16:49:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.983+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd85c0 con 0x55b731bd2360 2024-03-28T16:49:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.983+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94245301875184] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.983+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.983+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94245301878976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd8920 con 0x55b731bd2360 2024-03-28T16:49:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.983+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd9290 con 0x7fd97800f3d0 2024-03-28T16:49:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94245301882656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd9ae0 con 0x7fd97800f3d0 2024-03-28T16:49:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bda4b0 con 0x55b731bc9620 2024-03-28T16:49:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94245301878976] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94245301882656] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94245301886896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bc9620 2024-03-28T16:49:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.987+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bdad00 con 0x55b731bcdb60 2024-03-28T16:49:11.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.991+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94245301886896] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.991+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.991+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94245301889120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.991+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bdb500 con 0x55b731bcdb60 2024-03-28T16:49:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.991+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94245301889120] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94245301891168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bdb920 con 0x55b731bcdb60 2024-03-28T16:49:11.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bdc290 con 0x55b731bcdb60 2024-03-28T16:49:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94245301891168] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94245301892224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bdc290 con 0x55b731bcdb60 2024-03-28T16:49:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 20 add_watcher() i=0 2024-03-28T16:49:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 20 add_watcher() i=1 2024-03-28T16:49:11.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 20 add_watcher() i=2 2024-03-28T16:49:11.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 20 add_watcher() i=3 2024-03-28T16:49:11.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 20 add_watcher() i=4 2024-03-28T16:49:11.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 20 add_watcher() i=5 2024-03-28T16:49:11.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.995+0000 7fd984f0c980 20 add_watcher() i=6 2024-03-28T16:49:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94245301892224] v29'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 20 add_watcher() i=7 2024-03-28T16:49:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:11.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bdf6b0 con 0x55b731bc9620 2024-03-28T16:49:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:11.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94245301895104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bdcbf0 con 0x55b731bc9620 2024-03-28T16:49:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94245301895104] v29'24 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd8397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 5 note: GC not initialized 2024-03-28T16:49:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd7b3fff640 20 reqs_thread_entry: start 2024-03-28T16:49:12.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:12.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 20 Filter name: none 2024-03-28T16:49:12.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd7b1ffb640 20 reqs_thread_entry: start 2024-03-28T16:49:12.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:11.999+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731c00e60 con 0x55b731bd2360 2024-03-28T16:49:12.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(62 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731c00e60 con 0x55b731bd2360 2024-03-28T16:49:12.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(63 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731c00e60 con 0x55b731bc9620 2024-03-28T16:49:12.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(64 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:12.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:12.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:12.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(67 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:12.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:12.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(69 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:12.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:12.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(71 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.003+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:12.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(72 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bcdb60 2024-03-28T16:49:12.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(73 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:12.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(74 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bd2360 2024-03-28T16:49:12.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(75 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:12.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(76 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:12.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(77 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(78 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.007+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bd2360 2024-03-28T16:49:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(79 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x7fd97800f3d0 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(80 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bc9620 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(81 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bc9620 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(82 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bd2360 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(83 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x7fd97800f3d0 2024-03-28T16:49:12.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(84 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:12.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(85 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bc9620 2024-03-28T16:49:12.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(86 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:12.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(87 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.011+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bd2360 2024-03-28T16:49:12.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(88 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x7fd97800f3d0 2024-03-28T16:49:12.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(89 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bcdb60 2024-03-28T16:49:12.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(90 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bc9620 2024-03-28T16:49:12.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(91 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x55b731bd2360 2024-03-28T16:49:12.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(92 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd95c0 con 0x7fd97800f3d0 2024-03-28T16:49:12.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(93 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd8397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.015+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94245301895104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:12.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94245301895104] v29'25 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94245301870928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x7fd97800f3d0 2024-03-28T16:49:12.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94245301870928] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd984f0c980 20 remove_watcher() i=0 2024-03-28T16:49:12.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd984f0c980 2 removed watcher, disabling cache 2024-03-28T16:49:12.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94245301875184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bc9620 2024-03-28T16:49:12.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94245301875184] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd984f0c980 20 remove_watcher() i=1 2024-03-28T16:49:12.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.019+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94245301878976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bcce50 con 0x55b731bd2360 2024-03-28T16:49:12.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.023+0000 7fd982226640 1 -- 172.21.15.170:0/3253733518 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94245301878976] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd978032040 con 0x55b731bd2360 2024-03-28T16:49:12.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.023+0000 7fd984f0c980 20 remove_watcher() i=2 2024-03-28T16:49:12.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.023+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94245301882656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731c00e20 con 0x7fd97800f3d0 2024-03-28T16:49:12.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.023+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94245301882656] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x7fd97800f3d0 2024-03-28T16:49:12.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.023+0000 7fd984f0c980 20 remove_watcher() i=3 2024-03-28T16:49:12.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.023+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94245301886896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731c00e20 con 0x55b731bc9620 2024-03-28T16:49:12.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.027+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94245301886896] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd97c00a4d0 con 0x55b731bc9620 2024-03-28T16:49:12.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.027+0000 7fd984f0c980 20 remove_watcher() i=4 2024-03-28T16:49:12.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.027+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94245301889120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731c00e20 con 0x55b731bcdb60 2024-03-28T16:49:12.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.027+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94245301889120] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.027+0000 7fd984f0c980 20 remove_watcher() i=5 2024-03-28T16:49:12.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.027+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94245301891168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd71f0 con 0x55b731bcdb60 2024-03-28T16:49:12.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94245301891168] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 20 remove_watcher() i=6 2024-03-28T16:49:12.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94245301892224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b731bd80c0 con 0x55b731bcdb60 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd982a27640 1 -- 172.21.15.170:0/3253733518 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94245301892224] v29'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd974115070 con 0x55b731bcdb60 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 20 remove_watcher() i=7 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731bcdb60 msgr2=0x55b731bcff50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731bcdb60 0x55b731bcff50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731bc9620 msgr2=0x55b731bcbae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731bc9620 0x55b731bcbae0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd97800f3d0 msgr2=0x7fd97800f810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd97800f3d0 0x7fd97800f810 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b731bd2360 msgr2=0x55b731bd4820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b731bd2360 0x55b731bd4820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd84c0342b0 msgr2=0x7fd84c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd84c0342b0 0x7fd84c036770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55b731b9ca10 tx=0x7fd978011000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 msgr2=0x55b731bbe370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd983228640 1 -- 172.21.15.170:0/3253733518 reap_dead start 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.031+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bbe370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9740ded60 tx=0x7fd9740cdfd0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 shutdown_connections 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/3253733518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731bb7560 0x55b731bbe370 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 >> 172.21.15.170:0/3253733518 conn(0x55b731bb38a0 msgr2=0x55b731bb0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 shutdown_connections 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/3253733518 wait complete. 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd85c0342b0 msgr2=0x7fd85c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd85c0342b0 0x7fd85c036770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x55b731b94e60 tx=0x7fd978006000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731baa580 msgr2=0x55b731baa9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731baa580 0x55b731baa9a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9740b1040 tx=0x7fd9740cf5e0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 shutdown_connections 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd85c0342b0 0x7fd85c036770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/2425960555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731baa580 0x55b731baa9a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 >> 172.21.15.170:0/2425960555 conn(0x55b731b9f390 msgr2=0x55b731b9fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 shutdown_connections 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/2425960555 wait complete. 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731b95c90 msgr2=0x55b731b980b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b731b95c90 0x55b731b980b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731a9d160 msgr2=0x55b731b93a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b731a9d160 0x55b731b93a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b731b99c00 msgr2=0x55b731b9bff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b731b99c00 0x55b731b9bff0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd868034300 msgr2=0x7fd8680367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd868034300 0x7fd8680367c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd978000f70 tx=0x7fd978001420 comp rx=0 tx=0).stop 2024-03-28T16:49:12.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 msgr2=0x55b731b8ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 --2- 172.21.15.170:0/1236004177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b731a0e8a0 0x55b731b8ce30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd97401b820 tx=0x7fd9740180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd983228640 1 -- 172.21.15.170:0/1236004177 reap_dead start 2024-03-28T16:49:12.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 shutdown_connections 2024-03-28T16:49:12.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 >> 172.21.15.170:0/1236004177 conn(0x55b731a94700 msgr2=0x55b731a98c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 shutdown_connections 2024-03-28T16:49:12.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.035+0000 7fd984f0c980 1 -- 172.21.15.170:0/1236004177 wait complete. 2024-03-28T16:49:12.045 INFO:teuthology.orchestra.run.smithi170.stdout:[] 2024-03-28T16:49:12.045 DEBUG:tasks.util.rgw: json result: [] 2024-03-28T16:49:12.045 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-03-28T16:49:12.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-03-28T16:49:12.046 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-03-28T16:49:12.124 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:12.124 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:12.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.123+0000 7f39ffe61980 1 Processor -- start 2024-03-28T16:49:12.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.123+0000 7f39ffe61980 1 -- start start 2024-03-28T16:49:12.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.123+0000 7f39ffe61980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55793eea2630 0x55793eea4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.123+0000 7f39ffe61980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55793ede8520 con 0x55793ee9e130 2024-03-28T16:49:12.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.123+0000 7f39ffe61980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55793edaf0f0 con 0x55793eea2630 2024-03-28T16:49:12.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fd17b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55793eea2630 0x55793eea4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fd17b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55793eea2630 0x55793eea4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54786/0 (socket says 172.21.15.170:54786) 2024-03-28T16:49:12.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fd17b640 1 -- 172.21.15.170:0/1427715275 learned_addr learned my addr 172.21.15.170:0/1427715275 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fd97c640 1 --1- 172.21.15.170:0/1427715275 >> v1:172.21.15.119:6789/0 conn(0x55793ee9e130 0x55793ee9e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60768/0 (socket says 172.21.15.170:60768) 2024-03-28T16:49:12.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fd17b640 1 -- 172.21.15.170:0/1427715275 >> v1:172.21.15.119:6789/0 conn(0x55793ee9e130 legacy=0x55793ee9e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fd17b640 1 -- 172.21.15.170:0/1427715275 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793eea5f70 con 0x55793eea2630 2024-03-28T16:49:12.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fd17b640 1 --2- 172.21.15.170:0/1427715275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793eea2630 0x55793eea4a20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f39f000a4e0 tx=0x7f39f001b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f927d8a76c10cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fc97a640 1 -- 172.21.15.170:0/1427715275 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39f00aa540 con 0x55793eea2630 2024-03-28T16:49:12.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fc97a640 1 -- 172.21.15.170:0/1427715275 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f00a6070 con 0x55793eea2630 2024-03-28T16:49:12.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.127+0000 7f39fc97a640 1 -- 172.21.15.170:0/1427715275 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39f00aa850 con 0x55793eea2630 2024-03-28T16:49:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.131+0000 7f39ffe61980 1 -- 172.21.15.170:0/1427715275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793eea2630 msgr2=0x55793eea4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.131+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1427715275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793eea2630 0x55793eea4a20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f39f000a4e0 tx=0x7f39f001b090 comp rx=0 tx=0).stop 2024-03-28T16:49:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.131+0000 7f39ffe61980 1 -- 172.21.15.170:0/1427715275 shutdown_connections 2024-03-28T16:49:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.131+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1427715275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793eea2630 0x55793eea4a20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.131+0000 7f39ffe61980 1 -- 172.21.15.170:0/1427715275 >> 172.21.15.170:0/1427715275 conn(0x55793ee11850 msgr2=0x55793ee11c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.131+0000 7f39ffe61980 1 -- 172.21.15.170:0/1427715275 shutdown_connections 2024-03-28T16:49:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.131+0000 7f39ffe61980 1 -- 172.21.15.170:0/1427715275 wait complete. 2024-03-28T16:49:12.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 Processor -- start 2024-03-28T16:49:12.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- start start 2024-03-28T16:49:12.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef87fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793edac620 con 0x55793ee9e130 2024-03-28T16:49:12.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef87fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef87fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54802/0 (socket says 172.21.15.170:54802) 2024-03-28T16:49:12.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 -- 172.21.15.170:0/1269418083 learned_addr learned my addr 172.21.15.170:0/1269418083 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 -- 172.21.15.170:0/1269418083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f000a1c0 con 0x55793ee9e130 2024-03-28T16:49:12.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 --2- 172.21.15.170:0/1269418083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef87fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f39e8001d40 tx=0x7f39e800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaef3ac3d47262b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f38ebe07640 1 -- 172.21.15.170:0/1269418083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e8014070 con 0x55793ee9e130 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f38ebe07640 1 -- 172.21.15.170:0/1269418083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e800f590 con 0x55793ee9e130 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f38ebe07640 1 -- 172.21.15.170:0/1269418083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e8007040 con 0x55793ee9e130 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- 172.21.15.170:0/1269418083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 msgr2=0x55793ef87fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1269418083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef87fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f39e8001d40 tx=0x7f39e800b610 comp rx=0 tx=0).stop 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- 172.21.15.170:0/1269418083 shutdown_connections 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1269418083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef87fe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- 172.21.15.170:0/1269418083 >> 172.21.15.170:0/1269418083 conn(0x55793ee11850 msgr2=0x55793ef83b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- 172.21.15.170:0/1269418083 shutdown_connections 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- 172.21.15.170:0/1269418083 wait complete. 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 Processor -- start 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- start start 2024-03-28T16:49:12.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef8ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39ffe61980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793eda7e00 con 0x55793ee9e130 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef8ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef8ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54812/0 (socket says 172.21.15.170:54812) 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.147+0000 7f39fd97c640 1 -- 172.21.15.170:0/4057499017 learned_addr learned my addr 172.21.15.170:0/4057499017 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39fd97c640 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793ede8520 con 0x55793ee9e130 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39fd97c640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef8ee30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b650 tx=0x7f39e8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f38ea604640 1 -- 172.21.15.170:0/4057499017 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e8014030 con 0x55793ee9e130 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55793ef8f370 con 0x55793ee9e130 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f38ea604640 1 -- 172.21.15.170:0/4057499017 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8001530 con 0x55793ee9e130 2024-03-28T16:49:12.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f38ea604640 1 -- 172.21.15.170:0/4057499017 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e800eb10 con 0x55793ee9e130 2024-03-28T16:49:12.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793ef8f7a0 con 0x55793ee9e130 2024-03-28T16:49:12.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f38ea604640 1 -- 172.21.15.170:0/4057499017 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f39e800ed00 con 0x55793ee9e130 2024-03-28T16:49:12.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f38ea604640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38e40342b0 0x7f38e4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f38ea604640 1 -- 172.21.15.170:0/4057499017 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f39e8006070 con 0x55793ee9e130 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39fd17b640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38e40342b0 0x7f38e4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793eea2630 0x55793ef95ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793ef95ff0 con 0x55793eea2630 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39fe17d640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793eea2630 0x55793ef95ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39fd17b640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38e40342b0 0x7f38e4036770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39f00ab000 tx=0x7f39f00a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39fe17d640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793eea2630 0x55793ef95ab0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39fe17d640 1 -- 172.21.15.170:0/4057499017 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793eea2630 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.151+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793ef96350 con 0x55793eea2630 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fe17d640 1 -- 172.21.15.170:0/4057499017 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f39f80083f0 con 0x55793eea2630 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793ef98070 0x55793ef9a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793ef9a9a0 con 0x55793ef98070 2024-03-28T16:49:12.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fd97c640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793ef98070 0x55793ef9a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fd97c640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793ef98070 0x55793ef9a460 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fd97c640 1 -- 172.21.15.170:0/4057499017 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f39e8016070 con 0x55793ef98070 2024-03-28T16:49:12.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793ef96ee0 con 0x55793eea2630 2024-03-28T16:49:12.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fe17d640 1 -- 172.21.15.170:0/4057499017 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f39f80083f0 con 0x55793eea2630 2024-03-28T16:49:12.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55793ef9bd40 0x55793ef9e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793ef9e7d0 con 0x55793ef9bd40 2024-03-28T16:49:12.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fd17b640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55793ef9bd40 0x55793ef9e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fd17b640 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55793ef9bd40 0x55793ef9e130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39fd17b640 1 -- 172.21.15.170:0/4057499017 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793ef9bd40 2024-03-28T16:49:12.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39ffe61980 1 Processor -- start 2024-03-28T16:49:12.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.155+0000 7f39ffe61980 1 -- start start 2024-03-28T16:49:12.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efac700 0x55793efacb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793ed40e30 con 0x55793efac700 2024-03-28T16:49:12.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efac700 0x55793efacb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efac700 0x55793efacb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54818/0 (socket says 172.21.15.170:54818) 2024-03-28T16:49:12.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39fd97c640 1 -- 172.21.15.170:0/3676998324 learned_addr learned my addr 172.21.15.170:0/3676998324 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39fd97c640 1 -- 172.21.15.170:0/3676998324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793efad060 con 0x55793efac700 2024-03-28T16:49:12.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39fd97c640 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efac700 0x55793efacb20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39e8017820 tx=0x7f39e8017d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f38e37fe640 1 -- 172.21.15.170:0/3676998324 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e80417a0 con 0x55793efac700 2024-03-28T16:49:12.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55793efad2b0 con 0x55793efac700 2024-03-28T16:49:12.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f38e37fe640 1 -- 172.21.15.170:0/3676998324 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e8041900 con 0x55793efac700 2024-03-28T16:49:12.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f38e37fe640 1 -- 172.21.15.170:0/3676998324 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e8041ba0 con 0x55793efac700 2024-03-28T16:49:12.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793efade00 con 0x55793efac700 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f38e37fe640 1 -- 172.21.15.170:0/3676998324 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f39e80412c0 con 0x55793efac700 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f38e37fe640 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38d80342b0 0x7f38d8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f38e37fe640 1 -- 172.21.15.170:0/3676998324 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f39e8010070 con 0x55793efac700 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39fd17b640 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38d80342b0 0x7f38d8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 Processor -- start 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 -- start start 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39fd17b640 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38d80342b0 0x7f38d8036770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55793ef93540 tx=0x7f39f0005e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efb9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.159+0000 7f39ffe61980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793ed34790 con 0x55793efb9630 2024-03-28T16:49:12.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efb9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efb9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54820/0 (socket says 172.21.15.170:54820) 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 -- 172.21.15.170:0/2158132233 learned_addr learned my addr 172.21.15.170:0/2158132233 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 -- 172.21.15.170:0/2158132233 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793efba020 con 0x55793efb9630 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 --2- 172.21.15.170:0/2158132233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efb9a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39e8025a80 tx=0x7f39e8004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955d6e12fc57ef3f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f38e17fa640 1 -- 172.21.15.170:0/2158132233 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e801bcc0 con 0x55793efb9630 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f38e17fa640 1 -- 172.21.15.170:0/2158132233 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e801be20 con 0x55793efb9630 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f38e17fa640 1 -- 172.21.15.170:0/2158132233 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e80491a0 con 0x55793efb9630 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- 172.21.15.170:0/2158132233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 msgr2=0x55793efb9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 --2- 172.21.15.170:0/2158132233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efb9a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39e8025a80 tx=0x7f39e8004e60 comp rx=0 tx=0).stop 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- 172.21.15.170:0/2158132233 shutdown_connections 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 --2- 172.21.15.170:0/2158132233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efb9a50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- 172.21.15.170:0/2158132233 >> 172.21.15.170:0/2158132233 conn(0x55793efb59c0 msgr2=0x55793efb3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- 172.21.15.170:0/2158132233 shutdown_connections 2024-03-28T16:49:12.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- 172.21.15.170:0/2158132233 wait complete. 2024-03-28T16:49:12.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 Processor -- start 2024-03-28T16:49:12.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- start start 2024-03-28T16:49:12.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efc0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793efc0b70 con 0x55793efb9630 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efc0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efc0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54834/0 (socket says 172.21.15.170:54834) 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 learned_addr learned my addr 172.21.15.170:0/1598245117 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793efade00 con 0x55793efb9630 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39fd97c640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efc0630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39e8004ea0 tx=0x7f39e801b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f38cf7fe640 1 -- 172.21.15.170:0/1598245117 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e801a8b0 con 0x55793efb9630 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55793efc0d70 con 0x55793efb9630 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f38cf7fe640 1 -- 172.21.15.170:0/1598245117 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e801aa10 con 0x55793efb9630 2024-03-28T16:49:12.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f38cf7fe640 1 -- 172.21.15.170:0/1598245117 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e804a1c0 con 0x55793efb9630 2024-03-28T16:49:12.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.163+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55793efc11a0 con 0x55793efb9630 2024-03-28T16:49:12.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f38cf7fe640 1 -- 172.21.15.170:0/1598245117 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f39e804a320 con 0x55793efb9630 2024-03-28T16:49:12.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f38cf7fe640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38c8034300 0x7f38c80367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f38cf7fe640 1 -- 172.21.15.170:0/1598245117 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f39e8074b30 con 0x55793efb9630 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fd17b640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38c8034300 0x7f38c80367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793efcba70 0x55793efcdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efce680 con 0x55793efcba70 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fe17d640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793efcba70 0x55793efcdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fd17b640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38c8034300 0x7f38c80367c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55793ef9eb90 tx=0x7f39f00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fe17d640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793efcba70 0x55793efcdf30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 realm 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcea50 con 0x55793efcba70 2024-03-28T16:49:12.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf600 con 0x55793efcba70 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793efcffb0 0x55793efd23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.167+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd2a40 con 0x55793efcffb0 2024-03-28T16:49:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd97c640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793efcffb0 0x55793efd23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd97c640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793efcffb0 0x55793efd23a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39ffe61980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39ffe61980 20 searching for the correct realm 2024-03-28T16:49:12.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55793efd4c00 0x55793efd70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd7740 con 0x55793efd4c00 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd17b640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55793efd4c00 0x55793efd70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd17b640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55793efd4c00 0x55793efd70c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd17b640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39f00cd050 0x7f39f00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd7740 con 0x7f39f00cd050 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fe17d640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39f00cd050 0x7f39f00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fe17d640 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39f00cd050 0x7f39f00cd430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.171+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x7f39f00cd050 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.175+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x7f39f00cd050 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.179+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x7f39f00cd050 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x7f39f00cd050 2024-03-28T16:49:12.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efd4c00 2024-03-28T16:49:12.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.183+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcffb0 2024-03-28T16:49:12.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x55793efcba70 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efcf2a0 con 0x7f39f00cd050 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:12.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd3000 con 0x55793efcba70 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd3000 con 0x55793efcba70 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd2fb0 con 0x7f39f00cd050 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 zone default found 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 4 Realm: () 2024-03-28T16:49:12.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:12.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:12.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:12.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 zonegroup default 2024-03-28T16:49:12.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd3850 con 0x55793efcffb0 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd3c50 con 0x55793efd4c00 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd9600 con 0x55793efd4c00 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.187+0000 7f39ffe61980 20 started sync module instance, tier type = 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39ffe61980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:12.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd8e40 con 0x7f39f00cd050 2024-03-28T16:49:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93979236207008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd91f0 con 0x7f39f00cd050 2024-03-28T16:49:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efda340 con 0x55793efcba70 2024-03-28T16:49:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93979236207008] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93979236211200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efda320 con 0x55793efcba70 2024-03-28T16:49:12.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.191+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdadf0 con 0x55793efd4c00 2024-03-28T16:49:12.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.195+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93979236211200] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.195+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.195+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93979236214992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdb1a0 con 0x55793efd4c00 2024-03-28T16:49:12.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.195+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdbb10 con 0x7f39f00cd050 2024-03-28T16:49:12.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.195+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.195+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93979236218784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdc360 con 0x7f39f00cd050 2024-03-28T16:49:12.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.195+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdcd50 con 0x55793efcba70 2024-03-28T16:49:12.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.199+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93979236214992] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.199+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93979236218784] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.199+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.199+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93979236223024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdbe70 con 0x55793efcba70 2024-03-28T16:49:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.199+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdd5a0 con 0x55793efcffb0 2024-03-28T16:49:12.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.199+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93979236223024] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.203+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.203+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93979236225280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdbe70 con 0x55793efcffb0 2024-03-28T16:49:12.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.203+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efddda0 con 0x55793efcffb0 2024-03-28T16:49:12.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.203+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93979236225280] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.203+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.203+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93979236212256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efde5a0 con 0x55793efcffb0 2024-03-28T16:49:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.203+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdef10 con 0x55793efcffb0 2024-03-28T16:49:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93979236232064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdf660 con 0x55793efcffb0 2024-03-28T16:49:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=0 2024-03-28T16:49:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=1 2024-03-28T16:49:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=2 2024-03-28T16:49:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=3 2024-03-28T16:49:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=4 2024-03-28T16:49:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=5 2024-03-28T16:49:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93979236212256] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=6 2024-03-28T16:49:12.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93979236232064] v29'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 add_watcher() i=7 2024-03-28T16:49:12.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:12.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efde100 con 0x55793efcba70 2024-03-28T16:49:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93979236233472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efde100 con 0x55793efcba70 2024-03-28T16:49:12.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93979236233472] v29'26 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f38b3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:12.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:12.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.207+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeac70 con 0x55793efcba70 2024-03-28T16:49:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.211+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.211+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeb0b0 con 0x55793efcffb0 2024-03-28T16:49:12.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.211+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'10 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.211+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeb0b0 con 0x55793efd4c00 2024-03-28T16:49:12.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.215+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.215+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeba80 con 0x55793efcba70 2024-03-28T16:49:12.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.215+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'27 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.215+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efebfb0 con 0x7f39f00cd050 2024-03-28T16:49:12.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.215+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.215+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeba70 con 0x55793efd4c00 2024-03-28T16:49:12.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.219+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.219+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efecac0 con 0x55793efcba70 2024-03-28T16:49:12.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.219+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.219+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efecb50 con 0x55793efcffb0 2024-03-28T16:49:12.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.223+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'13 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.223+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efed080 con 0x7f39f00cd050 2024-03-28T16:49:12.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.223+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.223+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efed080 con 0x55793efd4c00 2024-03-28T16:49:12.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.223+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.223+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efed5b0 con 0x7f39f00cd050 2024-03-28T16:49:12.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.227+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'21 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.227+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efed5b0 con 0x55793efcffb0 2024-03-28T16:49:12.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.227+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.227+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeda50 con 0x7f39f00cd050 2024-03-28T16:49:12.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.231+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'10 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.231+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeda50 con 0x55793efcffb0 2024-03-28T16:49:12.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.231+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'19 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.231+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeda50 con 0x55793efcba70 2024-03-28T16:49:12.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.235+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'28 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.235+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efeda50 con 0x55793efd4c00 2024-03-28T16:49:12.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.235+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'14 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.235+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8df0 con 0x7f39f00cd050 2024-03-28T16:49:12.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.235+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'22 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.235+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8df0 con 0x55793efd4c00 2024-03-28T16:49:12.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.239+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'10 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.239+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8d90 con 0x55793efcba70 2024-03-28T16:49:12.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.239+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.239+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efcba70 2024-03-28T16:49:12.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.243+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'29 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.243+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efd4c00 2024-03-28T16:49:12.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.243+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'15 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.243+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x7f39f00cd050 2024-03-28T16:49:12.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.243+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.247+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efd4c00 2024-03-28T16:49:12.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.247+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.247+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efd4c00 2024-03-28T16:49:12.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.247+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.247+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efd4c00 2024-03-28T16:49:12.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.247+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'6 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.251+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efd4c00 2024-03-28T16:49:12.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.251+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'14 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.251+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efcffb0 2024-03-28T16:49:12.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.251+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.251+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efd4c00 2024-03-28T16:49:12.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.255+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.255+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efcffb0 2024-03-28T16:49:12.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.255+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'14 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.255+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efd4c00 2024-03-28T16:49:12.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.259+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'16 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.259+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x55793efcffb0 2024-03-28T16:49:12.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.259+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.259+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8f60 con 0x7f39f00cd050 2024-03-28T16:49:12.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f38b2fe5640 20 reqs_thread_entry: start 2024-03-28T16:49:12.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f39ffe61980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:12.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f39ffe61980 20 Filter name: none 2024-03-28T16:49:12.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f38b0fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:12.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f39ffe61980 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-03-28T16:49:12.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f39ffe61980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.263+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8010 con 0x7f39f00cd050 2024-03-28T16:49:12.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(94 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f39ffe61980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f39ffe61980 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-03-28T16:49:12.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f39ffe61980 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-03-28T16:49:12.266 INFO:teuthology.orchestra.run.smithi170.stderr:could not remove user: unable to remove user, user does not exist 2024-03-28T16:49:12.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f38b3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93979236233472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8010 con 0x55793efcba70 2024-03-28T16:49:12.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 93979236233472] v29'30 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.267+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93979236207008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efe8010 con 0x7f39f00cd050 2024-03-28T16:49:12.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.271+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 93979236207008] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.271+0000 7f39ffe61980 20 remove_watcher() i=0 2024-03-28T16:49:12.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.271+0000 7f39ffe61980 2 removed watcher, disabling cache 2024-03-28T16:49:12.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.271+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93979236211200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd89a0 con 0x55793efcba70 2024-03-28T16:49:12.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.271+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 93979236211200] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.271+0000 7f39ffe61980 20 remove_watcher() i=1 2024-03-28T16:49:12.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.271+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93979236214992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd89a0 con 0x55793efd4c00 2024-03-28T16:49:12.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.275+0000 7f39fd17b640 1 -- 172.21.15.170:0/1598245117 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 93979236214992] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f00aa9f0 con 0x55793efd4c00 2024-03-28T16:49:12.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.275+0000 7f39ffe61980 20 remove_watcher() i=2 2024-03-28T16:49:12.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.275+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93979236218784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efda8d0 con 0x7f39f00cd050 2024-03-28T16:49:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.275+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 93979236218784] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x7f39f00cd050 2024-03-28T16:49:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.275+0000 7f39ffe61980 20 remove_watcher() i=3 2024-03-28T16:49:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.275+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93979236223024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd89a0 con 0x55793efcba70 2024-03-28T16:49:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.279+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 93979236223024] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39f80083f0 con 0x55793efcba70 2024-03-28T16:49:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.279+0000 7f39ffe61980 20 remove_watcher() i=4 2024-03-28T16:49:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.279+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93979236225280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efdb7a0 con 0x55793efcffb0 2024-03-28T16:49:12.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.279+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 93979236225280] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.279+0000 7f39ffe61980 20 remove_watcher() i=5 2024-03-28T16:49:12.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.279+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93979236212256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd89a0 con 0x55793efcffb0 2024-03-28T16:49:12.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 93979236212256] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 20 remove_watcher() i=6 2024-03-28T16:49:12.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93979236232064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55793efd89a0 con 0x55793efcffb0 2024-03-28T16:49:12.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39fd97c640 1 -- 172.21.15.170:0/1598245117 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 93979236232064] v29'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39e8079040 con 0x55793efcffb0 2024-03-28T16:49:12.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 20 remove_watcher() i=7 2024-03-28T16:49:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793efcffb0 msgr2=0x55793efd23a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793efcffb0 0x55793efd23a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793efcba70 msgr2=0x55793efcdf30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793efcba70 0x55793efcdf30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39f00cd050 msgr2=0x7f39f00cd430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39f00cd050 0x7f39f00cd430 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55793efd4c00 msgr2=0x55793efd70c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55793efd4c00 0x55793efd70c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38c8034300 msgr2=0x7f38c80367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38c8034300 0x7f38c80367c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55793ef9eb90 tx=0x7f39f00a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39fe17d640 1 -- 172.21.15.170:0/1598245117 reap_dead start 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 msgr2=0x55793efc0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efc0630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39e8004ea0 tx=0x7f39e801b4e0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 shutdown_connections 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/1598245117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efb9630 0x55793efc0630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 >> 172.21.15.170:0/1598245117 conn(0x55793efb59c0 msgr2=0x55793efb3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 shutdown_connections 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/1598245117 wait complete. 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38d80342b0 msgr2=0x7f38d8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38d80342b0 0x7f38d8036770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55793ef93540 tx=0x7f39f0005e50 comp rx=0 tx=0).stop 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efac700 msgr2=0x55793efacb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efac700 0x55793efacb20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f39e8017820 tx=0x7f39e8017d70 comp rx=0 tx=0).stop 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 shutdown_connections 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38d80342b0 0x7f38d8036770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 --2- 172.21.15.170:0/3676998324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793efac700 0x55793efacb20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 >> 172.21.15.170:0/3676998324 conn(0x55793efa1510 msgr2=0x55793efa2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 shutdown_connections 2024-03-28T16:49:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.283+0000 7f39ffe61980 1 -- 172.21.15.170:0/3676998324 wait complete. 2024-03-28T16:49:12.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793ef98070 msgr2=0x55793ef9a460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55793ef98070 0x55793ef9a460 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793eea2630 msgr2=0x55793ef95ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55793eea2630 0x55793ef95ab0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55793ef9bd40 msgr2=0x55793ef9e130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55793ef9bd40 0x55793ef9e130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38e40342b0 msgr2=0x7f38e4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38e40342b0 0x7f38e4036770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39f00ab000 tx=0x7f39f00a5000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 msgr2=0x55793ef8ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 --2- 172.21.15.170:0/4057499017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55793ee9e130 0x55793ef8ee30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39e800b650 tx=0x7f39e8000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39fe17d640 1 -- 172.21.15.170:0/4057499017 reap_dead start 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 shutdown_connections 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 >> 172.21.15.170:0/4057499017 conn(0x55793ee11850 msgr2=0x55793ef82680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 shutdown_connections 2024-03-28T16:49:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.287+0000 7f39ffe61980 1 -- 172.21.15.170:0/4057499017 wait complete. 2024-03-28T16:49:12.301 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-28T16:49:12.301 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-03-28T16:49:12.302 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-03-28T16:49:12.302 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-03-28T16:49:12.382 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:12.382 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:12.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc517c980 1 Processor -- start 2024-03-28T16:49:12.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc517c980 1 -- start start 2024-03-28T16:49:12.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc517c980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ec389b1130 0x55ec389b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc517c980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ec388fb520 con 0x55ec389b1a50 2024-03-28T16:49:12.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc517c980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ec388c20f0 con 0x55ec389b1130 2024-03-28T16:49:12.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc2c97640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ec389b1130 0x55ec389b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc2496640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55ec389b1a50 0x55ec389b2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60776/0 (socket says 172.21.15.170:60776) 2024-03-28T16:49:12.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc2c97640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ec389b1130 0x55ec389b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54846/0 (socket says 172.21.15.170:54846) 2024-03-28T16:49:12.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc2496640 1 -- 172.21.15.170:0/1972374100 learned_addr learned my addr 172.21.15.170:0/1972374100 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc2c97640 1 -- 172.21.15.170:0/1972374100 >> v1:172.21.15.119:6789/0 conn(0x55ec389b1a50 legacy=0x55ec389b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.383+0000 7effc2c97640 1 -- 172.21.15.170:0/1972374100 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec389b2c30 con 0x55ec389b1130 2024-03-28T16:49:12.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc2c97640 1 --2- 172.21.15.170:0/1972374100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec389b1510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effb400a480 tx=0x7effb401b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82acf17ac591ce7e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc1c95640 1 -- 172.21.15.170:0/1972374100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40ab540 con 0x55ec389b1130 2024-03-28T16:49:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc1c95640 1 -- 172.21.15.170:0/1972374100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40ab6a0 con 0x55ec389b1130 2024-03-28T16:49:12.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc1c95640 1 -- 172.21.15.170:0/1972374100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40ab960 con 0x55ec389b1130 2024-03-28T16:49:12.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc517c980 1 -- 172.21.15.170:0/1972374100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 msgr2=0x55ec389b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc517c980 1 --2- 172.21.15.170:0/1972374100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec389b1510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effb400a480 tx=0x7effb401b280 comp rx=0 tx=0).stop 2024-03-28T16:49:12.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc517c980 1 -- 172.21.15.170:0/1972374100 shutdown_connections 2024-03-28T16:49:12.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc517c980 1 --2- 172.21.15.170:0/1972374100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec389b1510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc517c980 1 -- 172.21.15.170:0/1972374100 >> 172.21.15.170:0/1972374100 conn(0x55ec38924850 msgr2=0x55ec38924c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc517c980 1 -- 172.21.15.170:0/1972374100 shutdown_connections 2024-03-28T16:49:12.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.387+0000 7effc517c980 1 -- 172.21.15.170:0/1972374100 wait complete. 2024-03-28T16:49:12.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.403+0000 7effc517c980 1 Processor -- start 2024-03-28T16:49:12.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.403+0000 7effc517c980 1 -- start start 2024-03-28T16:49:12.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.403+0000 7effc517c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38a9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.403+0000 7effc517c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec388bf620 con 0x55ec389b1130 2024-03-28T16:49:12.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38a9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38a9b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54850/0 (socket says 172.21.15.170:54850) 2024-03-28T16:49:12.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 -- 172.21.15.170:0/3739121597 learned_addr learned my addr 172.21.15.170:0/3739121597 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 -- 172.21.15.170:0/3739121597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4005d30 con 0x55ec389b1130 2024-03-28T16:49:12.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 --2- 172.21.15.170:0/3739121597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38a9b5c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effb40ae040 tx=0x7effb4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea52cab299abb197 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7efeb0e09640 1 -- 172.21.15.170:0/3739121597 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40ab810 con 0x55ec389b1130 2024-03-28T16:49:12.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7efeb0e09640 1 -- 172.21.15.170:0/3739121597 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40ab970 con 0x55ec389b1130 2024-03-28T16:49:12.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7efeb0e09640 1 -- 172.21.15.170:0/3739121597 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40abc10 con 0x55ec389b1130 2024-03-28T16:49:12.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 -- 172.21.15.170:0/3739121597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 msgr2=0x55ec38a9b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 --2- 172.21.15.170:0/3739121597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38a9b5c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effb40ae040 tx=0x7effb4003030 comp rx=0 tx=0).stop 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 -- 172.21.15.170:0/3739121597 shutdown_connections 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 --2- 172.21.15.170:0/3739121597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38a9b5c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 -- 172.21.15.170:0/3739121597 >> 172.21.15.170:0/3739121597 conn(0x55ec38924850 msgr2=0x55ec38a97150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 -- 172.21.15.170:0/3739121597 shutdown_connections 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 -- 172.21.15.170:0/3739121597 wait complete. 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 Processor -- start 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 -- start start 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38aa22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc517c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec388bae00 con 0x55ec389b1130 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38aa22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38aa22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54856/0 (socket says 172.21.15.170:54856) 2024-03-28T16:49:12.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 -- 172.21.15.170:0/2158588078 learned_addr learned my addr 172.21.15.170:0/2158588078 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.407+0000 7effc2c97640 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec389b2c30 con 0x55ec389b1130 2024-03-28T16:49:12.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc2c97640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38aa22f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7effb401bb20 tx=0x7effb4018020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7efeaf606640 1 -- 172.21.15.170:0/2158588078 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40c2070 con 0x55ec389b1130 2024-03-28T16:49:12.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec38aa2830 con 0x55ec389b1130 2024-03-28T16:49:12.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7efeaf606640 1 -- 172.21.15.170:0/2158588078 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40b59d0 con 0x55ec389b1130 2024-03-28T16:49:12.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7efeaf606640 1 -- 172.21.15.170:0/2158588078 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40b5c70 con 0x55ec389b1130 2024-03-28T16:49:12.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec38aa2cc0 con 0x55ec389b1130 2024-03-28T16:49:12.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7efeaf606640 1 -- 172.21.15.170:0/2158588078 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7effb40b2070 con 0x55ec389b1130 2024-03-28T16:49:12.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7efeaf606640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efea80342b0 0x7efea8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7efeaf606640 1 -- 172.21.15.170:0/2158588078 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7effb40e32c0 con 0x55ec389b1130 2024-03-28T16:49:12.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc2496640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efea80342b0 0x7efea8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec389b1a50 0x55ec38aa8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aa9410 con 0x55ec389b1a50 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc3498640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec389b1a50 0x55ec38aa8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc2496640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efea80342b0 0x7efea8036770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7effb8001da0 tx=0x7effb8001300 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc3498640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec389b1a50 0x55ec38aa8ed0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc3498640 1 -- 172.21.15.170:0/2158588078 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec389b1a50 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aa9770 con 0x55ec389b1a50 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc3498640 1 -- 172.21.15.170:0/2158588078 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec389b1a50 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38aab490 0x55ec38aad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.411+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aaddc0 con 0x55ec38aab490 2024-03-28T16:49:12.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2c97640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38aab490 0x55ec38aad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2c97640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38aab490 0x55ec38aad880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2c97640 1 -- 172.21.15.170:0/2158588078 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7effb40bd5d0 con 0x55ec38aab490 2024-03-28T16:49:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aaa2d0 con 0x55ec389b1a50 2024-03-28T16:49:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc3498640 1 -- 172.21.15.170:0/2158588078 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec389b1a50 2024-03-28T16:49:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ec38aaf130 0x55ec38ab1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ab1bc0 con 0x55ec38aaf130 2024-03-28T16:49:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2496640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ec38aaf130 0x55ec38ab1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2496640 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ec38aaf130 0x55ec38ab1520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2496640 1 -- 172.21.15.170:0/2158588078 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38aaf130 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc517c980 1 Processor -- start 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc517c980 1 -- start start 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc517c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38abfaf0 0x55ec38abff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc517c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec38853e30 con 0x55ec38abfaf0 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38abfaf0 0x55ec38abff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38abfaf0 0x55ec38abff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54860/0 (socket says 172.21.15.170:54860) 2024-03-28T16:49:12.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.415+0000 7effc2c97640 1 -- 172.21.15.170:0/1256434719 learned_addr learned my addr 172.21.15.170:0/1256434719 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2c97640 1 -- 172.21.15.170:0/1256434719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec38ac0450 con 0x55ec38abfaf0 2024-03-28T16:49:12.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2c97640 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38abfaf0 0x55ec38abff10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effb40e3270 tx=0x7effb40c67e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7efeace01640 1 -- 172.21.15.170:0/1256434719 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40bfdf0 con 0x55ec38abfaf0 2024-03-28T16:49:12.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec38ac06a0 con 0x55ec38abfaf0 2024-03-28T16:49:12.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7efeace01640 1 -- 172.21.15.170:0/1256434719 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40c6b60 con 0x55ec38abfaf0 2024-03-28T16:49:12.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7efeace01640 1 -- 172.21.15.170:0/1256434719 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40c6e20 con 0x55ec38abfaf0 2024-03-28T16:49:12.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec38ac11f0 con 0x55ec38abfaf0 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7efeace01640 1 -- 172.21.15.170:0/1256434719 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7effb40dc070 con 0x55ec38abfaf0 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7efeace01640 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe9c0342b0 0x7efe9c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7efeace01640 1 -- 172.21.15.170:0/1256434719 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7effb40be330 con 0x55ec38abfaf0 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2496640 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe9c0342b0 0x7efe9c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc517c980 1 Processor -- start 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc517c980 1 -- start start 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2496640 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe9c0342b0 0x7efe9c036770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55ec38aa6900 tx=0x7effb800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc517c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38acce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc517c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec38847790 con 0x55ec38acc9e0 2024-03-28T16:49:12.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38acce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38acce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54868/0 (socket says 172.21.15.170:54868) 2024-03-28T16:49:12.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2c97640 1 -- 172.21.15.170:0/1467728970 learned_addr learned my addr 172.21.15.170:0/1467728970 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2c97640 1 -- 172.21.15.170:0/1467728970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec38acd3d0 con 0x55ec38acc9e0 2024-03-28T16:49:12.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.419+0000 7effc2c97640 1 --2- 172.21.15.170:0/1467728970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38acce00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7effb40eaf70 tx=0x7effb40d08f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd68e45ded34422 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7efea67fc640 1 -- 172.21.15.170:0/1467728970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40c6fd0 con 0x55ec38acc9e0 2024-03-28T16:49:12.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7efea67fc640 1 -- 172.21.15.170:0/1467728970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40d8170 con 0x55ec38acc9e0 2024-03-28T16:49:12.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7efea67fc640 1 -- 172.21.15.170:0/1467728970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40c5370 con 0x55ec38acc9e0 2024-03-28T16:49:12.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- 172.21.15.170:0/1467728970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 msgr2=0x55ec38acce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 --2- 172.21.15.170:0/1467728970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38acce00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7effb40eaf70 tx=0x7effb40d08f0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- 172.21.15.170:0/1467728970 shutdown_connections 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 --2- 172.21.15.170:0/1467728970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38acce00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- 172.21.15.170:0/1467728970 >> 172.21.15.170:0/1467728970 conn(0x55ec38ac8d70 msgr2=0x55ec38ac6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- 172.21.15.170:0/1467728970 shutdown_connections 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- 172.21.15.170:0/1467728970 wait complete. 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 Processor -- start 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- start start 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38ad3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec38ad3e70 con 0x55ec38acc9e0 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38ad3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc2c97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38ad3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54878/0 (socket says 172.21.15.170:54878) 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 learned_addr learned my addr 172.21.15.170:0/1537128083 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec38ac11f0 con 0x55ec38acc9e0 2024-03-28T16:49:12.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc2c97640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38ad3930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effb40d0930 tx=0x7effb40eaf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7efea4ff9640 1 -- 172.21.15.170:0/1537128083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40b48e0 con 0x55ec38acc9e0 2024-03-28T16:49:12.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec38ad4070 con 0x55ec38acc9e0 2024-03-28T16:49:12.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec38ad4070 con 0x55ec38acc9e0 2024-03-28T16:49:12.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7efea4ff9640 1 -- 172.21.15.170:0/1537128083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40aa890 con 0x55ec38acc9e0 2024-03-28T16:49:12.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.423+0000 7efea4ff9640 1 -- 172.21.15.170:0/1537128083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effb40aab30 con 0x55ec38acc9e0 2024-03-28T16:49:12.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7efea4ff9640 1 -- 172.21.15.170:0/1537128083 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7effb40aac90 con 0x55ec38acc9e0 2024-03-28T16:49:12.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7efea4ff9640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe8c034300 0x7efe8c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7efea4ff9640 1 -- 172.21.15.170:0/1537128083 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7effb40f1040 con 0x55ec38acc9e0 2024-03-28T16:49:12.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc2496640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe8c034300 0x7efe8c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec38adeb10 0x55ec38ae0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae1700 con 0x55ec38adeb10 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc3498640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec38adeb10 0x55ec38ae0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc2496640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe8c034300 0x7efe8c0367c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55ec38aaaf50 tx=0x7effb8010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc3498640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec38adeb10 0x55ec38ae0fb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 realm 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae1b20 con 0x55ec38adeb10 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:12.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae26d0 con 0x55ec38adeb10 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38ae3080 0x55ec38ae5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae5b10 con 0x55ec38ae3080 2024-03-28T16:49:12.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc2c97640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38ae3080 0x55ec38ae5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.427+0000 7effc2c97640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38ae3080 0x55ec38ae5470 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc517c980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:12.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc517c980 20 searching for the correct realm 2024-03-28T16:49:12.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ec38ae7c90 0x55ec38aea150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aea7d0 con 0x55ec38ae7c90 2024-03-28T16:49:12.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2496640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ec38ae7c90 0x55ec38aea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2496640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ec38ae7c90 0x55ec38aea150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2496640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7effb8037040 0x7effb8036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aea7d0 con 0x7effb8037040 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7effb8037040 0x7effb8036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7effb8037040 0x7effb8036410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.431+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x7effb8037040 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x7effb8037040 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.435+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x7effb8037040 2024-03-28T16:49:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.439+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x7effb8037040 2024-03-28T16:49:12.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae7c90 2024-03-28T16:49:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38ae3080 2024-03-28T16:49:12.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x55ec38adeb10 2024-03-28T16:49:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae2370 con 0x7effb8037040 2024-03-28T16:49:12.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae60b0 con 0x55ec38adeb10 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.443+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae60b0 con 0x55ec38adeb10 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae60b0 con 0x7effb8037040 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 zone default found 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 4 Realm: () 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 zonegroup default 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38ae6900 con 0x55ec38ae3080 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aec5f0 con 0x55ec38ae7c90 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aec950 con 0x55ec38ae7c90 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 started sync module instance, tier type = 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:12.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aebed0 con 0x7effb8037040 2024-03-28T16:49:12.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94473051617840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aec280 con 0x7effb8037040 2024-03-28T16:49:12.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.447+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aebe90 con 0x55ec38adeb10 2024-03-28T16:49:12.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94473051617840] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94473051622576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aed640 con 0x55ec38adeb10 2024-03-28T16:49:12.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aedfb0 con 0x55ec38ae7c90 2024-03-28T16:49:12.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94473051622576] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94473051626256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aee310 con 0x55ec38ae7c90 2024-03-28T16:49:12.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.451+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aeec80 con 0x7effb8037040 2024-03-28T16:49:12.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94473051629808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aef370 con 0x7effb8037040 2024-03-28T16:49:12.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aefe40 con 0x55ec38adeb10 2024-03-28T16:49:12.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94473051626256] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94473051629808] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94473051633952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aefe00 con 0x55ec38adeb10 2024-03-28T16:49:12.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.455+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38aeef40 con 0x55ec38ae3080 2024-03-28T16:49:12.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94473051633952] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94473051623632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af0690 con 0x55ec38ae3080 2024-03-28T16:49:12.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af0e90 con 0x55ec38ae3080 2024-03-28T16:49:12.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94473051623632] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94473051636208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af1690 con 0x55ec38ae3080 2024-03-28T16:49:12.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.459+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af2000 con 0x55ec38ae3080 2024-03-28T16:49:12.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94473051636208] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94473051640304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af2800 con 0x55ec38ae3080 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=0 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=1 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=2 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=3 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=4 2024-03-28T16:49:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=5 2024-03-28T16:49:12.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=6 2024-03-28T16:49:12.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94473051640304] v29'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 20 add_watcher() i=7 2024-03-28T16:49:12.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:12.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af5a70 con 0x55ec38adeb10 2024-03-28T16:49:12.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.463+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94473051645136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af2d90 con 0x55ec38adeb10 2024-03-28T16:49:12.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.467+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94473051645136] v29'31 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.467+0000 7efe797e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.467+0000 7effc517c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:12.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.467+0000 7effc517c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:12.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.467+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afe060 con 0x55ec38adeb10 2024-03-28T16:49:12.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.467+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.467+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afe520 con 0x55ec38ae3080 2024-03-28T16:49:12.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.471+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'11 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.471+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afe520 con 0x55ec38ae7c90 2024-03-28T16:49:12.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.471+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.471+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af4e60 con 0x55ec38adeb10 2024-03-28T16:49:12.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.471+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'32 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.471+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af49c0 con 0x7effb8037040 2024-03-28T16:49:12.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.475+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.475+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38af49c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.475+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.475+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b000d0 con 0x55ec38adeb10 2024-03-28T16:49:12.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.475+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.475+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b000d0 con 0x55ec38ae3080 2024-03-28T16:49:12.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.479+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'15 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.479+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b00600 con 0x7effb8037040 2024-03-28T16:49:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.479+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.479+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b00600 con 0x55ec38ae7c90 2024-03-28T16:49:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.483+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.483+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b00b30 con 0x7effb8037040 2024-03-28T16:49:12.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.483+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'24 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.483+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b00b30 con 0x55ec38ae3080 2024-03-28T16:49:12.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.483+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.483+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01060 con 0x7effb8037040 2024-03-28T16:49:12.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.487+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'11 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.487+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01060 con 0x55ec38ae3080 2024-03-28T16:49:12.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.487+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'21 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.487+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38adeb10 2024-03-28T16:49:12.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.487+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'33 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.487+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b00b30 con 0x55ec38ae7c90 2024-03-28T16:49:12.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.491+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.491+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x7effb8037040 2024-03-28T16:49:12.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.491+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'25 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.491+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.491+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'11 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.491+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38adeb10 2024-03-28T16:49:12.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.495+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.495+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38adeb10 2024-03-28T16:49:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.495+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'34 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.495+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'18 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x7effb8037040 2024-03-28T16:49:12.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.499+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.503+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'7 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.503+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.503+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'16 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.503+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae3080 2024-03-28T16:49:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.503+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.503+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.507+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.507+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae3080 2024-03-28T16:49:12.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.507+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'16 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.507+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae7c90 2024-03-28T16:49:12.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.511+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'19 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.511+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x55ec38ae3080 2024-03-28T16:49:12.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.511+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'22 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.511+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38b01500 con 0x7effb8037040 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7efdf3fff640 20 reqs_thread_entry: start 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 20 Filter name: none 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7efdf1ffb640 20 reqs_thread_entry: start 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:94 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc370 con 0x55ec38ae3080 2024-03-28T16:49:12.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(94 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-03-28T16:49:12.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:49:12.516 INFO:teuthology.orchestra.run.smithi170.stderr:could not remove user: unable to remove user, user does not exist 2024-03-28T16:49:12.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7efe797e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.515+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94473051645136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc370 con 0x55ec38adeb10 2024-03-28T16:49:12.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94473051645136] v29'35 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94473051617840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc370 con 0x7effb8037040 2024-03-28T16:49:12.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94473051617840] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc517c980 20 remove_watcher() i=0 2024-03-28T16:49:12.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc517c980 2 removed watcher, disabling cache 2024-03-28T16:49:12.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94473051622576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc340 con 0x55ec38adeb10 2024-03-28T16:49:12.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94473051622576] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc517c980 20 remove_watcher() i=1 2024-03-28T16:49:12.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.519+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94473051626256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc340 con 0x55ec38ae7c90 2024-03-28T16:49:12.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc2496640 1 -- 172.21.15.170:0/1537128083 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94473051626256] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb800e6c0 con 0x55ec38ae7c90 2024-03-28T16:49:12.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc517c980 20 remove_watcher() i=2 2024-03-28T16:49:12.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94473051629808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc340 con 0x7effb8037040 2024-03-28T16:49:12.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94473051629808] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x7effb8037040 2024-03-28T16:49:12.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc517c980 20 remove_watcher() i=3 2024-03-28T16:49:12.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94473051633952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc340 con 0x55ec38adeb10 2024-03-28T16:49:12.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94473051633952] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effbc00a4d0 con 0x55ec38adeb10 2024-03-28T16:49:12.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc517c980 20 remove_watcher() i=4 2024-03-28T16:49:12.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.523+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94473051623632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc340 con 0x55ec38ae3080 2024-03-28T16:49:12.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.527+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94473051623632] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.527+0000 7effc517c980 20 remove_watcher() i=5 2024-03-28T16:49:12.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.527+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94473051636208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc340 con 0x55ec38ae3080 2024-03-28T16:49:12.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.527+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94473051636208] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.527+0000 7effc517c980 20 remove_watcher() i=6 2024-03-28T16:49:12.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.527+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94473051640304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ec38afc340 con 0x55ec38ae3080 2024-03-28T16:49:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc2c97640 1 -- 172.21.15.170:0/1537128083 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94473051640304] v29'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7effb40e3700 con 0x55ec38ae3080 2024-03-28T16:49:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 20 remove_watcher() i=7 2024-03-28T16:49:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38ae3080 msgr2=0x55ec38ae5470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38ae3080 0x55ec38ae5470 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec38adeb10 msgr2=0x55ec38ae0fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec38adeb10 0x55ec38ae0fb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7effb8037040 msgr2=0x7effb8036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7effb8037040 0x7effb8036410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ec38ae7c90 msgr2=0x55ec38aea150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ec38ae7c90 0x55ec38aea150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe8c034300 msgr2=0x7efe8c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe8c034300 0x7efe8c0367c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55ec38aaaf50 tx=0x7effb8010000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc3498640 1 -- 172.21.15.170:0/1537128083 reap_dead start 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 msgr2=0x55ec38ad3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38ad3930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effb40d0930 tx=0x7effb40eaf00 comp rx=0 tx=0).stop 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 shutdown_connections 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1537128083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38acc9e0 0x55ec38ad3930 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 >> 172.21.15.170:0/1537128083 conn(0x55ec38ac8d70 msgr2=0x55ec38ac63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 shutdown_connections 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1537128083 wait complete. 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe9c0342b0 msgr2=0x7efe9c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe9c0342b0 0x7efe9c036770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55ec38aa6900 tx=0x7effb800f000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38abfaf0 msgr2=0x55ec38abff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38abfaf0 0x55ec38abff10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effb40e3270 tx=0x7effb40c67e0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 shutdown_connections 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efe9c0342b0 0x7efe9c036770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/1256434719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec38abfaf0 0x55ec38abff10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 >> 172.21.15.170:0/1256434719 conn(0x55ec38ab4900 msgr2=0x55ec38ab5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 shutdown_connections 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/1256434719 wait complete. 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38aab490 msgr2=0x55ec38aad880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ec38aab490 0x55ec38aad880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec389b1a50 msgr2=0x55ec38aa8ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ec389b1a50 0x55ec38aa8ed0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ec38aaf130 msgr2=0x55ec38ab1520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ec38aaf130 0x55ec38ab1520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efea80342b0 msgr2=0x7efea8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efea80342b0 0x7efea8036770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7effb8001da0 tx=0x7effb8001300 comp rx=0 tx=0).stop 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 msgr2=0x55ec38aa22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 --2- 172.21.15.170:0/2158588078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec389b1130 0x55ec38aa22f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7effb401bb20 tx=0x7effb4018020 comp rx=0 tx=0).stop 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc3498640 1 -- 172.21.15.170:0/2158588078 reap_dead start 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 shutdown_connections 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 >> 172.21.15.170:0/2158588078 conn(0x55ec38924850 msgr2=0x55ec38a95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 shutdown_connections 2024-03-28T16:49:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.531+0000 7effc517c980 1 -- 172.21.15.170:0/2158588078 wait complete. 2024-03-28T16:49:12.541 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-28T16:49:12.541 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-03-28T16:49:12.541 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-03-28T16:49:12.542 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-03-28T16:49:12.618 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:12.618 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:12.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9eaaf08980 1 Processor -- start 2024-03-28T16:49:12.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9eaaf08980 1 -- start start 2024-03-28T16:49:12.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9eaaf08980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x561f7dd29750 0x561f7dd2bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9eaaf08980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561f7dc6f520 con 0x561f7dc978b0 2024-03-28T16:49:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9eaaf08980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x561f7dc360f0 con 0x561f7dd29750 2024-03-28T16:49:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9ea8222640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x561f7dd29750 0x561f7dd2bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9ea8a23640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x561f7dc978b0 0x561f7dd28030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60792/0 (socket says 172.21.15.170:60792) 2024-03-28T16:49:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9ea8a23640 1 -- 172.21.15.170:0/159559413 learned_addr learned my addr 172.21.15.170:0/159559413 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9ea8222640 1 -- 172.21.15.170:0/159559413 >> v1:172.21.15.119:6789/0 conn(0x561f7dc978b0 legacy=0x561f7dd28030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.619+0000 7f9ea8222640 1 -- 172.21.15.170:0/159559413 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7dd23a50 con 0x561f7dd29750 2024-03-28T16:49:12.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9ea8222640 1 --2- 172.21.15.170:0/159559413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dd29750 0x561f7dd2bb40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00a480 tx=0x7f9e9c01b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec501b994d4bcec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9ea7a21640 1 -- 172.21.15.170:0/159559413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e9c0ab540 con 0x561f7dd29750 2024-03-28T16:49:12.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9ea7a21640 1 -- 172.21.15.170:0/159559413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0ab6a0 con 0x561f7dd29750 2024-03-28T16:49:12.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9ea7a21640 1 -- 172.21.15.170:0/159559413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e9c0ab960 con 0x561f7dd29750 2024-03-28T16:49:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9eaaf08980 1 -- 172.21.15.170:0/159559413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dd29750 msgr2=0x561f7dd2bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/159559413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dd29750 0x561f7dd2bb40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00a480 tx=0x7f9e9c01b280 comp rx=0 tx=0).stop 2024-03-28T16:49:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9eaaf08980 1 -- 172.21.15.170:0/159559413 shutdown_connections 2024-03-28T16:49:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/159559413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dd29750 0x561f7dd2bb40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9eaaf08980 1 -- 172.21.15.170:0/159559413 >> 172.21.15.170:0/159559413 conn(0x561f7dd1d710 msgr2=0x561f7dd1fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9eaaf08980 1 -- 172.21.15.170:0/159559413 shutdown_connections 2024-03-28T16:49:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.623+0000 7f9eaaf08980 1 -- 172.21.15.170:0/159559413 wait complete. 2024-03-28T16:49:12.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 Processor -- start 2024-03-28T16:49:12.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- start start 2024-03-28T16:49:12.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de0f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7dc33620 con 0x561f7dc978b0 2024-03-28T16:49:12.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de0f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de0f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54900/0 (socket says 172.21.15.170:54900) 2024-03-28T16:49:12.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 -- 172.21.15.170:0/4012932844 learned_addr learned my addr 172.21.15.170:0/4012932844 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 -- 172.21.15.170:0/4012932844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c005d30 con 0x561f7dc978b0 2024-03-28T16:49:12.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/4012932844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de0f0b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001590 tx=0x7f9ea0001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb9b8de2d4cce1a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9d96e0d640 1 -- 172.21.15.170:0/4012932844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0004530 con 0x561f7dc978b0 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9d96e0d640 1 -- 172.21.15.170:0/4012932844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0004690 con 0x561f7dc978b0 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9d96e0d640 1 -- 172.21.15.170:0/4012932844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0004950 con 0x561f7dc978b0 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- 172.21.15.170:0/4012932844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 msgr2=0x561f7de0f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/4012932844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de0f0b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001590 tx=0x7f9ea0001a50 comp rx=0 tx=0).stop 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- 172.21.15.170:0/4012932844 shutdown_connections 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/4012932844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de0f0b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- 172.21.15.170:0/4012932844 >> 172.21.15.170:0/4012932844 conn(0x561f7dd1d710 msgr2=0x561f7de0ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- 172.21.15.170:0/4012932844 shutdown_connections 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- 172.21.15.170:0/4012932844 wait complete. 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 Processor -- start 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- start start 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de15f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9eaaf08980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7dc2ee00 con 0x561f7dc978b0 2024-03-28T16:49:12.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de15f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de15f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54902/0 (socket says 172.21.15.170:54902) 2024-03-28T16:49:12.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 -- 172.21.15.170:0/581613834 learned_addr learned my addr 172.21.15.170:0/581613834 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.639+0000 7f9ea8a23640 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7dc6f520 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de15f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001a90 tx=0x7f9ea0001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9d9560a640 1 -- 172.21.15.170:0/581613834 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0013760 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561f7de16440 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9d9560a640 1 -- 172.21.15.170:0/581613834 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea00138c0 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9d9560a640 1 -- 172.21.15.170:0/581613834 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0013ba0 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f7de16870 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9d9560a640 1 -- 172.21.15.170:0/581613834 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9ea0013d00 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9d9560a640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d900342b0 0x7f9d90036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9d9560a640 1 -- 172.21.15.170:0/581613834 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f9ea0049430 con 0x561f7dc978b0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea8222640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d900342b0 0x7f9d90036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7dd29750 0x561f7de1cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de1d0c0 con 0x561f7dd29750 2024-03-28T16:49:12.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea9224640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7dd29750 0x561f7de1cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea8222640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d900342b0 0x7f9d90036770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0ad000 tx=0x7f9e9c0a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea9224640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7dd29750 0x561f7de1cb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea9224640 1 -- 172.21.15.170:0/581613834 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7dd29750 2024-03-28T16:49:12.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de1d420 con 0x561f7dd29750 2024-03-28T16:49:12.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea9224640 1 -- 172.21.15.170:0/581613834 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7dd29750 2024-03-28T16:49:12.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de1edb0 0x561f7de211d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de21960 con 0x561f7de1edb0 2024-03-28T16:49:12.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.643+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de1edb0 0x561f7de211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de1edb0 0x561f7de211d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8a23640 1 -- 172.21.15.170:0/581613834 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de1edb0 2024-03-28T16:49:12.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de1dc20 con 0x561f7dd29750 2024-03-28T16:49:12.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea9224640 1 -- 172.21.15.170:0/581613834 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7dd29750 2024-03-28T16:49:12.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561f7de22d80 0x561f7de25170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de25830 con 0x561f7de22d80 2024-03-28T16:49:12.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8222640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561f7de22d80 0x561f7de25170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8222640 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561f7de22d80 0x561f7de25170 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8222640 1 -- 172.21.15.170:0/581613834 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de22d80 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9eaaf08980 1 Processor -- start 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9eaaf08980 1 -- start start 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9eaaf08980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de33760 0x561f7de33b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9eaaf08980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7dbc7e30 con 0x561f7de33760 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de33760 0x561f7de33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de33760 0x561f7de33b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54904/0 (socket says 172.21.15.170:54904) 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8a23640 1 -- 172.21.15.170:0/3553295088 learned_addr learned my addr 172.21.15.170:0/3553295088 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.647+0000 7f9ea8a23640 1 -- 172.21.15.170:0/3553295088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7de340c0 con 0x561f7de33760 2024-03-28T16:49:12.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de33760 0x561f7de33b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0013fd0 tx=0x7f9ea001d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d8e7fc640 1 -- 172.21.15.170:0/3553295088 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0012aa0 con 0x561f7de33760 2024-03-28T16:49:12.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561f7de34310 con 0x561f7de33760 2024-03-28T16:49:12.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d8e7fc640 1 -- 172.21.15.170:0/3553295088 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0012c00 con 0x561f7de33760 2024-03-28T16:49:12.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d8e7fc640 1 -- 172.21.15.170:0/3553295088 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0003ed0 con 0x561f7de33760 2024-03-28T16:49:12.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f7de34e60 con 0x561f7de33760 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d8e7fc640 1 -- 172.21.15.170:0/3553295088 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9ea0004030 con 0x561f7de33760 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d8e7fc640 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d800342b0 0x7f9d80036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d8e7fc640 1 -- 172.21.15.170:0/3553295088 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f9ea004adc0 con 0x561f7de33760 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8222640 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d800342b0 0x7f9d80036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9eaaf08980 1 Processor -- start 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9eaaf08980 1 -- start start 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8222640 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d800342b0 0x7f9d80036770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x561f7de21cf0 tx=0x7f9e9c005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9eaaf08980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de408e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9eaaf08980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7dbbb790 con 0x561f7de404c0 2024-03-28T16:49:12.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54910/0 (socket says 172.21.15.170:54910) 2024-03-28T16:49:12.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8a23640 1 -- 172.21.15.170:0/2155497423 learned_addr learned my addr 172.21.15.170:0/2155497423 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8a23640 1 -- 172.21.15.170:0/2155497423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7de40eb0 con 0x561f7de404c0 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/2155497423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de408e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000f90 tx=0x7f9ea0028c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba243e4c5a68faa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d7ffff640 1 -- 172.21.15.170:0/2155497423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea00327a0 con 0x561f7de404c0 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d7ffff640 1 -- 172.21.15.170:0/2155497423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0035140 con 0x561f7de404c0 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.651+0000 7f9d7ffff640 1 -- 172.21.15.170:0/2155497423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0032960 con 0x561f7de404c0 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/2155497423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 msgr2=0x561f7de408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/2155497423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de408e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0000f90 tx=0x7f9ea0028c00 comp rx=0 tx=0).stop 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/2155497423 shutdown_connections 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/2155497423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de408e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/2155497423 >> 172.21.15.170:0/2155497423 conn(0x561f7de3c800 msgr2=0x561f7de3a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/2155497423 shutdown_connections 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/2155497423 wait complete. 2024-03-28T16:49:12.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 Processor -- start 2024-03-28T16:49:12.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- start start 2024-03-28T16:49:12.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7de47830 con 0x561f7de404c0 2024-03-28T16:49:12.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9ea8a23640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54920/0 (socket says 172.21.15.170:54920) 2024-03-28T16:49:12.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 learned_addr learned my addr 172.21.15.170:0/564459411 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7de340c0 con 0x561f7de404c0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de472f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0027f60 tx=0x7f9ea0015120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9d7e7fc640 1 -- 172.21.15.170:0/564459411 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea0063070 con 0x561f7de404c0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9d7e7fc640 1 -- 172.21.15.170:0/564459411 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0027bc0 con 0x561f7de404c0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561f7de340c0 con 0x561f7de404c0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9d7e7fc640 1 -- 172.21.15.170:0/564459411 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ea00460e0 con 0x561f7de404c0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f7de340c0 con 0x561f7de404c0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9d7e7fc640 1 -- 172.21.15.170:0/564459411 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9ea0011760 con 0x561f7de404c0 2024-03-28T16:49:12.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9d7e7fc640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d74034260 0x7f9d74036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9d7e7fc640 1 -- 172.21.15.170:0/564459411 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f9ea0073af0 con 0x561f7de404c0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9ea8222640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d74034260 0x7f9d74036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7de525b0 0x561f7de549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55130 con 0x561f7de525b0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.655+0000 7f9ea9224640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7de525b0 0x561f7de549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8222640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d74034260 0x7f9d74036720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c01b2c0 tx=0x7f9e9c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea9224640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7de525b0 0x561f7de549e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 realm 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55550 con 0x561f7de525b0 2024-03-28T16:49:12.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de56100 con 0x561f7de525b0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de56ab0 0x561f7de58ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de59540 con 0x561f7de56ab0 2024-03-28T16:49:12.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de56ab0 0x561f7de58ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8a23640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de56ab0 0x561f7de58ea0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 20 searching for the correct realm 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561f7de5b700 0x561f7de5dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5e240 con 0x561f7de5b700 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8222640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561f7de5b700 0x561f7de5dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8222640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561f7de5b700 0x561f7de5dbc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8222640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e9c0da040 0x7f9e9c0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5e240 con 0x7f9e9c0da040 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.659+0000 7f9ea9224640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e9c0da040 0x7f9e9c0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e9c0da040 0x7f9e9c0cc420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x7f9e9c0da040 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.663+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x7f9e9c0da040 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.667+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x7f9e9c0da040 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x7f9e9c0da040 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de5b700 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de56ab0 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x561f7de525b0 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de55da0 con 0x7f9e9c0da040 2024-03-28T16:49:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de59b00 con 0x561f7de525b0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.671+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de59b00 con 0x561f7de525b0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de59ab0 con 0x7f9e9c0da040 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 zone default found 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 4 Realm: () 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 zonegroup default 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5a350 con 0x561f7de56ab0 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5a770 con 0x561f7de5b700 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de60180 con 0x561f7de5b700 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 started sync module instance, tier type = 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:12.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5f940 con 0x7f9e9c0da040 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94693256197280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5fcf0 con 0x7f9e9c0da040 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.675+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de60ee0 con 0x561f7de525b0 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94693256197280] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94693256201632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de60e60 con 0x561f7de525b0 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de618e0 con 0x561f7de5b700 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94693256205280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de62130 con 0x561f7de5b700 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de62ae0 con 0x7f9e9c0da040 2024-03-28T16:49:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.679+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94693256201632] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94693256202688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de62ec0 con 0x7f9e9c0da040 2024-03-28T16:49:12.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de634d0 con 0x561f7de525b0 2024-03-28T16:49:12.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94693256205280] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94693256202688] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94693256207504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de62710 con 0x561f7de525b0 2024-03-28T16:49:12.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de63c10 con 0x561f7de56ab0 2024-03-28T16:49:12.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94693256214256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de64460 con 0x561f7de56ab0 2024-03-28T16:49:12.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de64e10 con 0x561f7de56ab0 2024-03-28T16:49:12.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.683+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94693256207504] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94693256212528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de65610 con 0x561f7de56ab0 2024-03-28T16:49:12.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de65f80 con 0x561f7de56ab0 2024-03-28T16:49:12.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94693256214256] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94693256223456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de663a0 con 0x561f7de56ab0 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 20 add_watcher() i=0 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 20 add_watcher() i=1 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 20 add_watcher() i=2 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 20 add_watcher() i=3 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 20 add_watcher() i=4 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 20 add_watcher() i=5 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94693256212528] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.687+0000 7f9eaaf08980 20 add_watcher() i=6 2024-03-28T16:49:12.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94693256223456] v29'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9eaaf08980 20 add_watcher() i=7 2024-03-28T16:49:12.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9eaaf08980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:12.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de693d0 con 0x561f7de525b0 2024-03-28T16:49:12.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94693256224512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de667c0 con 0x561f7de525b0 2024-03-28T16:49:12.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94693256224512] v29'36 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9d5efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9eaaf08980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:12.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9eaaf08980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:12.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.691+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de71790 con 0x561f7de525b0 2024-03-28T16:49:12.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de71840 con 0x561f7de56ab0 2024-03-28T16:49:12.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de72170 con 0x561f7de5b700 2024-03-28T16:49:12.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de72970 con 0x561f7de525b0 2024-03-28T16:49:12.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'37 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.695+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de72d50 con 0x7f9e9c0da040 2024-03-28T16:49:12.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.699+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'26 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.699+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de731f0 con 0x561f7de5b700 2024-03-28T16:49:12.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.699+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.699+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de6f7d0 con 0x561f7de525b0 2024-03-28T16:49:12.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.699+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.699+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de6f7d0 con 0x561f7de56ab0 2024-03-28T16:49:12.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.703+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'17 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.703+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de6f7d0 con 0x7f9e9c0da040 2024-03-28T16:49:12.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.703+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.703+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de6f7d0 con 0x561f7de5b700 2024-03-28T16:49:12.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.707+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.707+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de73f30 con 0x7f9e9c0da040 2024-03-28T16:49:12.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.707+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'27 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.707+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de73f30 con 0x561f7de56ab0 2024-03-28T16:49:12.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.707+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.707+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74460 con 0x7f9e9c0da040 2024-03-28T16:49:12.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.711+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.711+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74460 con 0x561f7de56ab0 2024-03-28T16:49:12.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.711+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'23 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.711+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de525b0 2024-03-28T16:49:12.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.711+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'38 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.711+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.715+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.715+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x7f9e9c0da040 2024-03-28T16:49:12.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.715+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'28 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.715+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.715+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.715+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de525b0 2024-03-28T16:49:12.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.719+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.719+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de525b0 2024-03-28T16:49:12.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.719+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'39 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.719+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.719+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'21 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.719+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x7f9e9c0da040 2024-03-28T16:49:12.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.723+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'22 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.723+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.723+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.723+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.723+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.723+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.727+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'8 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.727+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.727+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'18 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.727+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de56ab0 2024-03-28T16:49:12.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.727+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.727+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.731+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.731+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de56ab0 2024-03-28T16:49:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.731+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'18 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.731+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de5b700 2024-03-28T16:49:12.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.735+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'22 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.735+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x561f7de56ab0 2024-03-28T16:49:12.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.735+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'24 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.735+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de74900 con 0x7f9e9c0da040 2024-03-28T16:49:12.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9d5dfe3640 20 reqs_thread_entry: start 2024-03-28T16:49:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 20 Filter name: none 2024-03-28T16:49:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9ccf7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:49:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de6daf0 con 0x561f7de56ab0 2024-03-28T16:49:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(94 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-03-28T16:49:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:12.740 INFO:teuthology.orchestra.run.smithi170.stderr:could not remove user: unable to remove user, user does not exist 2024-03-28T16:49:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9d5efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.739+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94693256224512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de6e360 con 0x561f7de525b0 2024-03-28T16:49:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94693256224512] v29'40 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94693256197280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de6e360 con 0x7f9e9c0da040 2024-03-28T16:49:12.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94693256197280] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9eaaf08980 20 remove_watcher() i=0 2024-03-28T16:49:12.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9eaaf08980 2 removed watcher, disabling cache 2024-03-28T16:49:12.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94693256201632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5f4a0 con 0x561f7de525b0 2024-03-28T16:49:12.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94693256201632] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9eaaf08980 20 remove_watcher() i=1 2024-03-28T16:49:12.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.743+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94693256205280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de5f4a0 con 0x561f7de5b700 2024-03-28T16:49:12.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9ea8222640 1 -- 172.21.15.170:0/564459411 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94693256205280] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e9c0abbe0 con 0x561f7de5b700 2024-03-28T16:49:12.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9eaaf08980 20 remove_watcher() i=2 2024-03-28T16:49:12.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94693256202688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de613e0 con 0x7f9e9c0da040 2024-03-28T16:49:12.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94693256202688] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x7f9e9c0da040 2024-03-28T16:49:12.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9eaaf08980 20 remove_watcher() i=3 2024-03-28T16:49:12.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94693256207504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de60a40 con 0x561f7de525b0 2024-03-28T16:49:12.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94693256207504] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e980083f0 con 0x561f7de525b0 2024-03-28T16:49:12.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9eaaf08980 20 remove_watcher() i=4 2024-03-28T16:49:12.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.747+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94693256214256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de60a40 con 0x561f7de56ab0 2024-03-28T16:49:12.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.751+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94693256214256] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.751+0000 7f9eaaf08980 20 remove_watcher() i=5 2024-03-28T16:49:12.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.751+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94693256212528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de60a40 con 0x561f7de56ab0 2024-03-28T16:49:12.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.751+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94693256212528] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.751+0000 7f9eaaf08980 20 remove_watcher() i=6 2024-03-28T16:49:12.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.751+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94693256223456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561f7de60a40 con 0x561f7de56ab0 2024-03-28T16:49:12.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9ea8a23640 1 -- 172.21.15.170:0/564459411 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94693256223456] v29'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9ea0024020 con 0x561f7de56ab0 2024-03-28T16:49:12.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 20 remove_watcher() i=7 2024-03-28T16:49:12.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de56ab0 msgr2=0x561f7de58ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de56ab0 0x561f7de58ea0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7de525b0 msgr2=0x561f7de549e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7de525b0 0x561f7de549e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e9c0da040 msgr2=0x7f9e9c0cc420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e9c0da040 0x7f9e9c0cc420 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561f7de5b700 msgr2=0x561f7de5dbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561f7de5b700 0x561f7de5dbc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d74034260 msgr2=0x7f9d74036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d74034260 0x7f9d74036720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c01b2c0 tx=0x7f9e9c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 msgr2=0x561f7de472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/564459411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de404c0 0x561f7de472f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0027f60 tx=0x7f9ea0015120 comp rx=0 tx=0).stop 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9ea9224640 1 -- 172.21.15.170:0/564459411 reap_dead start 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 shutdown_connections 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 >> 172.21.15.170:0/564459411 conn(0x561f7de3c800 msgr2=0x561f7de39e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 shutdown_connections 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/564459411 wait complete. 2024-03-28T16:49:12.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d800342b0 msgr2=0x7f9d80036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d800342b0 0x7f9d80036770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x561f7de21cf0 tx=0x7f9e9c005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de33760 msgr2=0x561f7de33b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de33760 0x561f7de33b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0013fd0 tx=0x7f9ea001d440 comp rx=0 tx=0).stop 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 shutdown_connections 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d800342b0 0x7f9d80036770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/3553295088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7de33760 0x561f7de33b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 >> 172.21.15.170:0/3553295088 conn(0x561f7de28570 msgr2=0x561f7de29080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 shutdown_connections 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/3553295088 wait complete. 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de1edb0 msgr2=0x561f7de211d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561f7de1edb0 0x561f7de211d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7dd29750 msgr2=0x561f7de1cb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561f7dd29750 0x561f7de1cb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561f7de22d80 msgr2=0x561f7de25170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561f7de22d80 0x561f7de25170 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d900342b0 msgr2=0x7f9d90036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d900342b0 0x7f9d90036770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0ad000 tx=0x7f9e9c0a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 msgr2=0x561f7de15f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 --2- 172.21.15.170:0/581613834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561f7dc978b0 0x561f7de15f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001a90 tx=0x7f9ea0001260 comp rx=0 tx=0).stop 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9ea9224640 1 -- 172.21.15.170:0/581613834 reap_dead start 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 shutdown_connections 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 >> 172.21.15.170:0/581613834 conn(0x561f7dd1d710 msgr2=0x561f7de09750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 shutdown_connections 2024-03-28T16:49:12.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.755+0000 7f9eaaf08980 1 -- 172.21.15.170:0/581613834 wait complete. 2024-03-28T16:49:12.765 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-28T16:49:12.766 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-03-28T16:49:12.766 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-03-28T16:49:12.766 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-03-28T16:49:12.837 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:12.838 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:12.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d39bf8980 1 Processor -- start 2024-03-28T16:49:12.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d39bf8980 1 -- start start 2024-03-28T16:49:12.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d39bf8980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55a5e29f78b0 0x55a5e2a88030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d39bf8980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a5e29cf520 con 0x55a5e2a88570 2024-03-28T16:49:12.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d39bf8980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a5e29960f0 con 0x55a5e29f78b0 2024-03-28T16:49:12.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d37713640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55a5e29f78b0 0x55a5e2a88030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d37713640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55a5e29f78b0 0x55a5e2a88030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54928/0 (socket says 172.21.15.170:54928) 2024-03-28T16:49:12.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d37713640 1 -- 172.21.15.170:0/85879971 learned_addr learned my addr 172.21.15.170:0/85879971 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d37713640 1 -- 172.21.15.170:0/85879971 >> v1:172.21.15.119:6789/0 conn(0x55a5e2a88570 legacy=0x55a5e2a8a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.839+0000 7f8d37713640 1 -- 172.21.15.170:0/85879971 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e2a83a50 con 0x55a5e29f78b0 2024-03-28T16:49:12.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d37713640 1 --2- 172.21.15.170:0/85879971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2a88030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800a5c0 tx=0x7f8d2801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a56c1fe826ffb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d36711640 1 -- 172.21.15.170:0/85879971 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280aa470 con 0x55a5e29f78b0 2024-03-28T16:49:12.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d36711640 1 -- 172.21.15.170:0/85879971 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d280aa5d0 con 0x55a5e29f78b0 2024-03-28T16:49:12.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d36711640 1 -- 172.21.15.170:0/85879971 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280aa890 con 0x55a5e29f78b0 2024-03-28T16:49:12.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d39bf8980 1 -- 172.21.15.170:0/85879971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 msgr2=0x55a5e2a88030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/85879971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2a88030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800a5c0 tx=0x7f8d2801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:12.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d39bf8980 1 -- 172.21.15.170:0/85879971 shutdown_connections 2024-03-28T16:49:12.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/85879971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2a88030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d39bf8980 1 -- 172.21.15.170:0/85879971 >> 172.21.15.170:0/85879971 conn(0x55a5e2a7d710 msgr2=0x55a5e2a7fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d39bf8980 1 -- 172.21.15.170:0/85879971 shutdown_connections 2024-03-28T16:49:12.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.843+0000 7f8d39bf8980 1 -- 172.21.15.170:0/85879971 wait complete. 2024-03-28T16:49:12.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d39bf8980 1 Processor -- start 2024-03-28T16:49:12.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d39bf8980 1 -- start start 2024-03-28T16:49:12.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d39bf8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b6f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d39bf8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2993620 con 0x55a5e29f78b0 2024-03-28T16:49:12.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b6f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b6f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54938/0 (socket says 172.21.15.170:54938) 2024-03-28T16:49:12.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d37713640 1 -- 172.21.15.170:0/3487348478 learned_addr learned my addr 172.21.15.170:0/3487348478 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d37713640 1 -- 172.21.15.170:0/3487348478 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d28005df0 con 0x55a5e29f78b0 2024-03-28T16:49:12.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.855+0000 7f8d37713640 1 --2- 172.21.15.170:0/3487348478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b6f080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d28018f70 tx=0x7f8d28003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dade345579065235 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8c2560a640 1 -- 172.21.15.170:0/3487348478 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280aa9c0 con 0x55a5e29f78b0 2024-03-28T16:49:12.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8c2560a640 1 -- 172.21.15.170:0/3487348478 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d280aab20 con 0x55a5e29f78b0 2024-03-28T16:49:12.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8c2560a640 1 -- 172.21.15.170:0/3487348478 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280aadc0 con 0x55a5e29f78b0 2024-03-28T16:49:12.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3487348478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 msgr2=0x55a5e2b6f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3487348478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b6f080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d28018f70 tx=0x7f8d28003030 comp rx=0 tx=0).stop 2024-03-28T16:49:12.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3487348478 shutdown_connections 2024-03-28T16:49:12.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3487348478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b6f080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3487348478 >> 172.21.15.170:0/3487348478 conn(0x55a5e2a7d710 msgr2=0x55a5e2b6ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3487348478 shutdown_connections 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3487348478 wait complete. 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 Processor -- start 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- start start 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b75dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e298ee00 con 0x55a5e29f78b0 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b75dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b75dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54940/0 (socket says 172.21.15.170:54940) 2024-03-28T16:49:12.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d37713640 1 -- 172.21.15.170:0/554662561 learned_addr learned my addr 172.21.15.170:0/554662561 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d37713640 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e2a83a50 con 0x55a5e29f78b0 2024-03-28T16:49:12.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d37713640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b75dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8d28018f70 tx=0x7f8d280180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8c23e07640 1 -- 172.21.15.170:0/554662561 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280cb070 con 0x55a5e29f78b0 2024-03-28T16:49:12.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8c23e07640 1 -- 172.21.15.170:0/554662561 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d280bd810 con 0x55a5e29f78b0 2024-03-28T16:49:12.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a5e2b76310 con 0x55a5e29f78b0 2024-03-28T16:49:12.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8c23e07640 1 -- 172.21.15.170:0/554662561 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280bdab0 con 0x55a5e29f78b0 2024-03-28T16:49:12.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.859+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e2b766e0 con 0x55a5e29f78b0 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8c23e07640 1 -- 172.21.15.170:0/554662561 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8d280bdce0 con 0x55a5e29f78b0 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8c23e07640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c1c034300 0x7f8c1c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8c23e07640 1 -- 172.21.15.170:0/554662561 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8d280e48f0 con 0x55a5e29f78b0 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d36f12640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c1c034300 0x7f8c1c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2a88570 0x55a5e2b7ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2b7cf40 con 0x55a5e2a88570 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37f14640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2a88570 0x55a5e2b7ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d36f12640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c1c034300 0x7f8c1c0367c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c000f70 tx=0x7f8d2c001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37f14640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2a88570 0x55a5e2b7ca00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37f14640 1 -- 172.21.15.170:0/554662561 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2a88570 2024-03-28T16:49:12.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2b7d2a0 con 0x55a5e2a88570 2024-03-28T16:49:12.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37f14640 1 -- 172.21.15.170:0/554662561 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2a88570 2024-03-28T16:49:12.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2b7ec30 0x55a5e2b81020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2b81790 con 0x55a5e2b7ec30 2024-03-28T16:49:12.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37713640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2b7ec30 0x55a5e2b81020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37713640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2b7ec30 0x55a5e2b81020 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37713640 1 -- 172.21.15.170:0/554662561 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2b7ec30 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2b7daa0 con 0x55a5e2a88570 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d37f14640 1 -- 172.21.15.170:0/554662561 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2a88570 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a5e2b82b70 0x55a5e2b84f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.863+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2b85600 con 0x55a5e2b82b70 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d36f12640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a5e2b82b70 0x55a5e2b84f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d36f12640 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a5e2b82b70 0x55a5e2b84f60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d36f12640 1 -- 172.21.15.170:0/554662561 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f8d2c0106b0 con 0x55a5e2b82b70 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d39bf8980 1 Processor -- start 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d39bf8980 1 -- start start 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d39bf8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2b93530 0x55a5e2b93950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d39bf8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2927e30 con 0x55a5e2b93530 2024-03-28T16:49:12.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2b93530 0x55a5e2b93950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2b93530 0x55a5e2b93950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54956/0 (socket says 172.21.15.170:54956) 2024-03-28T16:49:12.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d37713640 1 -- 172.21.15.170:0/4278338090 learned_addr learned my addr 172.21.15.170:0/4278338090 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d37713640 1 -- 172.21.15.170:0/4278338090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e2b93e90 con 0x55a5e2b93530 2024-03-28T16:49:12.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d37713640 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2b93530 0x55a5e2b93950 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d280cf940 tx=0x7f8d280c5e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8c21602640 1 -- 172.21.15.170:0/4278338090 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280cfe80 con 0x55a5e2b93530 2024-03-28T16:49:12.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a5e2b940e0 con 0x55a5e2b93530 2024-03-28T16:49:12.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8c21602640 1 -- 172.21.15.170:0/4278338090 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d280c6690 con 0x55a5e2b93530 2024-03-28T16:49:12.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8c21602640 1 -- 172.21.15.170:0/4278338090 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280c6930 con 0x55a5e2b93530 2024-03-28T16:49:12.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.867+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e2b94c30 con 0x55a5e2b93530 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8c21602640 1 -- 172.21.15.170:0/4278338090 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8d280c53e0 con 0x55a5e2b93530 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8c21602640 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c100342b0 0x7f8c10036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8c21602640 1 -- 172.21.15.170:0/4278338090 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8d280eeb30 con 0x55a5e2b93530 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d36f12640 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c100342b0 0x7f8c10036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 Processor -- start 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- start start 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d36f12640 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c100342b0 0x7f8c10036770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55a5e2b7de00 tx=0x7f8d2c006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e291b790 con 0x55a5e2ba0500 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54962/0 (socket says 172.21.15.170:54962) 2024-03-28T16:49:12.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 -- 172.21.15.170:0/3526328184 learned_addr learned my addr 172.21.15.170:0/3526328184 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 -- 172.21.15.170:0/3526328184 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e2ba0ed0 con 0x55a5e2ba0500 2024-03-28T16:49:12.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 --2- 172.21.15.170:0/3526328184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba0900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8d280bdcb0 tx=0x7f8d280dd8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22919976ff9ee828 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8c1affd640 1 -- 172.21.15.170:0/3526328184 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280a7290 con 0x55a5e2ba0500 2024-03-28T16:49:12.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8c1affd640 1 -- 172.21.15.170:0/3526328184 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d280a4100 con 0x55a5e2ba0500 2024-03-28T16:49:12.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8c1affd640 1 -- 172.21.15.170:0/3526328184 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280eee10 con 0x55a5e2ba0500 2024-03-28T16:49:12.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3526328184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 msgr2=0x55a5e2ba0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3526328184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba0900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8d280bdcb0 tx=0x7f8d280dd8a0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3526328184 shutdown_connections 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3526328184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba0900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3526328184 >> 172.21.15.170:0/3526328184 conn(0x55a5e2b9c890 msgr2=0x55a5e2b9a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3526328184 shutdown_connections 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3526328184 wait complete. 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 Processor -- start 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- start start 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d39bf8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e2ba78f0 con 0x55a5e2ba0500 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54966/0 (socket says 172.21.15.170:54966) 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 learned_addr learned my addr 172.21.15.170:0/3466736817 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.871+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e2b94c30 con 0x55a5e2ba0500 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d37713640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba73b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d280ddbd0 tx=0x7f8d280e9f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8c197fa640 1 -- 172.21.15.170:0/3466736817 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280ddc40 con 0x55a5e2ba0500 2024-03-28T16:49:12.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a5e2b94c30 con 0x55a5e2ba0500 2024-03-28T16:49:12.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e2b94c30 con 0x55a5e2ba0500 2024-03-28T16:49:12.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8c197fa640 1 -- 172.21.15.170:0/3466736817 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d280ddda0 con 0x55a5e2ba0500 2024-03-28T16:49:12.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8c197fa640 1 -- 172.21.15.170:0/3466736817 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280a7b10 con 0x55a5e2ba0500 2024-03-28T16:49:12.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8c197fa640 1 -- 172.21.15.170:0/3466736817 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8d28108040 con 0x55a5e2ba0500 2024-03-28T16:49:12.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8c197fa640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c00034300 0x7f8c000367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8c197fa640 1 -- 172.21.15.170:0/3466736817 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8d280fc050 con 0x55a5e2ba0500 2024-03-28T16:49:12.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d36f12640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c00034300 0x7f8c000367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2bb2510 0x55a5e2bb49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5120 con 0x55a5e2bb2510 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d37f14640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2bb2510 0x55a5e2bb49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d36f12640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c00034300 0x7f8c000367c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55a5e2b859c0 tx=0x7f8d2c011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d37f14640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2bb2510 0x55a5e2bb49d0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 realm 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5540 con 0x55a5e2bb2510 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb60f0 con 0x55a5e2bb2510 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2bb6aa0 0x55a5e2bb8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.875+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb9530 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37713640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2bb6aa0 0x55a5e2bb8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37713640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2bb6aa0 0x55a5e2bb8e90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d39bf8980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d39bf8980 20 searching for the correct realm 2024-03-28T16:49:12.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a5e2bbb6b0 0x55a5e2bbdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbe1f0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d36f12640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a5e2bbb6b0 0x55a5e2bbdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d36f12640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a5e2bbb6b0 0x55a5e2bbdb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d36f12640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d2c00f3d0 0x7f8d2c00f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbe1f0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37f14640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d2c00f3d0 0x7f8d2c00f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37f14640 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d2c00f3d0 0x7f8d2c00f810 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.879+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.883+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.887+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x55a5e2bb2510 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb5d90 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb9ad0 con 0x55a5e2bb2510 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb9ad0 con 0x55a5e2bb2510 2024-03-28T16:49:12.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bb9ad0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 zone default found 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 4 Realm: () 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 zonegroup default 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.891+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bba320 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc0010 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc0370 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 20 started sync module instance, tier type = 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbf8f0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94170961933392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbfca0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbf870 con 0x55a5e2bb2510 2024-03-28T16:49:12.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94170961933392] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94170961938128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc0f90 con 0x55a5e2bb2510 2024-03-28T16:49:12.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.895+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc1980 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94170961938128] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94170961939184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc0f10 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc2670 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94170961945312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc2ec0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc3870 con 0x55a5e2bb2510 2024-03-28T16:49:12.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.899+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94170961939184] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94170961945312] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94170961943856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc29d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc40c0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94170961943856] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94170961951776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc29d0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.903+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc48c0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94170961954336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc5110 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc5b80 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94170961951776] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94170961958592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc5b80 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 20 add_watcher() i=0 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 20 add_watcher() i=1 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 20 add_watcher() i=2 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 20 add_watcher() i=3 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 20 add_watcher() i=4 2024-03-28T16:49:12.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.907+0000 7f8d39bf8980 20 add_watcher() i=5 2024-03-28T16:49:12.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94170961954336] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 20 add_watcher() i=6 2024-03-28T16:49:12.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94170961958592] v29'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 20 add_watcher() i=7 2024-03-28T16:49:12.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:12.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc4c70 con 0x55a5e2bb2510 2024-03-28T16:49:12.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94170961949584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc4c70 con 0x55a5e2bb2510 2024-03-28T16:49:12.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94170961949584] v29'41 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8bedfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:12.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:12.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.911+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd1110 con 0x55a5e2bb2510 2024-03-28T16:49:12.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.915+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.915+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd11c0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.915+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.915+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd1af0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.915+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.919+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd20b0 con 0x55a5e2bb2510 2024-03-28T16:49:12.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.919+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'42 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.919+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd2660 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.919+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'29 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.919+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd2b90 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.919+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.919+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd30c0 con 0x55a5e2bb2510 2024-03-28T16:49:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.923+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.923+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd3560 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.923+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'19 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.923+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bcf150 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.927+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.927+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bcf760 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.927+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.927+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bcf760 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.927+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'30 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.927+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd40d0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.931+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.931+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4570 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.931+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.931+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4570 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'25 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4570 con 0x55a5e2bb2510 2024-03-28T16:49:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'43 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4570 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4740 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'31 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.935+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4740 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.939+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.939+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4740 con 0x55a5e2bb2510 2024-03-28T16:49:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.939+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.939+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd47d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.939+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'44 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.939+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd4b50 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.943+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'24 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.943+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.943+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'23 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.943+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.943+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.943+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.947+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.947+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.947+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'9 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.947+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.947+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.947+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.951+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.951+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.951+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.951+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.955+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.955+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.955+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'25 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.955+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'26 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bceae0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8becfe1640 20 reqs_thread_entry: start 2024-03-28T16:49:12.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d39bf8980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:12.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d39bf8980 20 Filter name: none 2024-03-28T16:49:12.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8b667fc640 20 reqs_thread_entry: start 2024-03-28T16:49:12.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d39bf8980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:12.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d39bf8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:12.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.959+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bcd470 con 0x55a5e2bb2510 2024-03-28T16:49:12.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:could not remove user: unable to remove user, user does not exist 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8bedfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94170961949584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bcde40 con 0x55a5e2bb2510 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94170961949584] v29'45 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94170961933392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bd02f0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94170961933392] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 20 remove_watcher() i=0 2024-03-28T16:49:12.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 2 removed watcher, disabling cache 2024-03-28T16:49:12.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.963+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94170961938128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbf450 con 0x55a5e2bb2510 2024-03-28T16:49:12.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94170961938128] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d39bf8980 20 remove_watcher() i=1 2024-03-28T16:49:12.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94170961939184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbf450 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d36f12640 1 -- 172.21.15.170:0/3466736817 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94170961939184] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d2c032040 con 0x55a5e2bbb6b0 2024-03-28T16:49:12.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d39bf8980 20 remove_watcher() i=2 2024-03-28T16:49:12.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94170961945312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbf450 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94170961945312] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x7f8d2c00f3d0 2024-03-28T16:49:12.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d39bf8980 20 remove_watcher() i=3 2024-03-28T16:49:12.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.967+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94170961943856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbf450 con 0x55a5e2bb2510 2024-03-28T16:49:12.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.971+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94170961943856] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d3000a4d0 con 0x55a5e2bb2510 2024-03-28T16:49:12.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.971+0000 7f8d39bf8980 20 remove_watcher() i=4 2024-03-28T16:49:12.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.971+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94170961951776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc1d30 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.971+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94170961951776] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.971+0000 7f8d39bf8980 20 remove_watcher() i=5 2024-03-28T16:49:12.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.971+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94170961954336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bbf450 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94170961954336] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 20 remove_watcher() i=6 2024-03-28T16:49:12.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94170961958592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a5e2bc3c20 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d37713640 1 -- 172.21.15.170:0/3466736817 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94170961958592] v29'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d280cd070 con 0x55a5e2bb6aa0 2024-03-28T16:49:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 20 remove_watcher() i=7 2024-03-28T16:49:12.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2bb6aa0 msgr2=0x55a5e2bb8e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2bb6aa0 0x55a5e2bb8e90 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2bb2510 msgr2=0x55a5e2bb49d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2bb2510 0x55a5e2bb49d0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d2c00f3d0 msgr2=0x7f8d2c00f810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d2c00f3d0 0x7f8d2c00f810 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a5e2bbb6b0 msgr2=0x55a5e2bbdb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.975+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a5e2bbb6b0 0x55a5e2bbdb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c00034300 msgr2=0x7f8c000367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c00034300 0x7f8c000367c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55a5e2b859c0 tx=0x7f8d2c011000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d37f14640 1 -- 172.21.15.170:0/3466736817 reap_dead start 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 msgr2=0x55a5e2ba73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba73b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d280ddbd0 tx=0x7f8d280e9f40 comp rx=0 tx=0).stop 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 shutdown_connections 2024-03-28T16:49:12.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/3466736817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2ba0500 0x55a5e2ba73b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 >> 172.21.15.170:0/3466736817 conn(0x55a5e2b9c890 msgr2=0x55a5e2b99ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 shutdown_connections 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/3466736817 wait complete. 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c100342b0 msgr2=0x7f8c10036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c100342b0 0x7f8c10036770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55a5e2b7de00 tx=0x7f8d2c006000 comp rx=0 tx=0).stop 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2b93530 msgr2=0x55a5e2b93950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2b93530 0x55a5e2b93950 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d280cf940 tx=0x7f8d280c5e00 comp rx=0 tx=0).stop 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 shutdown_connections 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c100342b0 0x7f8c10036770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/4278338090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e2b93530 0x55a5e2b93950 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 >> 172.21.15.170:0/4278338090 conn(0x55a5e2b88340 msgr2=0x55a5e2b88e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 shutdown_connections 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/4278338090 wait complete. 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2b7ec30 msgr2=0x55a5e2b81020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a5e2b7ec30 0x55a5e2b81020 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2a88570 msgr2=0x55a5e2b7ca00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a5e2a88570 0x55a5e2b7ca00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a5e2b82b70 msgr2=0x55a5e2b84f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a5e2b82b70 0x55a5e2b84f60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c1c034300 msgr2=0x7f8c1c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c1c034300 0x7f8c1c0367c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c000f70 tx=0x7f8d2c001420 comp rx=0 tx=0).stop 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 msgr2=0x55a5e2b75dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 --2- 172.21.15.170:0/554662561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5e29f78b0 0x55a5e2b75dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8d28018f70 tx=0x7f8d280180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d37f14640 1 -- 172.21.15.170:0/554662561 reap_dead start 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 shutdown_connections 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 >> 172.21.15.170:0/554662561 conn(0x55a5e2a7d710 msgr2=0x55a5e2b69720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 shutdown_connections 2024-03-28T16:49:12.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:12.979+0000 7f8d39bf8980 1 -- 172.21.15.170:0/554662561 wait complete. 2024-03-28T16:49:12.987 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-28T16:49:12.987 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-03-28T16:49:12.987 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-03-28T16:49:12.988 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-03-28T16:49:13.065 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:13.065 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:13.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f588078f980 1 Processor -- start 2024-03-28T16:49:13.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f588078f980 1 -- start start 2024-03-28T16:49:13.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f588078f980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ed9753a9a0 0x55ed975cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f588078f980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ed975124c0 con 0x55ed975cb590 2024-03-28T16:49:13.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f588078f980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ed974d90f0 con 0x55ed9753a9a0 2024-03-28T16:49:13.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f587e2aa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ed9753a9a0 0x55ed975cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f587daa9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55ed975cb590 0x55ed975cd980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60810/0 (socket says 172.21.15.170:60810) 2024-03-28T16:49:13.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f587daa9640 1 -- 172.21.15.170:0/3200882608 learned_addr learned my addr 172.21.15.170:0/3200882608 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:13.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f587e2aa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ed9753a9a0 0x55ed975cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54970/0 (socket says 172.21.15.170:54970) 2024-03-28T16:49:13.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f587e2aa640 1 -- 172.21.15.170:0/3200882608 >> v1:172.21.15.119:6789/0 conn(0x55ed975cb590 legacy=0x55ed975cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.067+0000 7f587e2aa640 1 -- 172.21.15.170:0/3200882608 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed975c6a70 con 0x55ed9753a9a0 2024-03-28T16:49:13.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f587e2aa640 1 --2- 172.21.15.170:0/3200882608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed975cb050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f587800a420 tx=0x7f587801afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edced079927a35b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f587d2a8640 1 -- 172.21.15.170:0/3200882608 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780aa530 con 0x55ed9753a9a0 2024-03-28T16:49:13.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f587d2a8640 1 -- 172.21.15.170:0/3200882608 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58780a6050 con 0x55ed9753a9a0 2024-03-28T16:49:13.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f587d2a8640 1 -- 172.21.15.170:0/3200882608 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780aa800 con 0x55ed9753a9a0 2024-03-28T16:49:13.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f588078f980 1 -- 172.21.15.170:0/3200882608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 msgr2=0x55ed975cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f588078f980 1 --2- 172.21.15.170:0/3200882608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed975cb050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f587800a420 tx=0x7f587801afd0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f588078f980 1 -- 172.21.15.170:0/3200882608 shutdown_connections 2024-03-28T16:49:13.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f588078f980 1 --2- 172.21.15.170:0/3200882608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed975cb050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f588078f980 1 -- 172.21.15.170:0/3200882608 >> 172.21.15.170:0/3200882608 conn(0x55ed975c0730 msgr2=0x55ed975c2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f588078f980 1 -- 172.21.15.170:0/3200882608 shutdown_connections 2024-03-28T16:49:13.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.071+0000 7f588078f980 1 -- 172.21.15.170:0/3200882608 wait complete. 2024-03-28T16:49:13.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.083+0000 7f588078f980 1 Processor -- start 2024-03-28T16:49:13.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.083+0000 7f588078f980 1 -- start start 2024-03-28T16:49:13.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.083+0000 7f588078f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.083+0000 7f588078f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed974d6620 con 0x55ed9753a9a0 2024-03-28T16:49:13.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54974/0 (socket says 172.21.15.170:54974) 2024-03-28T16:49:13.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 -- 172.21.15.170:0/31481472 learned_addr learned my addr 172.21.15.170:0/31481472 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 -- 172.21.15.170:0/31481472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587800a100 con 0x55ed9753a9a0 2024-03-28T16:49:13.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 --2- 172.21.15.170:0/31481472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b21a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58780ad040 tx=0x7f58780033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb19b0fbbb01f08c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f576c608640 1 -- 172.21.15.170:0/31481472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780aa6c0 con 0x55ed9753a9a0 2024-03-28T16:49:13.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f576c608640 1 -- 172.21.15.170:0/31481472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58780aa820 con 0x55ed9753a9a0 2024-03-28T16:49:13.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f576c608640 1 -- 172.21.15.170:0/31481472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780aaac0 con 0x55ed9753a9a0 2024-03-28T16:49:13.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 -- 172.21.15.170:0/31481472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 msgr2=0x55ed976b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 --2- 172.21.15.170:0/31481472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b21a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f58780ad040 tx=0x7f58780033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 -- 172.21.15.170:0/31481472 shutdown_connections 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 --2- 172.21.15.170:0/31481472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b21a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 -- 172.21.15.170:0/31481472 >> 172.21.15.170:0/31481472 conn(0x55ed975c0730 msgr2=0x55ed976add30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 -- 172.21.15.170:0/31481472 shutdown_connections 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 -- 172.21.15.170:0/31481472 wait complete. 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 Processor -- start 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 -- start start 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f588078f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed974d1e00 con 0x55ed9753a9a0 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54986/0 (socket says 172.21.15.170:54986) 2024-03-28T16:49:13.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 -- 172.21.15.170:0/4168554550 learned_addr learned my addr 172.21.15.170:0/4168554550 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 -- 172.21.15.170:0/4168554550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed975c6a70 con 0x55ed9753a9a0 2024-03-28T16:49:13.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.087+0000 7f587e2aa640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b8ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f587801b4b0 tx=0x7f5878019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f576ae05640 1 -- 172.21.15.170:0/4168554550 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780c5070 con 0x55ed9753a9a0 2024-03-28T16:49:13.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f576ae05640 1 -- 172.21.15.170:0/4168554550 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58780b6bb0 con 0x55ed9753a9a0 2024-03-28T16:49:13.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f576ae05640 1 -- 172.21.15.170:0/4168554550 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780b6e50 con 0x55ed9753a9a0 2024-03-28T16:49:13.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ed976b9420 con 0x55ed9753a9a0 2024-03-28T16:49:13.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed976b98b0 con 0x55ed9753a9a0 2024-03-28T16:49:13.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 Processor -- start 2024-03-28T16:49:13.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- start start 2024-03-28T16:49:13.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed9746ae30 con 0x55ed976c5c60 2024-03-28T16:49:13.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f587eaab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f587eaab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54994/0 (socket says 172.21.15.170:54994) 2024-03-28T16:49:13.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f587eaab640 1 -- 172.21.15.170:0/3759463455 learned_addr learned my addr 172.21.15.170:0/3759463455 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f576ae05640 1 -- 172.21.15.170:0/4168554550 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f58780ca410 con 0x55ed9753a9a0 2024-03-28T16:49:13.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f576ae05640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57640342b0 0x7f5764036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f576ae05640 1 -- 172.21.15.170:0/4168554550 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f58780e3920 con 0x55ed9753a9a0 2024-03-28T16:49:13.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f587e2aa640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57640342b0 0x7f5764036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f587eaab640 1 -- 172.21.15.170:0/3759463455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed976c65c0 con 0x55ed976c5c60 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f587e2aa640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57640342b0 0x7f5764036770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58780b6740 tx=0x7f58780bff90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f587eaab640 1 --2- 172.21.15.170:0/3759463455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976c6080 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f587000a440 tx=0x7f587000a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f38e77824e80133 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f5768e01640 1 -- 172.21.15.170:0/3759463455 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5870008850 con 0x55ed976c5c60 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f5768e01640 1 -- 172.21.15.170:0/3759463455 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700089d0 con 0x55ed976c5c60 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f5768e01640 1 -- 172.21.15.170:0/3759463455 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5870008cb0 con 0x55ed976c5c60 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- 172.21.15.170:0/3759463455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 msgr2=0x55ed976c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 --2- 172.21.15.170:0/3759463455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976c6080 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f587000a440 tx=0x7f587000a900 comp rx=0 tx=0).stop 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- 172.21.15.170:0/3759463455 shutdown_connections 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 --2- 172.21.15.170:0/3759463455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976c6080 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- 172.21.15.170:0/3759463455 >> 172.21.15.170:0/3759463455 conn(0x55ed975cb590 msgr2=0x55ed976c10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- 172.21.15.170:0/3759463455 shutdown_connections 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.091+0000 7f588078f980 1 -- 172.21.15.170:0/3759463455 wait complete. 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 Processor -- start 2024-03-28T16:49:13.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 -- start start 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed9745e790 con 0x55ed976c5c60 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f587eaab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f587eaab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:54996/0 (socket says 172.21.15.170:54996) 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f587eaab640 1 -- 172.21.15.170:0/4142306622 learned_addr learned my addr 172.21.15.170:0/4142306622 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f587eaab640 1 -- 172.21.15.170:0/4142306622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870005cd0 con 0x55ed976c5c60 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f587eaab640 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976ccc00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ed976c6a40 tx=0x7f5870003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f5762ffd640 1 -- 172.21.15.170:0/4142306622 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5870013aa0 con 0x55ed976c5c60 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f5762ffd640 1 -- 172.21.15.170:0/4142306622 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5870013c00 con 0x55ed976c5c60 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ed976cd140 con 0x55ed976c5c60 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed976cd570 con 0x55ed976c5c60 2024-03-28T16:49:13.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f5762ffd640 1 -- 172.21.15.170:0/4142306622 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5870013ea0 con 0x55ed976c5c60 2024-03-28T16:49:13.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:13.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:13.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f5762ffd640 1 -- 172.21.15.170:0/4142306622 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f5870028460 con 0x55ed976c5c60 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 Processor -- start 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f5762ffd640 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57580342b0 0x7f5758036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f5762ffd640 1 -- 172.21.15.170:0/4142306622 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f5870049750 con 0x55ed976c5c60 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 -- start start 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f587eaab640 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57580342b0 0x7f5758036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976e6980 0x55ed976e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.095+0000 7f588078f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed976e72e0 con 0x55ed976e6980 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587eaab640 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57580342b0 0x7f5758036770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58700288d0 tx=0x7f5870028860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587e2aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976e6980 0x55ed976e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587e2aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976e6980 0x55ed976e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55010/0 (socket says 172.21.15.170:55010) 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587e2aa640 1 -- 172.21.15.170:0/1031036425 learned_addr learned my addr 172.21.15.170:0/1031036425 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587e2aa640 1 -- 172.21.15.170:0/1031036425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed976e7420 con 0x55ed976e6980 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587e2aa640 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976e6980 0x55ed976e6da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58780e1ec0 tx=0x7f58780bf990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f5742fe5640 1 -- 172.21.15.170:0/1031036425 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780b33a0 con 0x55ed976e6980 2024-03-28T16:49:13.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f5742fe5640 1 -- 172.21.15.170:0/1031036425 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58780b3500 con 0x55ed976e6980 2024-03-28T16:49:13.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ed976e7d10 con 0x55ed976e6980 2024-03-28T16:49:13.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f5742fe5640 1 -- 172.21.15.170:0/1031036425 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58780b38e0 con 0x55ed976e6980 2024-03-28T16:49:13.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed976e81a0 con 0x55ed976e6980 2024-03-28T16:49:13.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f5742fe5640 1 -- 172.21.15.170:0/1031036425 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f58780dd040 con 0x55ed976e6980 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f5742fe5640 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f56c0034300 0x7f56c00367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f5742fe5640 1 -- 172.21.15.170:0/1031036425 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f58780da040 con 0x55ed976e6980 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587daa9640 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f56c0034300 0x7f56c00367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f5740fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ed976eb8f0 0x55ed976edce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ed976ee220 con 0x55ed976eb8f0 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587daa9640 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f56c0034300 0x7f56c00367c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5874000f70 tx=0x7f5874001240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587eaab640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ed976eb8f0 0x55ed976edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.099+0000 7f587eaab640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ed976eb8f0 0x55ed976edce0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.103+0000 7f587eaab640 1 -- 172.21.15.170:0/4168554550 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5870024020 con 0x55ed976eb8f0 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.103+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ed976efc10 0x55ed976f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.103+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:2 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ed976f2770 con 0x55ed976efc10 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.103+0000 7f587e2aa640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ed976efc10 0x55ed976f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.103+0000 7f587e2aa640 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ed976efc10 0x55ed976f2000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.103+0000 7f587e2aa640 1 -- 172.21.15.170:0/4168554550 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(2 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f58780e9020 con 0x55ed976efc10 2024-03-28T16:49:13.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.103+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ed976f7270 con 0x55ed976efc10 2024-03-28T16:49:13.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f587e2aa640 1 -- 172.21.15.170:0/4168554550 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [stat,call,call,writefull 0~874] v29'2 uv2 ondisk = 0) v8 ==== 316+0+0 (crc 0 0 0) 0x7f58780e9020 con 0x55ed976efc10 2024-03-28T16:49:13.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57580342b0 msgr2=0x7f5758036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57580342b0 0x7f5758036770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58700288d0 tx=0x7f5870028860 comp rx=0 tx=0).stop 2024-03-28T16:49:13.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 msgr2=0x55ed976ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976ccc00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ed976c6a40 tx=0x7f5870003000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 shutdown_connections 2024-03-28T16:49:13.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57580342b0 0x7f5758036770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4142306622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976c5c60 0x55ed976ccc00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 >> 172.21.15.170:0/4142306622 conn(0x55ed975cb590 msgr2=0x55ed976c0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 shutdown_connections 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4142306622 wait complete. 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f56c0034300 msgr2=0x7f56c00367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f56c0034300 0x7f56c00367c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5874000f70 tx=0x7f5874001240 comp rx=0 tx=0).stop 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976e6980 msgr2=0x55ed976e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976e6980 0x55ed976e6da0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58780e1ec0 tx=0x7f58780bf990 comp rx=0 tx=0).stop 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 shutdown_connections 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f56c0034300 0x7f56c00367c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/1031036425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed976e6980 0x55ed976e6da0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 >> 172.21.15.170:0/1031036425 conn(0x55ed976db6a0 msgr2=0x55ed976dc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 shutdown_connections 2024-03-28T16:49:13.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/1031036425 wait complete. 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ed976efc10 msgr2=0x55ed976f2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ed976efc10 0x55ed976f2000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ed976eb8f0 msgr2=0x55ed976edce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ed976eb8f0 0x55ed976edce0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57640342b0 msgr2=0x7f5764036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57640342b0 0x7f5764036770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58780b6740 tx=0x7f58780bff90 comp rx=0 tx=0).stop 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 msgr2=0x55ed976b8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b8ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f587801b4b0 tx=0x7f5878019810 comp rx=0 tx=0).stop 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 shutdown_connections 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ed976efc10 0x55ed976f2000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ed976eb8f0 0x55ed976edce0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57640342b0 0x7f5764036770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 --2- 172.21.15.170:0/4168554550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9753a9a0 0x55ed976b8ee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 >> 172.21.15.170:0/4168554550 conn(0x55ed975c0730 msgr2=0x55ed976ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 shutdown_connections 2024-03-28T16:49:13.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.107+0000 7f588078f980 1 -- 172.21.15.170:0/4168554550 wait complete. 2024-03-28T16:49:13.117 INFO:teuthology.orchestra.run.smithi170.stdout:{"id":"b88c2392-6ef8-4660-b080-19480b748fc0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-28T16:49:13.117 DEBUG:tasks.util.rgw: json result: {'id': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-28T16:49:13.119 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-03-28T16:49:13.119 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-03-28T16:49:13.119 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-03-28T16:49:13.194 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:13.194 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:13.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7ce2907980 1 Processor -- start 2024-03-28T16:49:13.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7ce2907980 1 -- start start 2024-03-28T16:49:13.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7ce2907980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55704c162430 0x55704c164820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7ce2907980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55704c0aa510 con 0x55704c15df70 2024-03-28T16:49:13.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7ce2907980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55704c0710f0 con 0x55704c162430 2024-03-28T16:49:13.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7cdfc21640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55704c162430 0x55704c164820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7cdfc21640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55704c162430 0x55704c164820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55024/0 (socket says 172.21.15.170:55024) 2024-03-28T16:49:13.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7cdfc21640 1 -- 172.21.15.170:0/4092294186 learned_addr learned my addr 172.21.15.170:0/4092294186 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7ce0422640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55704c15df70 0x55704c15e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60812/0 (socket says 172.21.15.170:60812) 2024-03-28T16:49:13.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7cdfc21640 1 -- 172.21.15.170:0/4092294186 >> v1:172.21.15.119:6789/0 conn(0x55704c15df70 legacy=0x55704c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.195+0000 7f7cdfc21640 1 -- 172.21.15.170:0/4092294186 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704c165d70 con 0x55704c162430 2024-03-28T16:49:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/4092294186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c162430 0x55704c164820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400a5c0 tx=0x7f7cd401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fc33bf01636f60 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7cdf420640 1 -- 172.21.15.170:0/4092294186 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd40aa470 con 0x55704c162430 2024-03-28T16:49:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7cdf420640 1 -- 172.21.15.170:0/4092294186 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd40aa5d0 con 0x55704c162430 2024-03-28T16:49:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7cdf420640 1 -- 172.21.15.170:0/4092294186 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd40aa890 con 0x55704c162430 2024-03-28T16:49:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7ce2907980 1 -- 172.21.15.170:0/4092294186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c162430 msgr2=0x55704c164820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7ce2907980 1 --2- 172.21.15.170:0/4092294186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c162430 0x55704c164820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400a5c0 tx=0x7f7cd401b340 comp rx=0 tx=0).stop 2024-03-28T16:49:13.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7ce2907980 1 -- 172.21.15.170:0/4092294186 shutdown_connections 2024-03-28T16:49:13.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7ce2907980 1 --2- 172.21.15.170:0/4092294186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c162430 0x55704c164820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7ce2907980 1 -- 172.21.15.170:0/4092294186 >> 172.21.15.170:0/4092294186 conn(0x55704c159840 msgr2=0x55704c15bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7ce2907980 1 -- 172.21.15.170:0/4092294186 shutdown_connections 2024-03-28T16:49:13.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.199+0000 7f7ce2907980 1 -- 172.21.15.170:0/4092294186 wait complete. 2024-03-28T16:49:13.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.211+0000 7f7ce2907980 1 Processor -- start 2024-03-28T16:49:13.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.211+0000 7f7ce2907980 1 -- start start 2024-03-28T16:49:13.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.211+0000 7f7ce2907980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c247de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.211+0000 7f7ce2907980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c06e620 con 0x55704c15df70 2024-03-28T16:49:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c247de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c247de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55028/0 (socket says 172.21.15.170:55028) 2024-03-28T16:49:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 -- 172.21.15.170:0/1422869560 learned_addr learned my addr 172.21.15.170:0/1422869560 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 -- 172.21.15.170:0/1422869560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd4005df0 con 0x55704c15df70 2024-03-28T16:49:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 --2- 172.21.15.170:0/1422869560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c247de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8006fb0 tx=0x7f7cd8001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f15da0fcb1dc67 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7bce60c640 1 -- 172.21.15.170:0/1422869560 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8013420 con 0x55704c15df70 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7bce60c640 1 -- 172.21.15.170:0/1422869560 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8013580 con 0x55704c15df70 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7bce60c640 1 -- 172.21.15.170:0/1422869560 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8013860 con 0x55704c15df70 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- 172.21.15.170:0/1422869560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 msgr2=0x55704c247de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1422869560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c247de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8006fb0 tx=0x7f7cd8001940 comp rx=0 tx=0).stop 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- 172.21.15.170:0/1422869560 shutdown_connections 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1422869560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c247de0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- 172.21.15.170:0/1422869560 >> 172.21.15.170:0/1422869560 conn(0x55704c159840 msgr2=0x55704c243970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- 172.21.15.170:0/1422869560 shutdown_connections 2024-03-28T16:49:13.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- 172.21.15.170:0/1422869560 wait complete. 2024-03-28T16:49:13.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 Processor -- start 2024-03-28T16:49:13.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- start start 2024-03-28T16:49:13.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c0d4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c069e00 con 0x55704c15df70 2024-03-28T16:49:13.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c0d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c0d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55040/0 (socket says 172.21.15.170:55040) 2024-03-28T16:49:13.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 -- 172.21.15.170:0/1127122403 learned_addr learned my addr 172.21.15.170:0/1127122403 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704c0aa510 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce0422640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c0d4770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55704c248c70 tx=0x7f7cd8000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7bcce09640 1 -- 172.21.15.170:0/1127122403 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd80037a0 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7bcce09640 1 -- 172.21.15.170:0/1127122403 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8003900 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55704c0d4cb0 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7bcce09640 1 -- 172.21.15.170:0/1127122403 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8003ba0 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.215+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55704c0d3150 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7bcce09640 1 -- 172.21.15.170:0/1127122403 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7cd8003d00 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7bcce09640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bc8034300 0x7f7bc80367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7bcce09640 1 -- 172.21.15.170:0/1127122403 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7cd80493c0 con 0x55704c15df70 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c162430 0x55704c257b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2580b0 con 0x55704c162430 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bc8034300 0x7f7bc80367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce0c23640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c162430 0x55704c257b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce0c23640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c162430 0x55704c257b70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bc8034300 0x7f7bc80367c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40ae000 tx=0x7f7cd40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce0c23640 1 -- 172.21.15.170:0/1127122403 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c162430 2024-03-28T16:49:13.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c258410 con 0x55704c162430 2024-03-28T16:49:13.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce0c23640 1 -- 172.21.15.170:0/1127122403 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c162430 2024-03-28T16:49:13.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c259da0 0x55704c25c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c25c9a0 con 0x55704c259da0 2024-03-28T16:49:13.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce0422640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c259da0 0x55704c25c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.219+0000 7f7ce0422640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c259da0 0x55704c25c210 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce0422640 1 -- 172.21.15.170:0/1127122403 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f7cd8024020 con 0x55704c259da0 2024-03-28T16:49:13.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c258c10 con 0x55704c162430 2024-03-28T16:49:13.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce0c23640 1 -- 172.21.15.170:0/1127122403 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c162430 2024-03-28T16:49:13.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55704c25ddc0 0x55704c2601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c260870 con 0x55704c25ddc0 2024-03-28T16:49:13.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55704c25ddc0 0x55704c2601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55704c25ddc0 0x55704c2601b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7cdfc21640 1 -- 172.21.15.170:0/1127122403 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c25ddc0 2024-03-28T16:49:13.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 Processor -- start 2024-03-28T16:49:13.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 -- start start 2024-03-28T16:49:13.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c26e7a0 0x55704c26ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c002e30 con 0x55704c26e7a0 2024-03-28T16:49:13.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c26e7a0 0x55704c26ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c26e7a0 0x55704c26ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55048/0 (socket says 172.21.15.170:55048) 2024-03-28T16:49:13.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce0422640 1 -- 172.21.15.170:0/2443783249 learned_addr learned my addr 172.21.15.170:0/2443783249 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce0422640 1 -- 172.21.15.170:0/2443783249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704c26f100 con 0x55704c26e7a0 2024-03-28T16:49:13.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce0422640 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c26e7a0 0x55704c26ebc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8028530 tx=0x7f7cd8007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7bc5ffb640 1 -- 172.21.15.170:0/2443783249 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd803b070 con 0x55704c26e7a0 2024-03-28T16:49:13.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.223+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55704c26f350 con 0x55704c26e7a0 2024-03-28T16:49:13.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bc5ffb640 1 -- 172.21.15.170:0/2443783249 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd801de70 con 0x55704c26e7a0 2024-03-28T16:49:13.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55704c26fea0 con 0x55704c26e7a0 2024-03-28T16:49:13.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bc5ffb640 1 -- 172.21.15.170:0/2443783249 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8012b50 con 0x55704c26e7a0 2024-03-28T16:49:13.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bc5ffb640 1 -- 172.21.15.170:0/2443783249 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7cd8012cb0 con 0x55704c26e7a0 2024-03-28T16:49:13.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bc5ffb640 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb80342b0 0x7f7bb8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bc5ffb640 1 -- 172.21.15.170:0/2443783249 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7cd8035070 con 0x55704c26e7a0 2024-03-28T16:49:13.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb80342b0 0x7f7bb8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 Processor -- start 2024-03-28T16:49:13.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- start start 2024-03-28T16:49:13.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb80342b0 0x7f7bb8036770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55704c25cd30 tx=0x7f7cd4005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c27b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704bff6790 con 0x55704c27b4a0 2024-03-28T16:49:13.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c27b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c27b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55062/0 (socket says 172.21.15.170:55062) 2024-03-28T16:49:13.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce0422640 1 -- 172.21.15.170:0/1729934526 learned_addr learned my addr 172.21.15.170:0/1729934526 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce0422640 1 -- 172.21.15.170:0/1729934526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704c27be90 con 0x55704c27b4a0 2024-03-28T16:49:13.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce0422640 1 --2- 172.21.15.170:0/1729934526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c27b8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8012f80 tx=0x7f7cd8049cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21031a0c21785f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bbf7fe640 1 -- 172.21.15.170:0/1729934526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8058040 con 0x55704c27b4a0 2024-03-28T16:49:13.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bbf7fe640 1 -- 172.21.15.170:0/1729934526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8028640 con 0x55704c27b4a0 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7bbf7fe640 1 -- 172.21.15.170:0/1729934526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd80123c0 con 0x55704c27b4a0 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- 172.21.15.170:0/1729934526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 msgr2=0x55704c27b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1729934526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c27b8c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8012f80 tx=0x7f7cd8049cc0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- 172.21.15.170:0/1729934526 shutdown_connections 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1729934526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c27b8c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- 172.21.15.170:0/1729934526 >> 172.21.15.170:0/1729934526 conn(0x55704c277830 msgr2=0x55704c275890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- 172.21.15.170:0/1729934526 shutdown_connections 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.227+0000 7f7ce2907980 1 -- 172.21.15.170:0/1729934526 wait complete. 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 Processor -- start 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 -- start start 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c282390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704c2828d0 con 0x55704c27b4a0 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c282390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce0422640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c282390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55064/0 (socket says 172.21.15.170:55064) 2024-03-28T16:49:13.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 learned_addr learned my addr 172.21.15.170:0/823198015 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704c26f100 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce0422640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c282390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8012f80 tx=0x7f7cd8025f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7bbdffb640 1 -- 172.21.15.170:0/823198015 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8058070 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55704c26f100 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7bbdffb640 1 -- 172.21.15.170:0/823198015 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8007370 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7bbdffb640 1 -- 172.21.15.170:0/823198015 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd8007610 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55704c26f100 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7bbdffb640 1 -- 172.21.15.170:0/823198015 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7cd80113d0 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7bbdffb640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb00342b0 0x7f7bb0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7bbdffb640 1 -- 172.21.15.170:0/823198015 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7cd80623c0 con 0x55704c27b4a0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb00342b0 0x7f7bb0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c28d580 0x55704c28fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290170 con 0x55704c28d580 2024-03-28T16:49:13.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7ce0c23640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c28d580 0x55704c28fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.231+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb00342b0 0x7f7bb0036770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7cd401b380 tx=0x7f7cd40c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0c23640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c28d580 0x55704c28fa20 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 realm 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290540 con 0x55704c28d580 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2910f0 con 0x55704c28d580 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c291aa0 0x55704c293e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c294410 con 0x55704c291aa0 2024-03-28T16:49:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0422640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c291aa0 0x55704c293e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0422640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c291aa0 0x55704c293e90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 20 searching for the correct realm 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55704c296310 0x55704c298760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c298e30 con 0x55704c296310 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55704c296310 0x55704c298760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.235+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55704c296310 0x55704c298760 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7cdfc21640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7cd40e0040 0x7f7cd40d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c298e30 con 0x7f7cd40e0040 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7cd40e0040 0x7f7cd40d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7cd40e0040 0x7f7cd40d3400 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x7f7cd40e0040 2024-03-28T16:49:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.239+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x7f7cd40e0040 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.243+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x7f7cd40e0040 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x7f7cd40e0040 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c296310 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c291aa0 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x55704c28d580 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.247+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c290d90 con 0x7f7cd40e0040 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c294c80 con 0x55704c28d580 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c294c80 con 0x55704c28d580 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c294c50 con 0x7f7cd40e0040 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 zone default found 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 4 Realm: () 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 zonegroup default 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2954d0 con 0x55704c291aa0 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2958d0 con 0x55704c296310 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29ac50 con 0x55704c296310 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 started sync module instance, tier type = 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29a530 con 0x7f7cd40e0040 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93940802494608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29a8e0 con 0x7f7cd40e0040 2024-03-28T16:49:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.251+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29b9e0 con 0x55704c28d580 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93940802494608] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93940802498720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29b9c0 con 0x55704c28d580 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29c430 con 0x55704c296310 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93940802498720] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93940802502416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29c7e0 con 0x55704c296310 2024-03-28T16:49:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.255+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29d150 con 0x7f7cd40e0040 2024-03-28T16:49:13.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93940802506176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29d840 con 0x7f7cd40e0040 2024-03-28T16:49:13.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29e310 con 0x55704c28d580 2024-03-28T16:49:13.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93940802502416] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93940802506176] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93940802510320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29e2d0 con 0x55704c28d580 2024-03-28T16:49:13.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29d410 con 0x55704c291aa0 2024-03-28T16:49:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93940802510320] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93940802499776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29eb60 con 0x55704c291aa0 2024-03-28T16:49:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.259+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29f360 con 0x55704c291aa0 2024-03-28T16:49:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.263+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93940802499776] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.263+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.263+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93940802512576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c29fb60 con 0x55704c291aa0 2024-03-28T16:49:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.263+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2a04d0 con 0x55704c291aa0 2024-03-28T16:49:13.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.263+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93940802512576] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93940802519088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2a08f0 con 0x55704c291aa0 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=0 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=1 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=2 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=3 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=4 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=5 2024-03-28T16:49:13.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=6 2024-03-28T16:49:13.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93940802519088] v29'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 add_watcher() i=7 2024-03-28T16:49:13.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:13.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2a3cf0 con 0x55704c28d580 2024-03-28T16:49:13.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93940802533008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2a0a60 con 0x55704c28d580 2024-03-28T16:49:13.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93940802533008] v29'46 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7b967e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:13.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:13.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 5 note: GC not initialized 2024-03-28T16:49:13.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7b957e2640 20 reqs_thread_entry: start 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 20 Filter name: none 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7b06ffd640 20 reqs_thread_entry: start 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.267+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6220 con 0x55704c28d580 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c65e0 con 0x7f7cd40e0040 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c65e0 con 0x55704c28d580 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c65e0 con 0x55704c296310 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c296310 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c296310 2024-03-28T16:49:13.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c28d580 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c28d580 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c296310 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c296310 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c291aa0 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c28d580 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x7f7cd40e0040 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c296310 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.271+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c291aa0 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c291aa0 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c296310 2024-03-28T16:49:13.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c291aa0 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c28d580 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6940 con 0x55704c296310 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c69a0 con 0x55704c291aa0 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c69a0 con 0x7f7cd40e0040 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c291aa0 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.275+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c296310 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c296310 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c296310 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c296310 2024-03-28T16:49:13.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c291aa0 2024-03-28T16:49:13.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7b967e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93940802533008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c28d580 2024-03-28T16:49:13.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 93940802533008] v29'47 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.279+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93940802494608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x7f7cd40e0040 2024-03-28T16:49:13.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.283+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 93940802494608] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.283+0000 7f7ce2907980 20 remove_watcher() i=0 2024-03-28T16:49:13.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.283+0000 7f7ce2907980 2 removed watcher, disabling cache 2024-03-28T16:49:13.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.283+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93940802498720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c28d580 2024-03-28T16:49:13.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.283+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 93940802498720] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.283+0000 7f7ce2907980 20 remove_watcher() i=1 2024-03-28T16:49:13.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.283+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93940802502416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c296310 2024-03-28T16:49:13.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7cdfc21640 1 -- 172.21.15.170:0/823198015 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 93940802502416] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd40b0020 con 0x55704c296310 2024-03-28T16:49:13.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce2907980 20 remove_watcher() i=2 2024-03-28T16:49:13.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93940802506176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x7f7cd40e0040 2024-03-28T16:49:13.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 93940802506176] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x7f7cd40e0040 2024-03-28T16:49:13.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce2907980 20 remove_watcher() i=3 2024-03-28T16:49:13.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93940802510320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c28d580 2024-03-28T16:49:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 93940802510320] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd00083f0 con 0x55704c28d580 2024-03-28T16:49:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce2907980 20 remove_watcher() i=4 2024-03-28T16:49:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.287+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93940802499776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c291aa0 2024-03-28T16:49:13.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.291+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 93940802499776] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.291+0000 7f7ce2907980 20 remove_watcher() i=5 2024-03-28T16:49:13.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.291+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93940802512576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c291aa0 2024-03-28T16:49:13.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.291+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 93940802512576] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.291+0000 7f7ce2907980 20 remove_watcher() i=6 2024-03-28T16:49:13.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.291+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93940802519088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55704c2c6960 con 0x55704c291aa0 2024-03-28T16:49:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce0422640 1 -- 172.21.15.170:0/823198015 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 93940802519088] v29'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7cd8081040 con 0x55704c291aa0 2024-03-28T16:49:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 20 remove_watcher() i=7 2024-03-28T16:49:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c291aa0 msgr2=0x55704c293e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c291aa0 0x55704c293e90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c28d580 msgr2=0x55704c28fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c28d580 0x55704c28fa20 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7cd40e0040 msgr2=0x7f7cd40d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7cd40e0040 0x7f7cd40d3400 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55704c296310 msgr2=0x55704c298760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55704c296310 0x55704c298760 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb00342b0 msgr2=0x7f7bb0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb00342b0 0x7f7bb0036770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7cd401b380 tx=0x7f7cd40c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce0c23640 1 -- 172.21.15.170:0/823198015 reap_dead start 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 msgr2=0x55704c282390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c282390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8012f80 tx=0x7f7cd8025f00 comp rx=0 tx=0).stop 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 shutdown_connections 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/823198015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c27b4a0 0x55704c282390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 >> 172.21.15.170:0/823198015 conn(0x55704c277830 msgr2=0x55704c274e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 shutdown_connections 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/823198015 wait complete. 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb80342b0 msgr2=0x7f7bb8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb80342b0 0x7f7bb8036770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55704c25cd30 tx=0x7f7cd4005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c26e7a0 msgr2=0x55704c26ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c26e7a0 0x55704c26ebc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8028530 tx=0x7f7cd8007e40 comp rx=0 tx=0).stop 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 shutdown_connections 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bb80342b0 0x7f7bb8036770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/2443783249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c26e7a0 0x55704c26ebc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 >> 172.21.15.170:0/2443783249 conn(0x55704c2635b0 msgr2=0x55704c2640c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 shutdown_connections 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/2443783249 wait complete. 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c259da0 msgr2=0x55704c25c210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55704c259da0 0x55704c25c210 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c162430 msgr2=0x55704c257b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55704c162430 0x55704c257b70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55704c25ddc0 msgr2=0x55704c2601b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55704c25ddc0 0x55704c2601b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bc8034300 msgr2=0x7f7bc80367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7bc8034300 0x7f7bc80367c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40ae000 tx=0x7f7cd40a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 msgr2=0x55704c0d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 --2- 172.21.15.170:0/1127122403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55704c15df70 0x55704c0d4770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55704c248c70 tx=0x7f7cd8000f50 comp rx=0 tx=0).stop 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce0c23640 1 -- 172.21.15.170:0/1127122403 reap_dead start 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 shutdown_connections 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 >> 172.21.15.170:0/1127122403 conn(0x55704c159840 msgr2=0x55704c242480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 shutdown_connections 2024-03-28T16:49:13.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.295+0000 7f7ce2907980 1 -- 172.21.15.170:0/1127122403 wait complete. 2024-03-28T16:49:13.307 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[],"summary":[]} 2024-03-28T16:49:13.307 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-03-28T16:49:13.307 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:13.308 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:13.308 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:13.383 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:13.383 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f700abea980 1 Processor -- start 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f700abea980 1 -- start start 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f700abea980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5642b7f2f8b0 0x5642b7fbdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f700abea980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5642b7f07520 con 0x5642b7fbe170 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f700abea980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5642b7ece0f0 con 0x5642b7f2f8b0 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f7008705640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5642b7f2f8b0 0x5642b7fbdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f7008705640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5642b7f2f8b0 0x5642b7fbdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55068/0 (socket says 172.21.15.170:55068) 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f7008705640 1 -- 172.21.15.170:0/663035220 learned_addr learned my addr 172.21.15.170:0/663035220 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f7008705640 1 -- 172.21.15.170:0/663035220 >> v1:172.21.15.119:6789/0 conn(0x5642b7fbe170 legacy=0x5642b7fc0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.383+0000 7f7008705640 1 -- 172.21.15.170:0/663035220 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b7fc2cc0 con 0x5642b7f2f8b0 2024-03-28T16:49:13.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f7008705640 1 --2- 172.21.15.170:0/663035220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b7fbdc30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f700000a4e0 tx=0x7f700001b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60b5f74f1253ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f7007703640 1 -- 172.21.15.170:0/663035220 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000aa540 con 0x5642b7f2f8b0 2024-03-28T16:49:13.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f7007703640 1 -- 172.21.15.170:0/663035220 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000a6070 con 0x5642b7f2f8b0 2024-03-28T16:49:13.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f7007703640 1 -- 172.21.15.170:0/663035220 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000aa850 con 0x5642b7f2f8b0 2024-03-28T16:49:13.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f700abea980 1 -- 172.21.15.170:0/663035220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 msgr2=0x5642b7fbdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f700abea980 1 --2- 172.21.15.170:0/663035220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b7fbdc30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f700000a4e0 tx=0x7f700001b090 comp rx=0 tx=0).stop 2024-03-28T16:49:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f700abea980 1 -- 172.21.15.170:0/663035220 shutdown_connections 2024-03-28T16:49:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f700abea980 1 --2- 172.21.15.170:0/663035220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b7fbdc30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f700abea980 1 -- 172.21.15.170:0/663035220 >> 172.21.15.170:0/663035220 conn(0x5642b7fb5710 msgr2=0x5642b7fb7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f700abea980 1 -- 172.21.15.170:0/663035220 shutdown_connections 2024-03-28T16:49:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.387+0000 7f700abea980 1 -- 172.21.15.170:0/663035220 wait complete. 2024-03-28T16:49:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 Processor -- start 2024-03-28T16:49:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 -- start start 2024-03-28T16:49:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b7ecb620 con 0x5642b7f2f8b0 2024-03-28T16:49:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80a4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55074/0 (socket says 172.21.15.170:55074) 2024-03-28T16:49:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f7008705640 1 -- 172.21.15.170:0/2650653754 learned_addr learned my addr 172.21.15.170:0/2650653754 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f7008705640 1 -- 172.21.15.170:0/2650653754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700000a1c0 con 0x5642b7f2f8b0 2024-03-28T16:49:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f7008705640 1 --2- 172.21.15.170:0/2650653754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80a4eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7000018cc0 tx=0x7f70000033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e1dd16a56ec3f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f6ef660c640 1 -- 172.21.15.170:0/2650653754 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000aa700 con 0x5642b7f2f8b0 2024-03-28T16:49:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f6ef660c640 1 -- 172.21.15.170:0/2650653754 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000aa860 con 0x5642b7f2f8b0 2024-03-28T16:49:13.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f6ef660c640 1 -- 172.21.15.170:0/2650653754 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000aab20 con 0x5642b7f2f8b0 2024-03-28T16:49:13.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 -- 172.21.15.170:0/2650653754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 msgr2=0x5642b80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 --2- 172.21.15.170:0/2650653754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80a4eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7000018cc0 tx=0x7f70000033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 -- 172.21.15.170:0/2650653754 shutdown_connections 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 --2- 172.21.15.170:0/2650653754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80a4eb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 -- 172.21.15.170:0/2650653754 >> 172.21.15.170:0/2650653754 conn(0x5642b7fb5710 msgr2=0x5642b80a0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 -- 172.21.15.170:0/2650653754 shutdown_connections 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 -- 172.21.15.170:0/2650653754 wait complete. 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.403+0000 7f700abea980 1 Processor -- start 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f700abea980 1 -- start start 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f700abea980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f700abea980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b7ec6e00 con 0x5642b7f2f8b0 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55082/0 (socket says 172.21.15.170:55082) 2024-03-28T16:49:13.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008705640 1 -- 172.21.15.170:0/2807692491 learned_addr learned my addr 172.21.15.170:0/2807692491 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008705640 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b7fc2cc0 con 0x5642b7f2f8b0 2024-03-28T16:49:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008705640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80abb90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7000018cc0 tx=0x7f7000017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f6ef4e09640 1 -- 172.21.15.170:0/2807692491 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000c4070 con 0x5642b7f2f8b0 2024-03-28T16:49:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5642b80ac0d0 con 0x5642b7f2f8b0 2024-03-28T16:49:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f6ef4e09640 1 -- 172.21.15.170:0/2807692491 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000b5840 con 0x5642b7f2f8b0 2024-03-28T16:49:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f6ef4e09640 1 -- 172.21.15.170:0/2807692491 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000b5ae0 con 0x5642b7f2f8b0 2024-03-28T16:49:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642b80ac500 con 0x5642b7f2f8b0 2024-03-28T16:49:13.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f6ef4e09640 1 -- 172.21.15.170:0/2807692491 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f70000a8070 con 0x5642b7f2f8b0 2024-03-28T16:49:13.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f6ef4e09640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ef00342b0 0x7f6ef0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f6ef4e09640 1 -- 172.21.15.170:0/2807692491 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f70000e3530 con 0x5642b7f2f8b0 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7007f04640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ef00342b0 0x7f6ef0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b7fbe170 0x5642b80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80b2d50 con 0x5642b7fbe170 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008f06640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b7fbe170 0x5642b80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7007f04640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ef00342b0 0x7f6ef0036770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc000f70 tx=0x7f6ffc001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008f06640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b7fbe170 0x5642b80b2810 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.407+0000 7f7008f06640 1 -- 172.21.15.170:0/2807692491 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b7fbe170 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80b30b0 con 0x5642b7fbe170 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7008f06640 1 -- 172.21.15.170:0/2807692491 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b7fbe170 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80b4a40 0x5642b80b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80b75a0 con 0x5642b80b4a40 2024-03-28T16:49:13.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7008705640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80b4a40 0x5642b80b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7008705640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80b4a40 0x5642b80b6e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7008705640 1 -- 172.21.15.170:0/2807692491 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80b4a40 2024-03-28T16:49:13.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80b38b0 con 0x5642b7fbe170 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7008f06640 1 -- 172.21.15.170:0/2807692491 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b7fbe170 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5642b80b8980 0x5642b80bad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80bb410 con 0x5642b80b8980 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7007f04640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5642b80b8980 0x5642b80bad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7007f04640 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5642b80b8980 0x5642b80bad70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f7007f04640 1 -- 172.21.15.170:0/2807692491 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f6ffc0106b0 con 0x5642b80b8980 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 Processor -- start 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 -- start start 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b80c9320 0x5642b80c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.411+0000 7f700abea980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b7e5fe30 con 0x5642b80c9320 2024-03-28T16:49:13.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b80c9320 0x5642b80c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b80c9320 0x5642b80c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55096/0 (socket says 172.21.15.170:55096) 2024-03-28T16:49:13.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f7008705640 1 -- 172.21.15.170:0/3421586369 learned_addr learned my addr 172.21.15.170:0/3421586369 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f7008705640 1 -- 172.21.15.170:0/3421586369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b80c9c80 con 0x5642b80c9320 2024-03-28T16:49:13.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f7008705640 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b80c9320 0x5642b80c9740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70000033d0 tx=0x7f70000a9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f6eedffb640 1 -- 172.21.15.170:0/3421586369 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000e04f0 con 0x5642b80c9320 2024-03-28T16:49:13.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f6eedffb640 1 -- 172.21.15.170:0/3421586369 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000dc040 con 0x5642b80c9320 2024-03-28T16:49:13.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f6eedffb640 1 -- 172.21.15.170:0/3421586369 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000e0780 con 0x5642b80c9320 2024-03-28T16:49:13.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5642b80c9ed0 con 0x5642b80c9320 2024-03-28T16:49:13.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642b80caa20 con 0x5642b80c9320 2024-03-28T16:49:13.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f6eedffb640 1 -- 172.21.15.170:0/3421586369 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f70000b43a0 con 0x5642b80c9320 2024-03-28T16:49:13.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f6eedffb640 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ee0034260 0x7f6ee0036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f6eedffb640 1 -- 172.21.15.170:0/3421586369 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f70000eeaf0 con 0x5642b80c9320 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f7007f04640 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ee0034260 0x7f6ee0036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f700abea980 1 Processor -- start 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f700abea980 1 -- start start 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f7007f04640 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ee0034260 0x7f6ee0036720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5642b80b3c10 tx=0x7f6ffc006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f700abea980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b7fbb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.415+0000 7f700abea980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b7e53790 con 0x5642b7fbb1e0 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b7fbb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b7fbb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55112/0 (socket says 172.21.15.170:55112) 2024-03-28T16:49:13.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 -- 172.21.15.170:0/976504430 learned_addr learned my addr 172.21.15.170:0/976504430 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 -- 172.21.15.170:0/976504430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b7fb9a10 con 0x5642b7fbb1e0 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 --2- 172.21.15.170:0/976504430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b7fbb5c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f70000b37b0 tx=0x7f70000d3880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbb3d4edc828b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f6edf7fe640 1 -- 172.21.15.170:0/976504430 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000c9cd0 con 0x5642b7fbb1e0 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f6edf7fe640 1 -- 172.21.15.170:0/976504430 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000c9e30 con 0x5642b7fbb1e0 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f6edf7fe640 1 -- 172.21.15.170:0/976504430 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000d6c60 con 0x5642b7fbb1e0 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- 172.21.15.170:0/976504430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 msgr2=0x5642b7fbb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 --2- 172.21.15.170:0/976504430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b7fbb5c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f70000b37b0 tx=0x7f70000d3880 comp rx=0 tx=0).stop 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- 172.21.15.170:0/976504430 shutdown_connections 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 --2- 172.21.15.170:0/976504430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b7fbb5c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- 172.21.15.170:0/976504430 >> 172.21.15.170:0/976504430 conn(0x5642b80d25e0 msgr2=0x5642b80d0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- 172.21.15.170:0/976504430 shutdown_connections 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- 172.21.15.170:0/976504430 wait complete. 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 Processor -- start 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- start start 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b80df2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642b80df7e0 con 0x5642b7fbb1e0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b80df2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b80df2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55118/0 (socket says 172.21.15.170:55118) 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 learned_addr learned my addr 172.21.15.170:0/4078015727 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642b80caa20 con 0x5642b7fbb1e0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f7008705640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b80df2a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f70000d3b90 tx=0x7f70000b9fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f6eddffb640 1 -- 172.21.15.170:0/4078015727 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000c9cd0 con 0x5642b7fbb1e0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5642b80df9e0 con 0x5642b7fbb1e0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f6eddffb640 1 -- 172.21.15.170:0/4078015727 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70000d66a0 con 0x5642b7fbb1e0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f6eddffb640 1 -- 172.21.15.170:0/4078015727 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70000d6940 con 0x5642b7fbb1e0 2024-03-28T16:49:13.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.419+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642b80dfe10 con 0x5642b7fbb1e0 2024-03-28T16:49:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f6eddffb640 1 -- 172.21.15.170:0/4078015727 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f70000c9420 con 0x5642b7fbb1e0 2024-03-28T16:49:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f6eddffb640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ed0034260 0x7f6ed0036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f6eddffb640 1 -- 172.21.15.170:0/4078015727 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7000105b50 con 0x5642b7fbb1e0 2024-03-28T16:49:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7007f04640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ed0034260 0x7f6ed0036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b80ea680 0x5642b80ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80ed290 con 0x5642b80ea680 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008f06640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b80ea680 0x5642b80ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7007f04640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ed0034260 0x7f6ed0036720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5642b80bb7b0 tx=0x7f6ffc011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008f06640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b80ea680 0x5642b80ecb40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 realm 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80ed660 con 0x5642b80ea680 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:13.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80ee210 con 0x5642b80ea680 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80eebc0 0x5642b80f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f1570 con 0x5642b80eebc0 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008705640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80eebc0 0x5642b80f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.423+0000 7f7008705640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80eebc0 0x5642b80f0fb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f700abea980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f700abea980 20 searching for the correct realm 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5642b80f3400 0x5642b80f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f5f60 con 0x5642b80f3400 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7007f04640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5642b80f3400 0x5642b80f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7007f04640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5642b80f3400 0x5642b80f5890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7007f04640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6ffc00f3d0 0x7f6ffc00f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f5f60 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6ffc00f3d0 0x7f6ffc00f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6ffc00f3d0 0x7f6ffc00f810 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.427+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.431+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.435+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80f3400 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80eebc0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x5642b80ea680 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80edeb0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:13.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f1d70 con 0x5642b80ea680 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f1d70 con 0x5642b80ea680 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f1d70 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 zone default found 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 4 Realm: () 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 zonegroup default 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f25c0 con 0x5642b80eebc0 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f29c0 con 0x5642b80f3400 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f7e20 con 0x5642b80f3400 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 started sync module instance, tier type = 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.439+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f7660 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.443+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.443+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94844555850176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f7a10 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.443+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f8b80 con 0x5642b80ea680 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.443+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94844555850176] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.443+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.443+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94844555854400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f8ba0 con 0x5642b80ea680 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.443+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f9650 con 0x5642b80f3400 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94844555854400] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94844555858256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f9a30 con 0x5642b80f3400 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fa3a0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94844555862032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80faa90 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fb540 con 0x5642b80ea680 2024-03-28T16:49:13.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94844555858256] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.447+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94844555862032] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.451+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.451+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94844555866176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fb500 con 0x5642b80ea680 2024-03-28T16:49:13.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.451+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fb920 con 0x5642b80eebc0 2024-03-28T16:49:13.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.451+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94844555866176] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.451+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.451+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94844555855456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fbcd0 con 0x5642b80eebc0 2024-03-28T16:49:13.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.451+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fc640 con 0x5642b80eebc0 2024-03-28T16:49:13.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94844555855456] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94844555863280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fc640 con 0x5642b80eebc0 2024-03-28T16:49:13.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fcff0 con 0x5642b80eebc0 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94844555863280] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94844555873040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fd1d0 con 0x5642b80eebc0 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 20 add_watcher() i=0 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 20 add_watcher() i=1 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 20 add_watcher() i=2 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 20 add_watcher() i=3 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 20 add_watcher() i=4 2024-03-28T16:49:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 20 add_watcher() i=5 2024-03-28T16:49:13.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.455+0000 7f700abea980 20 add_watcher() i=6 2024-03-28T16:49:13.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94844555873040] v29'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 20 add_watcher() i=7 2024-03-28T16:49:13.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:13.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b8100590 con 0x5642b80ea680 2024-03-28T16:49:13.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94844555874496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fd9b0 con 0x5642b80ea680 2024-03-28T16:49:13.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94844555874496] v29'48 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f6ebe7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:13.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 5 note: GC not initialized 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f6ebd7e2640 20 reqs_thread_entry: start 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 20 Filter name: none 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f6e2effd640 20 reqs_thread_entry: start 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b8121fa0 con 0x5642b80ea680 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.461 INFO:teuthology.orchestra.run.smithi170.stderr:could not fetch user info: no user info saved 2024-03-28T16:49:13.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f6ebe7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.459+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94844555874496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b8121f60 con 0x5642b80ea680 2024-03-28T16:49:13.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94844555874496] v29'49 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94844555850176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b8121f60 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94844555850176] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f700abea980 20 remove_watcher() i=0 2024-03-28T16:49:13.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f700abea980 2 removed watcher, disabling cache 2024-03-28T16:49:13.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94844555854400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b8121f60 con 0x5642b80ea680 2024-03-28T16:49:13.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94844555854400] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f700abea980 20 remove_watcher() i=1 2024-03-28T16:49:13.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.463+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94844555858256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b8121f60 con 0x5642b80f3400 2024-03-28T16:49:13.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.467+0000 7f7007f04640 1 -- 172.21.15.170:0/4078015727 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94844555858256] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ffc032040 con 0x5642b80f3400 2024-03-28T16:49:13.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.467+0000 7f700abea980 20 remove_watcher() i=2 2024-03-28T16:49:13.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.467+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94844555862032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f71c0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.467+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94844555862032] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x7f6ffc00f3d0 2024-03-28T16:49:13.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.467+0000 7f700abea980 20 remove_watcher() i=3 2024-03-28T16:49:13.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.467+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94844555866176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f8240 con 0x5642b80ea680 2024-03-28T16:49:13.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.471+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94844555866176] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6ff80083f0 con 0x5642b80ea680 2024-03-28T16:49:13.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.471+0000 7f700abea980 20 remove_watcher() i=4 2024-03-28T16:49:13.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.471+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94844555855456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fa010 con 0x5642b80eebc0 2024-03-28T16:49:13.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.471+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94844555855456] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.471+0000 7f700abea980 20 remove_watcher() i=5 2024-03-28T16:49:13.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.471+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94844555863280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80f71c0 con 0x5642b80eebc0 2024-03-28T16:49:13.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94844555863280] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 20 remove_watcher() i=6 2024-03-28T16:49:13.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94844555873040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5642b80fa010 con 0x5642b80eebc0 2024-03-28T16:49:13.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f7008705640 1 -- 172.21.15.170:0/4078015727 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94844555873040] v29'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70000b49e0 con 0x5642b80eebc0 2024-03-28T16:49:13.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 20 remove_watcher() i=7 2024-03-28T16:49:13.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80eebc0 msgr2=0x5642b80f0fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80eebc0 0x5642b80f0fb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b80ea680 msgr2=0x5642b80ecb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b80ea680 0x5642b80ecb40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6ffc00f3d0 msgr2=0x7f6ffc00f810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6ffc00f3d0 0x7f6ffc00f810 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5642b80f3400 msgr2=0x5642b80f5890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5642b80f3400 0x5642b80f5890 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ed0034260 msgr2=0x7f6ed0036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ed0034260 0x7f6ed0036720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5642b80bb7b0 tx=0x7f6ffc011000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 msgr2=0x5642b80df2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f7008f06640 1 -- 172.21.15.170:0/4078015727 reap_dead start 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b80df2a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f70000d3b90 tx=0x7f70000b9fa0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 shutdown_connections 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/4078015727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7fbb1e0 0x5642b80df2a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 >> 172.21.15.170:0/4078015727 conn(0x5642b80d25e0 msgr2=0x5642b80cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 shutdown_connections 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/4078015727 wait complete. 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ee0034260 msgr2=0x7f6ee0036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ee0034260 0x7f6ee0036720 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5642b80b3c10 tx=0x7f6ffc006000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b80c9320 msgr2=0x5642b80c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b80c9320 0x5642b80c9740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f70000033d0 tx=0x7f70000a9da0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 shutdown_connections 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ee0034260 0x7f6ee0036720 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/3421586369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b80c9320 0x5642b80c9740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 >> 172.21.15.170:0/3421586369 conn(0x5642b80be130 msgr2=0x5642b80bec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 shutdown_connections 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/3421586369 wait complete. 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80b4a40 msgr2=0x5642b80b6e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5642b80b4a40 0x5642b80b6e30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b7fbe170 msgr2=0x5642b80b2810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5642b7fbe170 0x5642b80b2810 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5642b80b8980 msgr2=0x5642b80bad70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.475+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5642b80b8980 0x5642b80bad70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ef00342b0 msgr2=0x7f6ef0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ef00342b0 0x7f6ef0036770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc000f70 tx=0x7f6ffc001420 comp rx=0 tx=0).stop 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 msgr2=0x5642b80abb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 --2- 172.21.15.170:0/2807692491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5642b7f2f8b0 0x5642b80abb90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7000018cc0 tx=0x7f7000017e30 comp rx=0 tx=0).stop 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f7008f06640 1 -- 172.21.15.170:0/2807692491 reap_dead start 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 shutdown_connections 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 >> 172.21.15.170:0/2807692491 conn(0x5642b7fb5710 msgr2=0x5642b809f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 shutdown_connections 2024-03-28T16:49:13.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.479+0000 7f700abea980 1 -- 172.21.15.170:0/2807692491 wait complete. 2024-03-28T16:49:13.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-03-28T16:49:13.487 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-03-28T16:49:13.487 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-03-28T16:49:13.487 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-03-28T16:49:13.572 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:13.572 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.571+0000 7fe2bdf87980 1 Processor -- start 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.571+0000 7fe2bdf87980 1 -- start start 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.571+0000 7fe2bdf87980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56241d215850 0x56241d2a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.571+0000 7fe2bdf87980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56241d1ed4c0 con 0x56241d2a6510 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bdf87980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56241d1b40f0 con 0x56241d215850 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bbaa2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56241d215850 0x56241d2a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bbaa2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56241d215850 0x56241d2a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55122/0 (socket says 172.21.15.170:55122) 2024-03-28T16:49:13.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/2644925077 learned_addr learned my addr 172.21.15.170:0/2644925077 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/2644925077 >> v1:172.21.15.119:6789/0 conn(0x56241d2a6510 legacy=0x56241d2a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/2644925077 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56241d2a19f0 con 0x56241d215850 2024-03-28T16:49:13.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/2644925077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d2a5fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00a5c0 tx=0x7fe2ac01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63236777c62fbcfa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bb2a1640 1 -- 172.21.15.170:0/2644925077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0aa470 con 0x56241d215850 2024-03-28T16:49:13.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bb2a1640 1 -- 172.21.15.170:0/2644925077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0aa5d0 con 0x56241d215850 2024-03-28T16:49:13.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.575+0000 7fe2bb2a1640 1 -- 172.21.15.170:0/2644925077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0aa890 con 0x56241d215850 2024-03-28T16:49:13.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.579+0000 7fe2bdf87980 1 -- 172.21.15.170:0/2644925077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 msgr2=0x56241d2a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.579+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/2644925077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d2a5fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00a5c0 tx=0x7fe2ac01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:13.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.579+0000 7fe2bdf87980 1 -- 172.21.15.170:0/2644925077 shutdown_connections 2024-03-28T16:49:13.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.579+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/2644925077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d2a5fd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.579+0000 7fe2bdf87980 1 -- 172.21.15.170:0/2644925077 >> 172.21.15.170:0/2644925077 conn(0x56241d29b6b0 msgr2=0x56241d29dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.579+0000 7fe2bdf87980 1 -- 172.21.15.170:0/2644925077 shutdown_connections 2024-03-28T16:49:13.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.579+0000 7fe2bdf87980 1 -- 172.21.15.170:0/2644925077 wait complete. 2024-03-28T16:49:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 Processor -- start 2024-03-28T16:49:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- start start 2024-03-28T16:49:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d38d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56241d1b1620 con 0x56241d215850 2024-03-28T16:49:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d38d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d38d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55136/0 (socket says 172.21.15.170:55136) 2024-03-28T16:49:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/1234844645 learned_addr learned my addr 172.21.15.170:0/1234844645 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/1234844645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac005df0 con 0x56241d215850 2024-03-28T16:49:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/1234844645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d38d120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac018f70 tx=0x7fe2ac003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7457c90b8d42bf68 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe1a9e0b640 1 -- 172.21.15.170:0/1234844645 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0aa9c0 con 0x56241d215850 2024-03-28T16:49:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe1a9e0b640 1 -- 172.21.15.170:0/1234844645 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0aab20 con 0x56241d215850 2024-03-28T16:49:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe1a9e0b640 1 -- 172.21.15.170:0/1234844645 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0aadc0 con 0x56241d215850 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1234844645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 msgr2=0x56241d38d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/1234844645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d38d120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac018f70 tx=0x7fe2ac003030 comp rx=0 tx=0).stop 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1234844645 shutdown_connections 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/1234844645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d38d120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1234844645 >> 172.21.15.170:0/1234844645 conn(0x56241d29b6b0 msgr2=0x56241d388cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1234844645 shutdown_connections 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1234844645 wait complete. 2024-03-28T16:49:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 Processor -- start 2024-03-28T16:49:13.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.591+0000 7fe2bdf87980 1 -- start start 2024-03-28T16:49:13.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bdf87980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d393e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bdf87980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56241d1ace00 con 0x56241d215850 2024-03-28T16:49:13.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d393e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d393e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55138/0 (socket says 172.21.15.170:55138) 2024-03-28T16:49:13.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/146049968 learned_addr learned my addr 172.21.15.170:0/146049968 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56241d2a19f0 con 0x56241d215850 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d393e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac018f70 tx=0x7fe2ac0180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe1a3fff640 1 -- 172.21.15.170:0/146049968 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0cb070 con 0x56241d215850 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56241d3943b0 con 0x56241d215850 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe1a3fff640 1 -- 172.21.15.170:0/146049968 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0bd810 con 0x56241d215850 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe1a3fff640 1 -- 172.21.15.170:0/146049968 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0bdab0 con 0x56241d215850 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56241d394780 con 0x56241d215850 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe1a3fff640 1 -- 172.21.15.170:0/146049968 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe2ac0bdce0 con 0x56241d215850 2024-03-28T16:49:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe1a3fff640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1a40342b0 0x7fe1a4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe1a3fff640 1 -- 172.21.15.170:0/146049968 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe2ac0e48f0 con 0x56241d215850 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1a40342b0 0x7fe1a4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d2a6510 0x56241d39aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d39afe0 con 0x56241d2a6510 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bc2a3640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d2a6510 0x56241d39aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1a40342b0 0x7fe1a4036770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4001e20 tx=0x7fe2a40013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bc2a3640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d2a6510 0x56241d39aaa0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/146049968 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d2a6510 2024-03-28T16:49:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.595+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d39b340 con 0x56241d2a6510 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/146049968 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d2a6510 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d39ccd0 0x56241d39f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d39f830 con 0x56241d39ccd0 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d39ccd0 0x56241d39f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d39ccd0 0x56241d39f0c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/146049968 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fe2ac0cd070 con 0x56241d39ccd0 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d39bb40 con 0x56241d2a6510 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/146049968 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d2a6510 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56241d3a0c30 0x56241d3a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3a36c0 con 0x56241d3a0c30 2024-03-28T16:49:13.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56241d3a0c30 0x56241d3a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56241d3a0c30 0x56241d3a3020 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2b3fff640 1 -- 172.21.15.170:0/146049968 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fe2a4010630 con 0x56241d3a0c30 2024-03-28T16:49:13.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 Processor -- start 2024-03-28T16:49:13.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 -- start start 2024-03-28T16:49:13.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3b15f0 0x56241d3b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bdf87980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56241d145e30 con 0x56241d3b15f0 2024-03-28T16:49:13.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3b15f0 0x56241d3b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3b15f0 0x56241d3b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55142/0 (socket says 172.21.15.170:55142) 2024-03-28T16:49:13.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.599+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/4101610546 learned_addr learned my addr 172.21.15.170:0/4101610546 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/4101610546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56241d3b1f50 con 0x56241d3b15f0 2024-03-28T16:49:13.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3b15f0 0x56241d3b1a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0cf940 tx=0x7fe2ac0e5e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe1a17fa640 1 -- 172.21.15.170:0/4101610546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0c5e50 con 0x56241d3b15f0 2024-03-28T16:49:13.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe1a17fa640 1 -- 172.21.15.170:0/4101610546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0e4250 con 0x56241d3b15f0 2024-03-28T16:49:13.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56241d3b21a0 con 0x56241d3b15f0 2024-03-28T16:49:13.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe1a17fa640 1 -- 172.21.15.170:0/4101610546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0e44f0 con 0x56241d3b15f0 2024-03-28T16:49:13.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56241d3b2cf0 con 0x56241d3b15f0 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe1a17fa640 1 -- 172.21.15.170:0/4101610546 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe2ac0cf9b0 con 0x56241d3b15f0 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe1a17fa640 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1940342b0 0x7fe194036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe1a17fa640 1 -- 172.21.15.170:0/4101610546 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe2ac0f7aa0 con 0x56241d3b15f0 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1940342b0 0x7fe194036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bdf87980 1 Processor -- start 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bdf87980 1 -- start start 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bdf87980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bdf87980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56241d139790 con 0x56241d3be570 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1940342b0 0x7fe194036770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x56241d39bea0 tx=0x7fe2a4006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3be950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55150/0 (socket says 172.21.15.170:55150) 2024-03-28T16:49:13.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/1754961467 learned_addr learned my addr 172.21.15.170:0/1754961467 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.603+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/1754961467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56241d3bef20 con 0x56241d3be570 2024-03-28T16:49:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/1754961467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3be950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0bcf50 tx=0x7fe2ac0d3d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7b392f47e390d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe19affd640 1 -- 172.21.15.170:0/1754961467 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac107070 con 0x56241d3be570 2024-03-28T16:49:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe19affd640 1 -- 172.21.15.170:0/1754961467 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0c6480 con 0x56241d3be570 2024-03-28T16:49:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe19affd640 1 -- 172.21.15.170:0/1754961467 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0a7290 con 0x56241d3be570 2024-03-28T16:49:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1754961467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 msgr2=0x56241d3be950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/1754961467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3be950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0bcf50 tx=0x7fe2ac0d3d80 comp rx=0 tx=0).stop 2024-03-28T16:49:13.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1754961467 shutdown_connections 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/1754961467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3be950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1754961467 >> 172.21.15.170:0/1754961467 conn(0x56241d3ba8b0 msgr2=0x56241d3b8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1754961467 shutdown_connections 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- 172.21.15.170:0/1754961467 wait complete. 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 Processor -- start 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- start start 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56241d3c5890 con 0x56241d3be570 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bbaa2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55156/0 (socket says 172.21.15.170:55156) 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 learned_addr learned my addr 172.21.15.170:0/812443188 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56241d3b2cf0 con 0x56241d3be570 2024-03-28T16:49:13.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3c5350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0c5a80 tx=0x7fe2ac0c0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe1997fa640 1 -- 172.21.15.170:0/812443188 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac107040 con 0x56241d3be570 2024-03-28T16:49:13.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56241d3b2cf0 con 0x56241d3be570 2024-03-28T16:49:13.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe1997fa640 1 -- 172.21.15.170:0/812443188 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac0c65f0 con 0x56241d3be570 2024-03-28T16:49:13.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe1997fa640 1 -- 172.21.15.170:0/812443188 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0bc970 con 0x56241d3be570 2024-03-28T16:49:13.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56241d3b2cf0 con 0x56241d3be570 2024-03-28T16:49:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.607+0000 7fe1997fa640 1 -- 172.21.15.170:0/812443188 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe2ac0a7660 con 0x56241d3be570 2024-03-28T16:49:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe1997fa640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1840342b0 0x7fe184036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe1997fa640 1 -- 172.21.15.170:0/812443188 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe2ac118ac0 con 0x56241d3be570 2024-03-28T16:49:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1840342b0 0x7fe184036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d3d0550 0x56241d3d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3160 con 0x56241d3d0550 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bc2a3640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d3d0550 0x56241d3d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1840342b0 0x7fe184036770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x56241d3a3a80 tx=0x7fe2a4011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bc2a3640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d3d0550 0x56241d3d2a10 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 realm 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3530 con 0x56241d3d0550 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d40e0 con 0x56241d3d0550 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d3d4a90 0x56241d3d6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d74d0 con 0x56241d3d4a90 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d3d4a90 0x56241d3d6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bbaa2640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d3d4a90 0x56241d3d6e80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 20 searching for the correct realm 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56241d3d96b0 0x56241d3dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3dc240 con 0x56241d3d96b0 2024-03-28T16:49:13.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56241d3d96b0 0x56241d3dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.611+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56241d3d96b0 0x56241d3dbb70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2b3fff640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe2a400f3d0 0x7fe2a400f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3dc240 con 0x7fe2a400f3d0 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe2a400f3d0 0x7fe2a400f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe2a400f3d0 0x7fe2a400f820 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.615+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x7fe2a400f3d0 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x7fe2a400f3d0 2024-03-28T16:49:13.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.619+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x7fe2a400f3d0 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x7fe2a400f3d0 2024-03-28T16:49:13.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d96b0 2024-03-28T16:49:13.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d4a90 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.623+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x56241d3d0550 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d3d80 con 0x7fe2a400f3d0 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:13.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d7ab0 con 0x56241d3d0550 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d7ab0 con 0x56241d3d0550 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d7ab0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 zone default found 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 4 Realm: () 2024-03-28T16:49:13.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 zonegroup default 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d8300 con 0x56241d3d4a90 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3d8730 con 0x56241d3d96b0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3de1a0 con 0x56241d3d96b0 2024-03-28T16:49:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 started sync module instance, tier type = 2024-03-28T16:49:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3dd940 con 0x7fe2a400f3d0 2024-03-28T16:49:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94713109402784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3ddcf0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.627+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3def00 con 0x56241d3d0550 2024-03-28T16:49:13.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.631+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94713109402784] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.631+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.631+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94713109407168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3def20 con 0x56241d3d0550 2024-03-28T16:49:13.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.631+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3df9f0 con 0x56241d3d96b0 2024-03-28T16:49:13.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.631+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94713109407168] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94713109411024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3dfda0 con 0x56241d3d96b0 2024-03-28T16:49:13.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e0710 con 0x7fe2a400f3d0 2024-03-28T16:49:13.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94713109414784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e0de0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e1890 con 0x56241d3d0550 2024-03-28T16:49:13.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94713109411024] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94713109414784] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94713109418896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e1850 con 0x56241d3d0550 2024-03-28T16:49:13.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.635+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e1c70 con 0x56241d3d4a90 2024-03-28T16:49:13.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94713109418896] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94713109416000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e2020 con 0x56241d3d4a90 2024-03-28T16:49:13.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e2990 con 0x56241d3d4a90 2024-03-28T16:49:13.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94713109423344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e2db0 con 0x56241d3d4a90 2024-03-28T16:49:13.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e3850 con 0x56241d3d4a90 2024-03-28T16:49:13.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.639+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94713109416000] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94713109423344] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94713109426992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e2910 con 0x56241d3d4a90 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=0 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=1 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=2 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=3 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=4 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=5 2024-03-28T16:49:13.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=6 2024-03-28T16:49:13.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94713109426992] v29'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 20 add_watcher() i=7 2024-03-28T16:49:13.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:13.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e6950 con 0x56241d3d0550 2024-03-28T16:49:13.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.643+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94713109408224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e3dc0 con 0x56241d3d0550 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94713109408224] v29'50 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe171fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 5 note: GC not initialized 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe170fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 Filter name: none 2024-03-28T16:49:13.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe0ea7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d408780 con 0x56241d3d0550 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d408780 con 0x56241d3d0550 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 197+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-03-28T16:49:13.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-03-28T16:49:13.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-03-28T16:49:13.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d406fa0 con 0x56241d3d4a90 2024-03-28T16:49:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d407840 con 0x56241d3d0550 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~357] v29'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94713109574560 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d4077c0 con 0x56241d3d0550 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== watch-notify(notify (1) cookie 94713109418896 notify 124554051584 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7fe2b4043c30 con 0x56241d3d0550 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe19affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051584 cookie 94713109418896 notifier 4284 bl.length()=533 2024-03-28T16:49:13.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe19affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe19affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe19affd640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fe180003ea0 con 0x56241d3d0550 2024-03-28T16:49:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.647+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(66 notify.4 [notify cookie 94713109574560 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94713109574560 notify 124554051584 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fe2b4043e90 con 0x56241d3d0550 2024-03-28T16:49:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d409cc0 con 0x56241d3d0550 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 239+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bdf87980 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bdf87980 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bdf87980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94713109574560 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d40c180 con 0x7fe2a400f3d0 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== watch-notify(notify (1) cookie 94713109414784 notify 124554051584 ret 0) v3 ==== 211+0+0 (crc 0 0 0) 0x7fe2b4043e90 con 0x7fe2a400f3d0 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe19a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051584 cookie 94713109414784 notifier 4284 bl.length()=169 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe19a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-03-28T16:49:13.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe19a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-03-28T16:49:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(69 notify.3 [notify cookie 94713109574560 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe19a7fc640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fe0f0003400 con 0x7fe2a400f3d0 2024-03-28T16:49:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== watch-notify(notify_complete (2) cookie 94713109574560 notify 124554051584 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fe2b4018740 con 0x7fe2a400f3d0 2024-03-28T16:49:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.651+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d409cc0 con 0x56241d3d4a90 2024-03-28T16:49:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bdf87980 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-03-28T16:49:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bdf87980 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-03-28T16:49:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bdf87980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-03-28T16:49:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94713109574560 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d409cc0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== watch-notify(notify (1) cookie 94713109414784 notify 124554051585 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7fe2b4018740 con 0x7fe2a400f3d0 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe19a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051585 cookie 94713109414784 notifier 4284 bl.length()=177 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe19a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe19a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(72 notify.3 [notify cookie 94713109574560 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe19a7fc640 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fe0f0003430 con 0x7fe2a400f3d0 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== watch-notify(notify_complete (2) cookie 94713109574560 notify 124554051585 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fe2b4018bb0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.656 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe171fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94713109408224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e8fd0 con 0x56241d3d0550 2024-03-28T16:49:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94713109408224] v29'51 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.655+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94713109402784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3de9e0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.659+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94713109402784] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.659+0000 7fe2bdf87980 20 remove_watcher() i=0 2024-03-28T16:49:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.659+0000 7fe2bdf87980 2 removed watcher, disabling cache 2024-03-28T16:49:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.659+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94713109407168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3de9e0 con 0x56241d3d0550 2024-03-28T16:49:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.659+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94713109407168] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.659+0000 7fe2bdf87980 20 remove_watcher() i=1 2024-03-28T16:49:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.659+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94713109411024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3dea60 con 0x56241d3d96b0 2024-03-28T16:49:13.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2b3fff640 1 -- 172.21.15.170:0/812443188 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94713109411024] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2a4032040 con 0x56241d3d96b0 2024-03-28T16:49:13.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bdf87980 20 remove_watcher() i=2 2024-03-28T16:49:13.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94713109414784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3df4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94713109414784] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x7fe2a400f3d0 2024-03-28T16:49:13.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bdf87980 20 remove_watcher() i=3 2024-03-28T16:49:13.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94713109418896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e0380 con 0x56241d3d0550 2024-03-28T16:49:13.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94713109418896] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2b400a4d0 con 0x56241d3d0550 2024-03-28T16:49:13.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bdf87980 20 remove_watcher() i=4 2024-03-28T16:49:13.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.663+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94713109416000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e1390 con 0x56241d3d4a90 2024-03-28T16:49:13.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.667+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94713109416000] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.667+0000 7fe2bdf87980 20 remove_watcher() i=5 2024-03-28T16:49:13.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.667+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94713109423344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e0380 con 0x56241d3d4a90 2024-03-28T16:49:13.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.667+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94713109423344] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.667+0000 7fe2bdf87980 20 remove_watcher() i=6 2024-03-28T16:49:13.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.667+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94713109426992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56241d3e24f0 con 0x56241d3d4a90 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bbaa2640 1 -- 172.21.15.170:0/812443188 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94713109426992] v29'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe2ac114070 con 0x56241d3d4a90 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 20 remove_watcher() i=7 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d3d4a90 msgr2=0x56241d3d6e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d3d4a90 0x56241d3d6e80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d3d0550 msgr2=0x56241d3d2a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d3d0550 0x56241d3d2a10 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe2a400f3d0 msgr2=0x7fe2a400f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe2a400f3d0 0x7fe2a400f820 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56241d3d96b0 msgr2=0x56241d3dbb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56241d3d96b0 0x56241d3dbb70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1840342b0 msgr2=0x7fe184036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1840342b0 0x7fe184036770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x56241d3a3a80 tx=0x7fe2a4011000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/812443188 reap_dead start 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 msgr2=0x56241d3c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3c5350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0c5a80 tx=0x7fe2ac0c0180 comp rx=0 tx=0).stop 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 shutdown_connections 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/812443188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3be570 0x56241d3c5350 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 >> 172.21.15.170:0/812443188 conn(0x56241d3ba8b0 msgr2=0x56241d3b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 shutdown_connections 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/812443188 wait complete. 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1940342b0 msgr2=0x7fe194036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1940342b0 0x7fe194036770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x56241d39bea0 tx=0x7fe2a4006000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3b15f0 msgr2=0x56241d3b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3b15f0 0x56241d3b1a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0cf940 tx=0x7fe2ac0e5e50 comp rx=0 tx=0).stop 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 shutdown_connections 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1940342b0 0x7fe194036770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/4101610546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d3b15f0 0x56241d3b1a10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 >> 172.21.15.170:0/4101610546 conn(0x56241d3a6400 msgr2=0x56241d3a6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 shutdown_connections 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/4101610546 wait complete. 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d39ccd0 msgr2=0x56241d39f0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56241d39ccd0 0x56241d39f0c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d2a6510 msgr2=0x56241d39aaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56241d2a6510 0x56241d39aaa0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56241d3a0c30 msgr2=0x56241d3a3020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56241d3a0c30 0x56241d3a3020 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1a40342b0 msgr2=0x7fe1a4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe1a40342b0 0x7fe1a4036770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4001e20 tx=0x7fe2a40013a0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 msgr2=0x56241d393e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 --2- 172.21.15.170:0/146049968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56241d215850 0x56241d393e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac018f70 tx=0x7fe2ac0180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bc2a3640 1 -- 172.21.15.170:0/146049968 reap_dead start 2024-03-28T16:49:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 shutdown_connections 2024-03-28T16:49:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 >> 172.21.15.170:0/146049968 conn(0x56241d29b6b0 msgr2=0x56241d3877c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 shutdown_connections 2024-03-28T16:49:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.671+0000 7fe2bdf87980 1 -- 172.21.15.170:0/146049968 wait complete. 2024-03-28T16:49:13.681 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:13.681 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-03-28T16:49:13.681 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-03-28T16:49:13.681 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-03-28T16:49:13.751 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:13.752 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:13.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbceb3ca980 1 Processor -- start 2024-03-28T16:49:13.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbceb3ca980 1 -- start start 2024-03-28T16:49:13.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbceb3ca980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5557cd634f20 0x5557cd635300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbceb3ca980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5557cd5814c0 con 0x5557cd635840 2024-03-28T16:49:13.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbceb3ca980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5557cd5480f0 con 0x5557cd634f20 2024-03-28T16:49:13.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbce8ee5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5557cd634f20 0x5557cd635300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbce8ee5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5557cd634f20 0x5557cd635300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55158/0 (socket says 172.21.15.170:55158) 2024-03-28T16:49:13.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.751+0000 7fbce8ee5640 1 -- 172.21.15.170:0/548996571 learned_addr learned my addr 172.21.15.170:0/548996571 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbce8ee5640 1 -- 172.21.15.170:0/548996571 >> v1:172.21.15.119:6789/0 conn(0x5557cd635840 legacy=0x5557cd6361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbce8ee5640 1 -- 172.21.15.170:0/548996571 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd6369e0 con 0x5557cd634f20 2024-03-28T16:49:13.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/548996571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd635300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800a5c0 tx=0x7fbcd801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bf38366f0b2dc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbce7ee3640 1 -- 172.21.15.170:0/548996571 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80aa470 con 0x5557cd634f20 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbce7ee3640 1 -- 172.21.15.170:0/548996571 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80aa5d0 con 0x5557cd634f20 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbce7ee3640 1 -- 172.21.15.170:0/548996571 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80aa890 con 0x5557cd634f20 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbceb3ca980 1 -- 172.21.15.170:0/548996571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 msgr2=0x5557cd635300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/548996571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd635300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800a5c0 tx=0x7fbcd801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbceb3ca980 1 -- 172.21.15.170:0/548996571 shutdown_connections 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/548996571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd635300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbceb3ca980 1 -- 172.21.15.170:0/548996571 >> 172.21.15.170:0/548996571 conn(0x5557cd6307f0 msgr2=0x5557cd632c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbceb3ca980 1 -- 172.21.15.170:0/548996571 shutdown_connections 2024-03-28T16:49:13.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.755+0000 7fbceb3ca980 1 -- 172.21.15.170:0/548996571 wait complete. 2024-03-28T16:49:13.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 Processor -- start 2024-03-28T16:49:13.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- start start 2024-03-28T16:49:13.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd71f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd545620 con 0x5557cd634f20 2024-03-28T16:49:13.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd71f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd71f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55168/0 (socket says 172.21.15.170:55168) 2024-03-28T16:49:13.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 -- 172.21.15.170:0/1342577110 learned_addr learned my addr 172.21.15.170:0/1342577110 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 -- 172.21.15.170:0/1342577110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8005df0 con 0x5557cd634f20 2024-03-28T16:49:13.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/1342577110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd71f340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80ac040 tx=0x7fbcd8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7436910983ff88b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbbd6e0d640 1 -- 172.21.15.170:0/1342577110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80aa9b0 con 0x5557cd634f20 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbbd6e0d640 1 -- 172.21.15.170:0/1342577110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80aab30 con 0x5557cd634f20 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbbd6e0d640 1 -- 172.21.15.170:0/1342577110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80aae10 con 0x5557cd634f20 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1342577110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 msgr2=0x5557cd71f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/1342577110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd71f340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80ac040 tx=0x7fbcd8003030 comp rx=0 tx=0).stop 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1342577110 shutdown_connections 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/1342577110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd71f340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1342577110 >> 172.21.15.170:0/1342577110 conn(0x5557cd6307f0 msgr2=0x5557cd71aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1342577110 shutdown_connections 2024-03-28T16:49:13.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1342577110 wait complete. 2024-03-28T16:49:13.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 Processor -- start 2024-03-28T16:49:13.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- start start 2024-03-28T16:49:13.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd728340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbceb3ca980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd540e00 con 0x5557cd634f20 2024-03-28T16:49:13.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd728340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd728340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55182/0 (socket says 172.21.15.170:55182) 2024-03-28T16:49:13.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.771+0000 7fbce8ee5640 1 -- 172.21.15.170:0/4255324631 learned_addr learned my addr 172.21.15.170:0/4255324631 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce8ee5640 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd6369e0 con 0x5557cd634f20 2024-03-28T16:49:13.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd728340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8005940 tx=0x7fbcd80180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbbd560a640 1 -- 172.21.15.170:0/4255324631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80c6690 con 0x5557cd634f20 2024-03-28T16:49:13.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5557cd728880 con 0x5557cd634f20 2024-03-28T16:49:13.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbbd560a640 1 -- 172.21.15.170:0/4255324631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80c67f0 con 0x5557cd634f20 2024-03-28T16:49:13.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbbd560a640 1 -- 172.21.15.170:0/4255324631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80c6ab0 con 0x5557cd634f20 2024-03-28T16:49:13.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557cd728cb0 con 0x5557cd634f20 2024-03-28T16:49:13.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbbd560a640 1 -- 172.21.15.170:0/4255324631 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbcd80c6c10 con 0x5557cd634f20 2024-03-28T16:49:13.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbbd560a640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbd00342b0 0x7fbbd0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbbd560a640 1 -- 172.21.15.170:0/4255324631 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbcd80e3fe0 con 0x5557cd634f20 2024-03-28T16:49:13.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce86e4640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbd00342b0 0x7fbbd0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd635840 0x5557cd72efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd72f500 con 0x5557cd635840 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce96e6640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd635840 0x5557cd72efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce86e4640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbd00342b0 0x7fbbd0036770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc001e20 tx=0x7fbcdc0013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce96e6640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd635840 0x5557cd72efc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce96e6640 1 -- 172.21.15.170:0/4255324631 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd635840 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd72f860 con 0x5557cd635840 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbce96e6640 1 -- 172.21.15.170:0/4255324631 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd635840 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd731580 0x5557cd733970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.775+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd733eb0 con 0x5557cd731580 2024-03-28T16:49:13.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd731580 0x5557cd733970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd731580 0x5557cd733970 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce8ee5640 1 -- 172.21.15.170:0/4255324631 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd731580 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd7303c0 con 0x5557cd635840 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce96e6640 1 -- 172.21.15.170:0/4255324631 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd635840 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5557cd735270 0x5557cd737660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd737d00 con 0x5557cd735270 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce86e4640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5557cd735270 0x5557cd737660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce86e4640 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5557cd735270 0x5557cd737660 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce86e4640 1 -- 172.21.15.170:0/4255324631 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fbcdc010630 con 0x5557cd735270 2024-03-28T16:49:13.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbceb3ca980 1 Processor -- start 2024-03-28T16:49:13.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbceb3ca980 1 -- start start 2024-03-28T16:49:13.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbceb3ca980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd745cd0 0x5557cd7460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbceb3ca980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd4d9e30 con 0x5557cd745cd0 2024-03-28T16:49:13.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd745cd0 0x5557cd7460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd745cd0 0x5557cd7460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55196/0 (socket says 172.21.15.170:55196) 2024-03-28T16:49:13.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.779+0000 7fbce8ee5640 1 -- 172.21.15.170:0/1779822841 learned_addr learned my addr 172.21.15.170:0/1779822841 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce8ee5640 1 -- 172.21.15.170:0/1779822841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd746630 con 0x5557cd745cd0 2024-03-28T16:49:13.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd745cd0 0x5557cd7460f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80e2560 tx=0x7fbcd80c5bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbce7fc640 1 -- 172.21.15.170:0/1779822841 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80aca80 con 0x5557cd745cd0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbce7fc640 1 -- 172.21.15.170:0/1779822841 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80e0040 con 0x5557cd745cd0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5557cd746f90 con 0x5557cd745cd0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbce7fc640 1 -- 172.21.15.170:0/1779822841 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80acd30 con 0x5557cd745cd0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557cd746f90 con 0x5557cd745cd0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbce7fc640 1 -- 172.21.15.170:0/1779822841 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbcd80b33f0 con 0x5557cd745cd0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbce7fc640 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbc00342b0 0x7fbbc0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbce7fc640 1 -- 172.21.15.170:0/1779822841 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbcd80b50b0 con 0x5557cd745cd0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce86e4640 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbc00342b0 0x7fbbc0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbceb3ca980 1 Processor -- start 2024-03-28T16:49:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbceb3ca980 1 -- start start 2024-03-28T16:49:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce86e4640 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbc00342b0 0x7fbbc0036770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5557cd730720 tx=0x7fbcdc006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbceb3ca980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd752e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbceb3ca980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd4cd790 con 0x5557cd752a30 2024-03-28T16:49:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd752e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd752e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55206/0 (socket says 172.21.15.170:55206) 2024-03-28T16:49:13.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce8ee5640 1 -- 172.21.15.170:0/4058324407 learned_addr learned my addr 172.21.15.170:0/4058324407 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce8ee5640 1 -- 172.21.15.170:0/4058324407 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd753420 con 0x5557cd752a30 2024-03-28T16:49:13.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/4058324407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd752e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80d1e50 tx=0x7fbcd80e2960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b28b449ae075f7f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbbffff640 1 -- 172.21.15.170:0/4058324407 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80bc5d0 con 0x5557cd752a30 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbbffff640 1 -- 172.21.15.170:0/4058324407 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80a4e60 con 0x5557cd752a30 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.783+0000 7fbbbffff640 1 -- 172.21.15.170:0/4058324407 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80c53e0 con 0x5557cd752a30 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4058324407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 msgr2=0x5557cd752e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4058324407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd752e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80d1e50 tx=0x7fbcd80e2960 comp rx=0 tx=0).stop 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4058324407 shutdown_connections 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4058324407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd752e50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4058324407 >> 172.21.15.170:0/4058324407 conn(0x5557cd74edc0 msgr2=0x5557cd74ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4058324407 shutdown_connections 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4058324407 wait complete. 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 Processor -- start 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- start start 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd7598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557cd759e10 con 0x5557cd752a30 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd7598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbce8ee5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd7598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55222/0 (socket says 172.21.15.170:55222) 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 learned_addr learned my addr 172.21.15.170:0/3842110948 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557cd746630 con 0x5557cd752a30 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd7598d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80eb040 tx=0x7fbcd80afed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbbbe7fc640 1 -- 172.21.15.170:0/3842110948 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80b1820 con 0x5557cd752a30 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5557cd746630 con 0x5557cd752a30 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbbbe7fc640 1 -- 172.21.15.170:0/3842110948 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80bc5d0 con 0x5557cd752a30 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbbbe7fc640 1 -- 172.21.15.170:0/3842110948 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80b37d0 con 0x5557cd752a30 2024-03-28T16:49:13.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557cd746630 con 0x5557cd752a30 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbbbe7fc640 1 -- 172.21.15.170:0/3842110948 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbcd80efc80 con 0x5557cd752a30 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbbbe7fc640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbb40342b0 0x7fbbb4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbbbe7fc640 1 -- 172.21.15.170:0/3842110948 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbcd8119af0 con 0x5557cd752a30 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbce86e4640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbb40342b0 0x7fbbb4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd764b10 0x5557cd766fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd767720 con 0x5557cd764b10 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.787+0000 7fbce96e6640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd764b10 0x5557cd766fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce96e6640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd764b10 0x5557cd766fd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce86e4640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbb40342b0 0x7fbbb4036770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5557cd7380c0 tx=0x7fbcdc011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 realm 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd767af0 con 0x5557cd764b10 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd7686d0 con 0x5557cd764b10 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd769080 0x5557cd76b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd76ba80 con 0x5557cd769080 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd769080 0x5557cd76b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce8ee5640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd769080 0x5557cd76b470 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 20 searching for the correct realm 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5557cd76dc50 0x5557cd770110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd7707e0 con 0x5557cd76dc50 2024-03-28T16:49:13.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce86e4640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5557cd76dc50 0x5557cd770110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.791+0000 7fbce86e4640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5557cd76dc50 0x5557cd770110 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce86e4640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbcdc00f3d0 0x7fbcdc00f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd7707e0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbcdc00f3d0 0x7fbcdc00f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbcdc00f3d0 0x7fbcdc00f820 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.795+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.799+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd76dc50 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd769080 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x5557cd764b10 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.803+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd768370 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:13.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd76c030 con 0x5557cd764b10 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd76c030 con 0x5557cd764b10 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd76c030 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 zone default found 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 4 Realm: () 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 zonegroup default 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd76c880 con 0x5557cd769080 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd76cc80 con 0x5557cd76dc50 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd772730 con 0x5557cd76dc50 2024-03-28T16:49:13.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 started sync module instance, tier type = 2024-03-28T16:49:13.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:13.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd771ee0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93835597650496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd772290 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.807+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd773490 con 0x5557cd764b10 2024-03-28T16:49:13.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93835597650496] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93835597654864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd773410 con 0x5557cd764b10 2024-03-28T16:49:13.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd773ec0 con 0x5557cd76dc50 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93835597654864] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93835597655920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd773390 con 0x5557cd76dc50 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.811+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd774bb0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93835597662240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd775400 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd775dd0 con 0x5557cd764b10 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93835597655920] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93835597662240] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93835597666512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd774ee0 con 0x5557cd764b10 2024-03-28T16:49:13.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.815+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd776620 con 0x5557cd769080 2024-03-28T16:49:13.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93835597666512] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93835597668736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd774ee0 con 0x5557cd769080 2024-03-28T16:49:13.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd776e20 con 0x5557cd769080 2024-03-28T16:49:13.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93835597668736] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93835597660784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd777620 con 0x5557cd769080 2024-03-28T16:49:13.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.819+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd777f90 con 0x5557cd769080 2024-03-28T16:49:13.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93835597675248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd7783b0 con 0x5557cd769080 2024-03-28T16:49:13.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=0 2024-03-28T16:49:13.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=1 2024-03-28T16:49:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=2 2024-03-28T16:49:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=3 2024-03-28T16:49:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=4 2024-03-28T16:49:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=5 2024-03-28T16:49:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93835597660784] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=6 2024-03-28T16:49:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93835597675248] v29'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 20 add_watcher() i=7 2024-03-28T16:49:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd777180 con 0x5557cd764b10 2024-03-28T16:49:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.823+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93835597676304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd777180 con 0x5557cd764b10 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93835597676304] v29'52 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbb9f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 5 note: GC not initialized 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbb9e7e4640 20 reqs_thread_entry: start 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 Filter name: none 2024-03-28T16:49:13.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbb977fe640 20 reqs_thread_entry: start 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd79d370 con 0x5557cd769080 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd79d370 con 0x5557cd764b10 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+23 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rados_obj.operate() r=0 bl.length=7 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:13.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd79bc20 con 0x5557cd764b10 2024-03-28T16:49:13.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 foo [call out=48b,read 0~357 out=357b] v0'0 uv1 ondisk = 0) v8 ==== 189+0+405 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:13.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x11 2024-03-28T16:49:13.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:13.830 INFO:teuthology.orchestra.run.smithi170.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-03-28T16:49:13.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbb9f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.827+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93835597676304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd79c7f0 con 0x5557cd764b10 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93835597676304] v29'53 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93835597650496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd79c350 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93835597650496] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbceb3ca980 20 remove_watcher() i=0 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbceb3ca980 2 removed watcher, disabling cache 2024-03-28T16:49:13.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93835597654864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd771a40 con 0x5557cd764b10 2024-03-28T16:49:13.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93835597654864] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbceb3ca980 20 remove_watcher() i=1 2024-03-28T16:49:13.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.831+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93835597655920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd771a40 con 0x5557cd76dc50 2024-03-28T16:49:13.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.835+0000 7fbce86e4640 1 -- 172.21.15.170:0/3842110948 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93835597655920] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcdc032040 con 0x5557cd76dc50 2024-03-28T16:49:13.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.835+0000 7fbceb3ca980 20 remove_watcher() i=2 2024-03-28T16:49:13.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.835+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93835597662240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd772b50 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.835+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93835597662240] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x7fbcdc00f3d0 2024-03-28T16:49:13.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.835+0000 7fbceb3ca980 20 remove_watcher() i=3 2024-03-28T16:49:13.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.835+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93835597666512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd774820 con 0x5557cd764b10 2024-03-28T16:49:13.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.839+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93835597666512] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbce000a4d0 con 0x5557cd764b10 2024-03-28T16:49:13.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.839+0000 7fbceb3ca980 20 remove_watcher() i=4 2024-03-28T16:49:13.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.839+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93835597668736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd774820 con 0x5557cd769080 2024-03-28T16:49:13.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.839+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93835597668736] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.839+0000 7fbceb3ca980 20 remove_watcher() i=5 2024-03-28T16:49:13.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.839+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93835597660784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd776180 con 0x5557cd769080 2024-03-28T16:49:13.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93835597660784] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 20 remove_watcher() i=6 2024-03-28T16:49:13.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93835597675248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5557cd774820 con 0x5557cd769080 2024-03-28T16:49:13.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbce8ee5640 1 -- 172.21.15.170:0/3842110948 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93835597675248] v29'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbcd80b0020 con 0x5557cd769080 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 20 remove_watcher() i=7 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd769080 msgr2=0x5557cd76b470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd769080 0x5557cd76b470 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd764b10 msgr2=0x5557cd766fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd764b10 0x5557cd766fd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbcdc00f3d0 msgr2=0x7fbcdc00f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbcdc00f3d0 0x7fbcdc00f820 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5557cd76dc50 msgr2=0x5557cd770110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5557cd76dc50 0x5557cd770110 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbb40342b0 msgr2=0x7fbbb4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbb40342b0 0x7fbbb4036770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5557cd7380c0 tx=0x7fbcdc011000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 msgr2=0x5557cd7598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/3842110948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd752a30 0x5557cd7598d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80eb040 tx=0x7fbcd80afed0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 reap_dead start 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbce96e6640 1 -- 172.21.15.170:0/3842110948 reap_dead start 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 shutdown_connections 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 >> 172.21.15.170:0/3842110948 conn(0x5557cd74edc0 msgr2=0x5557cd74c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 shutdown_connections 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/3842110948 wait complete. 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbc00342b0 msgr2=0x7fbbc0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbc00342b0 0x7fbbc0036770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5557cd730720 tx=0x7fbcdc006000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd745cd0 msgr2=0x5557cd7460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd745cd0 0x5557cd7460f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80e2560 tx=0x7fbcd80c5bb0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 shutdown_connections 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbc00342b0 0x7fbbc0036770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/1779822841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd745cd0 0x5557cd7460f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 >> 172.21.15.170:0/1779822841 conn(0x5557cd73aa40 msgr2=0x5557cd73b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 shutdown_connections 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/1779822841 wait complete. 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd731580 msgr2=0x5557cd733970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5557cd731580 0x5557cd733970 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd635840 msgr2=0x5557cd72efc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5557cd635840 0x5557cd72efc0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5557cd735270 msgr2=0x5557cd737660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5557cd735270 0x5557cd737660 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbd00342b0 msgr2=0x7fbbd0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbbd00342b0 0x7fbbd0036770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc001e20 tx=0x7fbcdc0013a0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 msgr2=0x5557cd728340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbceb3ca980 1 --2- 172.21.15.170:0/4255324631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5557cd634f20 0x5557cd728340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8005940 tx=0x7fbcd80180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.843+0000 7fbce96e6640 1 -- 172.21.15.170:0/4255324631 reap_dead start 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.847+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 shutdown_connections 2024-03-28T16:49:13.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.847+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 >> 172.21.15.170:0/4255324631 conn(0x5557cd6307f0 msgr2=0x5557cd7199e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.847+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 shutdown_connections 2024-03-28T16:49:13.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.847+0000 7fbceb3ca980 1 -- 172.21.15.170:0/4255324631 wait complete. 2024-03-28T16:49:13.855 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-03-28T16:49:13.855 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:13.855 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:13.856 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:13.930 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:13.931 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:13.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4da24980 1 Processor -- start 2024-03-28T16:49:13.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4da24980 1 -- start start 2024-03-28T16:49:13.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4da24980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x563575f5a0b0 0x563575fe8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4da24980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563575f32520 con 0x563575fe90f0 2024-03-28T16:49:13.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4da24980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x563575ef90f0 con 0x563575f5a0b0 2024-03-28T16:49:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4b53f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x563575f5a0b0 0x563575fe8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4b53f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x563575f5a0b0 0x563575fe8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55226/0 (socket says 172.21.15.170:55226) 2024-03-28T16:49:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4b53f640 1 -- 172.21.15.170:0/4226397576 learned_addr learned my addr 172.21.15.170:0/4226397576 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4ad3e640 1 --1- 172.21.15.170:0/4226397576 >> v1:172.21.15.119:6789/0 conn(0x563575fe90f0 0x563575feb4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60854/0 (socket says 172.21.15.170:60854) 2024-03-28T16:49:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4b53f640 1 -- 172.21.15.170:0/4226397576 >> v1:172.21.15.119:6789/0 conn(0x563575fe90f0 legacy=0x563575feb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.931+0000 7fae4b53f640 1 -- 172.21.15.170:0/4226397576 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563575fedcd0 con 0x563575f5a0b0 2024-03-28T16:49:13.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4b53f640 1 --2- 172.21.15.170:0/4226397576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x563575fe8bb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00a4e0 tx=0x7fae3c01b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34467b952dcad32 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4a53d640 1 -- 172.21.15.170:0/4226397576 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0aa540 con 0x563575f5a0b0 2024-03-28T16:49:13.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4a53d640 1 -- 172.21.15.170:0/4226397576 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c0a6070 con 0x563575f5a0b0 2024-03-28T16:49:13.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4a53d640 1 -- 172.21.15.170:0/4226397576 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0aa850 con 0x563575f5a0b0 2024-03-28T16:49:13.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4da24980 1 -- 172.21.15.170:0/4226397576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 msgr2=0x563575fe8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4da24980 1 --2- 172.21.15.170:0/4226397576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x563575fe8bb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00a4e0 tx=0x7fae3c01b090 comp rx=0 tx=0).stop 2024-03-28T16:49:13.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4da24980 1 -- 172.21.15.170:0/4226397576 shutdown_connections 2024-03-28T16:49:13.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4da24980 1 --2- 172.21.15.170:0/4226397576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x563575fe8bb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4da24980 1 -- 172.21.15.170:0/4226397576 >> 172.21.15.170:0/4226397576 conn(0x563575fe06d0 msgr2=0x563575fe2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4da24980 1 -- 172.21.15.170:0/4226397576 shutdown_connections 2024-03-28T16:49:13.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.935+0000 7fae4da24980 1 -- 172.21.15.170:0/4226397576 wait complete. 2024-03-28T16:49:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4da24980 1 Processor -- start 2024-03-28T16:49:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4da24980 1 -- start start 2024-03-28T16:49:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4da24980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4da24980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563575ef6620 con 0x563575f5a0b0 2024-03-28T16:49:13.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55240/0 (socket says 172.21.15.170:55240) 2024-03-28T16:49:13.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4b53f640 1 -- 172.21.15.170:0/247746912 learned_addr learned my addr 172.21.15.170:0/247746912 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4b53f640 1 -- 172.21.15.170:0/247746912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c00a1c0 con 0x563575f5a0b0 2024-03-28T16:49:13.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fae4b53f640 1 --2- 172.21.15.170:0/247746912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760cfe60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae3c017ca0 tx=0x7fae3c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a7781a42e002f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fad3960a640 1 -- 172.21.15.170:0/247746912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0aa780 con 0x563575f5a0b0 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fad3960a640 1 -- 172.21.15.170:0/247746912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c0aa8e0 con 0x563575f5a0b0 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.951+0000 7fad3960a640 1 -- 172.21.15.170:0/247746912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0aab80 con 0x563575f5a0b0 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- 172.21.15.170:0/247746912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 msgr2=0x5635760cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 --2- 172.21.15.170:0/247746912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760cfe60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae3c017ca0 tx=0x7fae3c0033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- 172.21.15.170:0/247746912 shutdown_connections 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 --2- 172.21.15.170:0/247746912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760cfe60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- 172.21.15.170:0/247746912 >> 172.21.15.170:0/247746912 conn(0x563575fe06d0 msgr2=0x5635760cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- 172.21.15.170:0/247746912 shutdown_connections 2024-03-28T16:49:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- 172.21.15.170:0/247746912 wait complete. 2024-03-28T16:49:13.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 Processor -- start 2024-03-28T16:49:13.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- start start 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563575ef1e00 con 0x563575f5a0b0 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760d6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55242/0 (socket says 172.21.15.170:55242) 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4b53f640 1 -- 172.21.15.170:0/3144209514 learned_addr learned my addr 172.21.15.170:0/3144209514 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4b53f640 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563575fedcd0 con 0x563575f5a0b0 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4b53f640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760d6bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae3c01aec0 tx=0x7fae3c017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fad37e07640 1 -- 172.21.15.170:0/3144209514 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0c4070 con 0x563575f5a0b0 2024-03-28T16:49:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fad37e07640 1 -- 172.21.15.170:0/3144209514 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c005150 con 0x563575f5a0b0 2024-03-28T16:49:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5635760d70f0 con 0x563575f5a0b0 2024-03-28T16:49:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fad37e07640 1 -- 172.21.15.170:0/3144209514 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0053f0 con 0x563575f5a0b0 2024-03-28T16:49:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635760d7520 con 0x563575f5a0b0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fad37e07640 1 -- 172.21.15.170:0/3144209514 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fae3c0a8070 con 0x563575f5a0b0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fad37e07640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad300342b0 0x7fad30036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fad37e07640 1 -- 172.21.15.170:0/3144209514 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fae3c0e3190 con 0x563575f5a0b0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.955+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad300342b0 0x7fad30036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563575fe90f0 0x5635760dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635760ddd20 con 0x563575fe90f0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4bd40640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563575fe90f0 0x5635760dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad300342b0 0x7fad30036770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae40001e50 tx=0x7fae400013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4bd40640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563575fe90f0 0x5635760dd7e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4bd40640 1 -- 172.21.15.170:0/3144209514 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563575fe90f0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635760de080 con 0x563575fe90f0 2024-03-28T16:49:13.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4bd40640 1 -- 172.21.15.170:0/3144209514 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fae440083f0 con 0x563575fe90f0 2024-03-28T16:49:13.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635760dfda0 0x5635760e2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635760e26d0 con 0x5635760dfda0 2024-03-28T16:49:13.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4b53f640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635760dfda0 0x5635760e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4b53f640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635760dfda0 0x5635760e2190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4b53f640 1 -- 172.21.15.170:0/3144209514 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fae3c0a97e0 con 0x5635760dfda0 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635760debb0 con 0x563575fe90f0 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4bd40640 1 -- 172.21.15.170:0/3144209514 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fae440083f0 con 0x563575fe90f0 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635760e3a40 0x5635760e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635760e6480 con 0x5635760e3a40 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635760e3a40 0x5635760e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.959+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635760e3a40 0x5635760e5e30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3144209514 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fae4000f700 con 0x5635760e3a40 2024-03-28T16:49:13.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4da24980 1 Processor -- start 2024-03-28T16:49:13.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4da24980 1 -- start start 2024-03-28T16:49:13.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4da24980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635760f43b0 0x5635760f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4da24980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563575e8ae30 con 0x5635760f43b0 2024-03-28T16:49:13.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635760f43b0 0x5635760f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635760f43b0 0x5635760f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55250/0 (socket says 172.21.15.170:55250) 2024-03-28T16:49:13.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4b53f640 1 -- 172.21.15.170:0/2783534653 learned_addr learned my addr 172.21.15.170:0/2783534653 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4b53f640 1 -- 172.21.15.170:0/2783534653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635760f4d10 con 0x5635760f43b0 2024-03-28T16:49:13.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4b53f640 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635760f43b0 0x5635760f47d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0ac010 tx=0x7fae3c0bce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fad35602640 1 -- 172.21.15.170:0/2783534653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0b38d0 con 0x5635760f43b0 2024-03-28T16:49:13.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fad35602640 1 -- 172.21.15.170:0/2783534653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c0d8070 con 0x5635760f43b0 2024-03-28T16:49:13.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5635760f4f60 con 0x5635760f43b0 2024-03-28T16:49:13.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fad35602640 1 -- 172.21.15.170:0/2783534653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0e0120 con 0x5635760f43b0 2024-03-28T16:49:13.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635760f5ab0 con 0x5635760f43b0 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fad35602640 1 -- 172.21.15.170:0/2783534653 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fae3c0e0280 con 0x5635760f43b0 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fad35602640 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad24034300 0x7fad240367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fad35602640 1 -- 172.21.15.170:0/2783534653 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fae3c0df6d0 con 0x5635760f43b0 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.963+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad24034300 0x7fad240367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 Processor -- start 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- start start 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad24034300 0x7fad240367c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5635760db210 tx=0x7fae4000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x563575fe6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563575e7e790 con 0x563575fe6140 2024-03-28T16:49:13.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x563575fe6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x563575fe6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55260/0 (socket says 172.21.15.170:55260) 2024-03-28T16:49:13.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 -- 172.21.15.170:0/3790419863 learned_addr learned my addr 172.21.15.170:0/3790419863 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 -- 172.21.15.170:0/3790419863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563575fe4990 con 0x563575fe6140 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 --2- 172.21.15.170:0/3790419863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x563575fe6560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0b3630 tx=0x7fae3c0bb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a515ee19f152b1b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fad2effd640 1 -- 172.21.15.170:0/3790419863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0f1030 con 0x563575fe6140 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fad2effd640 1 -- 172.21.15.170:0/3790419863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c0f2040 con 0x563575fe6140 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fad2effd640 1 -- 172.21.15.170:0/3790419863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0b4880 con 0x563575fe6140 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- 172.21.15.170:0/3790419863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 msgr2=0x563575fe6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 --2- 172.21.15.170:0/3790419863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x563575fe6560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0b3630 tx=0x7fae3c0bb000 comp rx=0 tx=0).stop 2024-03-28T16:49:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- 172.21.15.170:0/3790419863 shutdown_connections 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 --2- 172.21.15.170:0/3790419863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x563575fe6560 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- 172.21.15.170:0/3790419863 >> 172.21.15.170:0/3790419863 conn(0x5635760fd610 msgr2=0x5635760fb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- 172.21.15.170:0/3790419863 shutdown_connections 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- 172.21.15.170:0/3790419863 wait complete. 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 Processor -- start 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- start start 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x56357610a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4da24980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56357610a800 con 0x563575fe6140 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x56357610a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x56357610a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55264/0 (socket says 172.21.15.170:55264) 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 learned_addr learned my addr 172.21.15.170:0/3322868985 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.967+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635760f5ab0 con 0x563575fe6140 2024-03-28T16:49:13.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4b53f640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x56357610a2c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0e3de0 tx=0x7fae3c0a8f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fad2d7fa640 1 -- 172.21.15.170:0/3322868985 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0f1040 con 0x563575fe6140 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56357610aa00 con 0x563575fe6140 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fad2d7fa640 1 -- 172.21.15.170:0/3322868985 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c0df2b0 con 0x563575fe6140 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56357610aa00 con 0x563575fe6140 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fad2d7fa640 1 -- 172.21.15.170:0/3322868985 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae3c0df550 con 0x563575fe6140 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fad2d7fa640 1 -- 172.21.15.170:0/3322868985 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fae3c0fe040 con 0x563575fe6140 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fad2d7fa640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad140342b0 0x7fad14036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fad2d7fa640 1 -- 172.21.15.170:0/3322868985 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fae3c0f2660 con 0x563575fe6140 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad140342b0 0x7fad14036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563576115500 0x5635761179a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761180f0 con 0x563576115500 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4bd40640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563576115500 0x5635761179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad140342b0 0x7fad14036770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5635760e6840 tx=0x7fae40010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4bd40640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563576115500 0x5635761179a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 realm 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118510 con 0x563576115500 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761190c0 con 0x563576115500 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563576119a70 0x56357611be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357611c3e0 con 0x563576119a70 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.971+0000 7fae4b53f640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563576119a70 0x56357611be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4b53f640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563576119a70 0x56357611be60 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4da24980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:13.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4da24980 20 searching for the correct realm 2024-03-28T16:49:13.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56357611e270 0x5635761206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576120db0 con 0x56357611e270 2024-03-28T16:49:13.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56357611e270 0x5635761206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56357611e270 0x5635761206e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4ad3e640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fae4001fbe0 0x7fae4003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576120db0 con 0x7fae4001fbe0 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4bd40640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fae4001fbe0 0x7fae4003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4bd40640 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fae4001fbe0 0x7fae4003a3c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.975+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x7fae4001fbe0 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x7fae4001fbe0 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.979+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x7fae4001fbe0 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.983+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x7fae4001fbe0 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x56357611e270 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576119a70 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x563576115500 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576118d60 con 0x7fae4001fbe0 2024-03-28T16:49:13.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357611cbe0 con 0x563576115500 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.987+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357611cbe0 con 0x563576115500 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:13.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357611cbe0 con 0x7fae4001fbe0 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 zone default found 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 4 Realm: () 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 zonegroup default 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357611d430 con 0x563576119a70 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357611d810 con 0x56357611e270 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357611d0b0 con 0x56357611e270 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 started sync module instance, tier type = 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761224b0 con 0x7fae4001fbe0 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94787614154768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576122860 con 0x7fae4001fbe0 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.991+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576123aa0 con 0x563576115500 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.995+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94787614154768] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:13.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.995+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:13.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.995+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94787614159200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576123a20 con 0x563576115500 2024-03-28T16:49:13.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.995+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761244f0 con 0x56357611e270 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.995+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94787614159200] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.995+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.995+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94787614160256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761239a0 con 0x56357611e270 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576125220 con 0x7fae4001fbe0 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94787614166336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576125a70 con 0x7fae4001fbe0 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576126420 con 0x563576115500 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94787614160256] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:14.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94787614166336] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:14.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94787614165152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576125580 con 0x563576115500 2024-03-28T16:49:14.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:13.999+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576126c70 con 0x563576119a70 2024-03-28T16:49:14.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.003+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94787614165152] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.003+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.003+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94787614173136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576125580 con 0x563576119a70 2024-03-28T16:49:14.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.003+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576127470 con 0x563576119a70 2024-03-28T16:49:14.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.003+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94787614173136] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.003+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.003+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94787614175184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576127890 con 0x563576119a70 2024-03-28T16:49:14.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576128200 con 0x563576119a70 2024-03-28T16:49:14.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94787614178656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576128680 con 0x563576119a70 2024-03-28T16:49:14.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=0 2024-03-28T16:49:14.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=1 2024-03-28T16:49:14.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=2 2024-03-28T16:49:14.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=3 2024-03-28T16:49:14.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=4 2024-03-28T16:49:14.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=5 2024-03-28T16:49:14.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94787614175184] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=6 2024-03-28T16:49:14.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94787614178656] v29'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 20 add_watcher() i=7 2024-03-28T16:49:14.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:14.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.007+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761273f0 con 0x563576115500 2024-03-28T16:49:14.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94787614176240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576128c20 con 0x563576115500 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94787614176240] v29'54 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fad01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 5 note: GC not initialized 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fad00fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 20 Filter name: none 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fac7a7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357614d0f0 con 0x563576115500 2024-03-28T16:49:14.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:14.013 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:14.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fad01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.011+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94787614176240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56357612dca0 con 0x563576115500 2024-03-28T16:49:14.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.015+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94787614176240] v29'55 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.015+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94787614154768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761273f0 con 0x7fae4001fbe0 2024-03-28T16:49:14.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.015+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94787614154768] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:14.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.015+0000 7fae4da24980 20 remove_watcher() i=0 2024-03-28T16:49:14.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.015+0000 7fae4da24980 2 removed watcher, disabling cache 2024-03-28T16:49:14.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.015+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94787614159200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761273f0 con 0x563576115500 2024-03-28T16:49:14.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94787614159200] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4da24980 20 remove_watcher() i=1 2024-03-28T16:49:14.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94787614160256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563576123160 con 0x56357611e270 2024-03-28T16:49:14.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4ad3e640 1 -- 172.21.15.170:0/3322868985 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94787614160256] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae40033040 con 0x56357611e270 2024-03-28T16:49:14.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4da24980 20 remove_watcher() i=2 2024-03-28T16:49:14.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94787614166336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761273f0 con 0x7fae4001fbe0 2024-03-28T16:49:14.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94787614166336] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x7fae4001fbe0 2024-03-28T16:49:14.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4da24980 20 remove_watcher() i=3 2024-03-28T16:49:14.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.019+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94787614165152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761273f0 con 0x563576115500 2024-03-28T16:49:14.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.023+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94787614165152] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae440083f0 con 0x563576115500 2024-03-28T16:49:14.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.023+0000 7fae4da24980 20 remove_watcher() i=4 2024-03-28T16:49:14.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.023+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94787614173136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761273f0 con 0x563576119a70 2024-03-28T16:49:14.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.023+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94787614173136] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.023+0000 7fae4da24980 20 remove_watcher() i=5 2024-03-28T16:49:14.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.023+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94787614175184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761267d0 con 0x563576119a70 2024-03-28T16:49:14.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94787614175184] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 20 remove_watcher() i=6 2024-03-28T16:49:14.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94787614178656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5635761267d0 con 0x563576119a70 2024-03-28T16:49:14.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4b53f640 1 -- 172.21.15.170:0/3322868985 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94787614178656] v29'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fae3c0f4070 con 0x563576119a70 2024-03-28T16:49:14.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 20 remove_watcher() i=7 2024-03-28T16:49:14.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563576119a70 msgr2=0x56357611be60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563576119a70 0x56357611be60 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563576115500 msgr2=0x5635761179a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563576115500 0x5635761179a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fae4001fbe0 msgr2=0x7fae4003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fae4001fbe0 0x7fae4003a3c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56357611e270 msgr2=0x5635761206e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56357611e270 0x5635761206e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad140342b0 msgr2=0x7fad14036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad140342b0 0x7fad14036770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5635760e6840 tx=0x7fae40010000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 msgr2=0x56357610a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 --2- 172.21.15.170:0/3322868985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575fe6140 0x56357610a2c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0e3de0 tx=0x7fae3c0a8f00 comp rx=0 tx=0).stop 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4bd40640 1 -- 172.21.15.170:0/3322868985 reap_dead start 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 shutdown_connections 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.027+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 >> 172.21.15.170:0/3322868985 conn(0x5635760fd610 msgr2=0x5635760fac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 shutdown_connections 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3322868985 wait complete. 2024-03-28T16:49:14.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad24034300 msgr2=0x7fad240367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad24034300 0x7fad240367c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5635760db210 tx=0x7fae4000f000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635760f43b0 msgr2=0x5635760f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635760f43b0 0x5635760f47d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0ac010 tx=0x7fae3c0bce70 comp rx=0 tx=0).stop 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 shutdown_connections 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad24034300 0x7fad240367c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/2783534653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635760f43b0 0x5635760f47d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 >> 172.21.15.170:0/2783534653 conn(0x5635760e91c0 msgr2=0x5635760e9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 shutdown_connections 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/2783534653 wait complete. 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635760dfda0 msgr2=0x5635760e2190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635760dfda0 0x5635760e2190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563575fe90f0 msgr2=0x5635760dd7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563575fe90f0 0x5635760dd7e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635760e3a40 msgr2=0x5635760e5e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635760e3a40 0x5635760e5e30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad300342b0 msgr2=0x7fad30036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fad300342b0 0x7fad30036770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae40001e50 tx=0x7fae400013b0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 msgr2=0x5635760d6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 --2- 172.21.15.170:0/3144209514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563575f5a0b0 0x5635760d6bb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae3c01aec0 tx=0x7fae3c017e30 comp rx=0 tx=0).stop 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4bd40640 1 -- 172.21.15.170:0/3144209514 reap_dead start 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 shutdown_connections 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 >> 172.21.15.170:0/3144209514 conn(0x563575fe06d0 msgr2=0x5635760ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 shutdown_connections 2024-03-28T16:49:14.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.031+0000 7fae4da24980 1 -- 172.21.15.170:0/3144209514 wait complete. 2024-03-28T16:49:14.041 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:14.041 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-03-28T16:49:14.041 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-03-28T16:49:14.042 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-03-28T16:49:14.118 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:14.118 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3623a6a980 1 Processor -- start 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3623a6a980 1 -- start start 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3623a6a980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x563a9a8df130 0x563a9a8df510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3623a6a980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563a9a829520 con 0x563a9a8dfa50 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3623a6a980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x563a9a7f00f0 con 0x563a9a8df130 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3621585640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x563a9a8df130 0x563a9a8df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3620d84640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x563a9a8dfa50 0x563a9a8e0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60860/0 (socket says 172.21.15.170:60860) 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3620d84640 1 -- 172.21.15.170:0/863914527 learned_addr learned my addr 172.21.15.170:0/863914527 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:14.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3621585640 1 -- 172.21.15.170:0/863914527 >> v1:172.21.15.119:6789/0 conn(0x563a9a8dfa50 legacy=0x563a9a8e0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.119+0000 7f3621585640 1 -- 172.21.15.170:0/863914527 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a9a8e0c30 con 0x563a9a8df130 2024-03-28T16:49:14.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.123+0000 7f3621585640 1 --2- 172.21.15.170:0/863914527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a8df510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f361400a480 tx=0x7f361401b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9104a9ab4ec88f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.123+0000 7f3613fff640 1 -- 172.21.15.170:0/863914527 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140ab540 con 0x563a9a8df130 2024-03-28T16:49:14.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.123+0000 7f3613fff640 1 -- 172.21.15.170:0/863914527 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140ab6a0 con 0x563a9a8df130 2024-03-28T16:49:14.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.123+0000 7f3613fff640 1 -- 172.21.15.170:0/863914527 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140ab960 con 0x563a9a8df130 2024-03-28T16:49:14.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.127+0000 7f3623a6a980 1 -- 172.21.15.170:0/863914527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 msgr2=0x563a9a8df510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.127+0000 7f3623a6a980 1 --2- 172.21.15.170:0/863914527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a8df510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f361400a480 tx=0x7f361401b280 comp rx=0 tx=0).stop 2024-03-28T16:49:14.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.127+0000 7f3623a6a980 1 -- 172.21.15.170:0/863914527 shutdown_connections 2024-03-28T16:49:14.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.127+0000 7f3623a6a980 1 --2- 172.21.15.170:0/863914527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a8df510 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.127+0000 7f3623a6a980 1 -- 172.21.15.170:0/863914527 >> 172.21.15.170:0/863914527 conn(0x563a9a852850 msgr2=0x563a9a852c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.127+0000 7f3623a6a980 1 -- 172.21.15.170:0/863914527 shutdown_connections 2024-03-28T16:49:14.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.127+0000 7f3623a6a980 1 -- 172.21.15.170:0/863914527 wait complete. 2024-03-28T16:49:14.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 Processor -- start 2024-03-28T16:49:14.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- start start 2024-03-28T16:49:14.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9a7ed620 con 0x563a9a8df130 2024-03-28T16:49:14.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55282/0 (socket says 172.21.15.170:55282) 2024-03-28T16:49:14.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 -- 172.21.15.170:0/4271716068 learned_addr learned my addr 172.21.15.170:0/4271716068 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 -- 172.21.15.170:0/4271716068 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614005d30 con 0x563a9a8df130 2024-03-28T16:49:14.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 --2- 172.21.15.170:0/4271716068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9c9620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3614018eb0 tx=0x7f3614003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b760d99f91772fd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f350f606640 1 -- 172.21.15.170:0/4271716068 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140ab810 con 0x563a9a8df130 2024-03-28T16:49:14.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f350f606640 1 -- 172.21.15.170:0/4271716068 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140ab970 con 0x563a9a8df130 2024-03-28T16:49:14.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f350f606640 1 -- 172.21.15.170:0/4271716068 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140abc30 con 0x563a9a8df130 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- 172.21.15.170:0/4271716068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 msgr2=0x563a9a9c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 --2- 172.21.15.170:0/4271716068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9c9620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3614018eb0 tx=0x7f3614003030 comp rx=0 tx=0).stop 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- 172.21.15.170:0/4271716068 shutdown_connections 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 --2- 172.21.15.170:0/4271716068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9c9620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- 172.21.15.170:0/4271716068 >> 172.21.15.170:0/4271716068 conn(0x563a9a852850 msgr2=0x563a9a9c51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- 172.21.15.170:0/4271716068 shutdown_connections 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- 172.21.15.170:0/4271716068 wait complete. 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 Processor -- start 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- start start 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3623a6a980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9a7e8e00 con 0x563a9a8df130 2024-03-28T16:49:14.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55290/0 (socket says 172.21.15.170:55290) 2024-03-28T16:49:14.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.143+0000 7f3621585640 1 -- 172.21.15.170:0/3382677430 learned_addr learned my addr 172.21.15.170:0/3382677430 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3621585640 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a9a8e0c30 con 0x563a9a8df130 2024-03-28T16:49:14.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3621585640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9d0350 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f361401b2c0 tx=0x7f3614018020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f350de03640 1 -- 172.21.15.170:0/3382677430 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140c3070 con 0x563a9a8df130 2024-03-28T16:49:14.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f350de03640 1 -- 172.21.15.170:0/3382677430 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140beaa0 con 0x563a9a8df130 2024-03-28T16:49:14.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f350de03640 1 -- 172.21.15.170:0/3382677430 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140bed40 con 0x563a9a8df130 2024-03-28T16:49:14.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a9a9d0890 con 0x563a9a8df130 2024-03-28T16:49:14.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a9a9d0cc0 con 0x563a9a8df130 2024-03-28T16:49:14.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f350de03640 1 -- 172.21.15.170:0/3382677430 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f36140b2070 con 0x563a9a8df130 2024-03-28T16:49:14.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f350de03640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f350de03640 1 -- 172.21.15.170:0/3382677430 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f36140e3750 con 0x563a9a8df130 2024-03-28T16:49:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3620d84640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9a8dfa50 0x563a9a9d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9a9d7470 con 0x563a9a8dfa50 2024-03-28T16:49:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3621d86640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9a8dfa50 0x563a9a9d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3620d84640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f36180012f0 tx=0x7f3618006700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3621d86640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9a8dfa50 0x563a9a9d6f30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3621d86640 1 -- 172.21.15.170:0/3382677430 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9a8dfa50 2024-03-28T16:49:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9a9d77d0 con 0x563a9a8dfa50 2024-03-28T16:49:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.147+0000 7f3621d86640 1 -- 172.21.15.170:0/3382677430 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9a8dfa50 2024-03-28T16:49:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9a9d9160 0x563a9a9db570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9a9dbd00 con 0x563a9a9d9160 2024-03-28T16:49:14.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3621585640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9a9d9160 0x563a9a9db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3621585640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9a9d9160 0x563a9a9db570 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3621585640 1 -- 172.21.15.170:0/3382677430 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f36140b5930 con 0x563a9a9d9160 2024-03-28T16:49:14.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9a9d7fd0 con 0x563a9a8dfa50 2024-03-28T16:49:14.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3621d86640 1 -- 172.21.15.170:0/3382677430 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9a8dfa50 2024-03-28T16:49:14.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x563a9a9dd120 0x563a9a9df510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9a9dfb70 con 0x563a9a9dd120 2024-03-28T16:49:14.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3620d84640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x563a9a9dd120 0x563a9a9df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3620d84640 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x563a9a9dd120 0x563a9a9df510 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3620d84640 1 -- 172.21.15.170:0/3382677430 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f3618004530 con 0x563a9a9dd120 2024-03-28T16:49:14.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 Processor -- start 2024-03-28T16:49:14.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 -- start start 2024-03-28T16:49:14.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9edaa0 0x563a9a9edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.151+0000 7f3623a6a980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9a781e30 con 0x563a9a9edaa0 2024-03-28T16:49:14.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9edaa0 0x563a9a9edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9edaa0 0x563a9a9edec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55306/0 (socket says 172.21.15.170:55306) 2024-03-28T16:49:14.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 -- 172.21.15.170:0/1239434844 learned_addr learned my addr 172.21.15.170:0/1239434844 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 -- 172.21.15.170:0/1239434844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a9a9ee400 con 0x563a9a9edaa0 2024-03-28T16:49:14.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9edaa0 0x563a9a9edec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36140b5fd0 tx=0x7f36140bfe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3506ffd640 1 -- 172.21.15.170:0/1239434844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140e3af0 con 0x563a9a9edaa0 2024-03-28T16:49:14.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a9a9ee650 con 0x563a9a9edaa0 2024-03-28T16:49:14.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3506ffd640 1 -- 172.21.15.170:0/1239434844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140e3c50 con 0x563a9a9edaa0 2024-03-28T16:49:14.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3506ffd640 1 -- 172.21.15.170:0/1239434844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140aa5c0 con 0x563a9a9edaa0 2024-03-28T16:49:14.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a9a9ef1a0 con 0x563a9a9edaa0 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3506ffd640 1 -- 172.21.15.170:0/1239434844 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f36140aa720 con 0x563a9a9edaa0 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3506ffd640 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34fc0342b0 0x7f34fc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3506ffd640 1 -- 172.21.15.170:0/1239434844 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f36140edaa0 con 0x563a9a9edaa0 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3620d84640 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34fc0342b0 0x7f34fc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3623a6a980 1 Processor -- start 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3623a6a980 1 -- start start 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3620d84640 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34fc0342b0 0x7f34fc036770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x563a9a9d8330 tx=0x7f361800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3623a6a980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9a9facc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3623a6a980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9a775790 con 0x563a9a9fa8e0 2024-03-28T16:49:14.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9a9facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9a9facc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55314/0 (socket says 172.21.15.170:55314) 2024-03-28T16:49:14.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.155+0000 7f3621585640 1 -- 172.21.15.170:0/657754727 learned_addr learned my addr 172.21.15.170:0/657754727 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3621585640 1 -- 172.21.15.170:0/657754727 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a9a9fb290 con 0x563a9a9fa8e0 2024-03-28T16:49:14.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3621585640 1 --2- 172.21.15.170:0/657754727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9a9facc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36140e3010 tx=0x7f36140da4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2025ef94b4c3678 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3504ff9640 1 -- 172.21.15.170:0/657754727 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140db830 con 0x563a9a9fa8e0 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3504ff9640 1 -- 172.21.15.170:0/657754727 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140dba90 con 0x563a9a9fa8e0 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3504ff9640 1 -- 172.21.15.170:0/657754727 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140dbe80 con 0x563a9a9fa8e0 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- 172.21.15.170:0/657754727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 msgr2=0x563a9a9facc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 --2- 172.21.15.170:0/657754727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9a9facc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36140e3010 tx=0x7f36140da4d0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- 172.21.15.170:0/657754727 shutdown_connections 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 --2- 172.21.15.170:0/657754727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9a9facc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- 172.21.15.170:0/657754727 >> 172.21.15.170:0/657754727 conn(0x563a9a9f6c20 msgr2=0x563a9a9f4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- 172.21.15.170:0/657754727 shutdown_connections 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- 172.21.15.170:0/657754727 wait complete. 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 Processor -- start 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- start start 2024-03-28T16:49:14.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9aa017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a9aa01cf0 con 0x563a9a9fa8e0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9aa017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3621585640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9aa017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55326/0 (socket says 172.21.15.170:55326) 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 learned_addr learned my addr 172.21.15.170:0/3574563423 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36140e7030 con 0x563a9a9fa8e0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3621585640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9aa017b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f36140d1e90 tx=0x7f36140f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f34f2ffd640 1 -- 172.21.15.170:0/3574563423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140fe040 con 0x563a9a9fa8e0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a9aa01ef0 con 0x563a9a9fa8e0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f34f2ffd640 1 -- 172.21.15.170:0/3574563423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36140d17f0 con 0x563a9a9fa8e0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f34f2ffd640 1 -- 172.21.15.170:0/3574563423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36140bf2f0 con 0x563a9a9fa8e0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.159+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a9aa02320 con 0x563a9a9fa8e0 2024-03-28T16:49:14.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f34f2ffd640 1 -- 172.21.15.170:0/3574563423 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f36140bf450 con 0x563a9a9fa8e0 2024-03-28T16:49:14.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f34f2ffd640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34ec0342b0 0x7f34ec036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f34f2ffd640 1 -- 172.21.15.170:0/3574563423 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f361410eaa0 con 0x563a9a9fa8e0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3620d84640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34ec0342b0 0x7f34ec036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9aa0cb60 0x563a9aa0f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa0f770 con 0x563a9aa0cb60 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3621d86640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9aa0cb60 0x563a9aa0f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3620d84640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34ec0342b0 0x7f34ec036770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36180062e0 tx=0x7f3618010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3621d86640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9aa0cb60 0x563a9aa0f020 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 realm 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa0fb40 con 0x563a9aa0cb60 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa10720 con 0x563a9aa0cb60 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9aa110d0 0x563a9aa134c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa13ad0 con 0x563a9aa110d0 2024-03-28T16:49:14.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.163+0000 7f3621585640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9aa110d0 0x563a9aa134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621585640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9aa110d0 0x563a9aa134c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f36140b5930 con 0x563a9aa110d0 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3623a6a980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3623a6a980 20 searching for the correct realm 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x563a9aa15cd0 0x563a9aa18190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa18860 con 0x563a9aa15cd0 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3620d84640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x563a9aa15cd0 0x563a9aa18190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3620d84640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x563a9aa15cd0 0x563a9aa18190 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3620d84640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3618024070 0x7f3618032980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa18860 con 0x7f3618024070 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621d86640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3618024070 0x7f3618032980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621d86640 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3618024070 0x7f3618032980 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36140b5930 con 0x563a9aa110d0 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.167+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x7f3618024070 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.171+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x7f3618024070 2024-03-28T16:49:14.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.175+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x7f3618024070 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x7f3618024070 2024-03-28T16:49:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa15cd0 2024-03-28T16:49:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa110d0 2024-03-28T16:49:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.179+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa103c0 con 0x7f3618024070 2024-03-28T16:49:14.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa140d0 con 0x563a9aa0cb60 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa140d0 con 0x563a9aa0cb60 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f361c0083f0 con 0x563a9aa0cb60 2024-03-28T16:49:14.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa14080 con 0x7f3618024070 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 zone default found 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 4 Realm: () 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 zonegroup default 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa14920 con 0x563a9aa110d0 2024-03-28T16:49:14.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1a680 con 0x563a9aa15cd0 2024-03-28T16:49:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1a9e0 con 0x563a9aa15cd0 2024-03-28T16:49:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 started sync module instance, tier type = 2024-03-28T16:49:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa19f60 con 0x7f3618024070 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c0083f0 con 0x7f3618024070 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94809702374080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa19ee0 con 0x7f3618024070 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.183+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1b680 con 0x563a9aa0cb60 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.187+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94809702374080] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x7f3618024070 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.187+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.187+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94809702378816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1a2c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.187+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1bc10 con 0x563a9aa15cd0 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94809702378816] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94809702381424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1bf70 con 0x563a9aa15cd0 2024-03-28T16:49:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1c8e0 con 0x7f3618024070 2024-03-28T16:49:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x7f3618024070 2024-03-28T16:49:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94809702384976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1cf90 con 0x7f3618024070 2024-03-28T16:49:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1da60 con 0x563a9aa0cb60 2024-03-28T16:49:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94809702381424] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94809702384976] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x7f3618024070 2024-03-28T16:49:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94809702389056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1da20 con 0x563a9aa0cb60 2024-03-28T16:49:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.191+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1cb60 con 0x563a9aa110d0 2024-03-28T16:49:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.195+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.195+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94809702391312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1e6d0 con 0x563a9aa110d0 2024-03-28T16:49:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.195+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1f040 con 0x563a9aa110d0 2024-03-28T16:49:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.195+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94809702389056] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.195+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94809702391312] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94809702394784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1f3a0 con 0x563a9aa110d0 2024-03-28T16:49:14.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1fbe0 con 0x563a9aa110d0 2024-03-28T16:49:14.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94809702394784] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94809702397696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1b1e0 con 0x563a9aa110d0 2024-03-28T16:49:14.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 20 add_watcher() i=0 2024-03-28T16:49:14.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 20 add_watcher() i=1 2024-03-28T16:49:14.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 20 add_watcher() i=2 2024-03-28T16:49:14.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 20 add_watcher() i=3 2024-03-28T16:49:14.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 20 add_watcher() i=4 2024-03-28T16:49:14.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 20 add_watcher() i=5 2024-03-28T16:49:14.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.199+0000 7f3623a6a980 20 add_watcher() i=6 2024-03-28T16:49:14.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94809702397696] v29'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 20 add_watcher() i=7 2024-03-28T16:49:14.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:14.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa23460 con 0x563a9aa0cb60 2024-03-28T16:49:14.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94809702399424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa23460 con 0x563a9aa0cb60 2024-03-28T16:49:14.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94809702399424] v29'56 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f34d77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:14.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:14.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 5 note: GC not initialized 2024-03-28T16:49:14.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f34d67e4640 20 reqs_thread_entry: start 2024-03-28T16:49:14.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:14.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 20 Filter name: none 2024-03-28T16:49:14.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3447fff640 20 reqs_thread_entry: start 2024-03-28T16:49:14.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:14.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa452c0 con 0x563a9aa0cb60 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa477e0 con 0x7f3618024070 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x7f3618024070 2024-03-28T16:49:14.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.203+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa4af60 con 0x563a9aa0cb60 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v29'2 uv2 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3623a6a980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3623a6a980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3623a6a980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94809702575104 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa47c80 con 0x563a9aa0cb60 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94809702389056 notify 124554051585 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7f361c00aab0 con 0x563a9aa0cb60 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3504ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051585 cookie 94809702389056 notifier 4305 bl.length()=533 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3504ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3504ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(65 notify.4 [notify cookie 94809702575104 out=8b] v0'0 uv33 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3504ff9640 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f34e4003ea0 con 0x563a9aa0cb60 2024-03-28T16:49:14.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94809702575104 notify 124554051585 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f361c00ac40 con 0x563a9aa0cb60 2024-03-28T16:49:14.209 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:14.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f34d77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.207+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94809702399424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa0a200 con 0x563a9aa0cb60 2024-03-28T16:49:14.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94809702399424] v29'57 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94809702374080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1fdc0 con 0x7f3618024070 2024-03-28T16:49:14.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94809702374080] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x7f3618024070 2024-03-28T16:49:14.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3623a6a980 20 remove_watcher() i=0 2024-03-28T16:49:14.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3623a6a980 2 removed watcher, disabling cache 2024-03-28T16:49:14.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94809702378816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1fdc0 con 0x563a9aa0cb60 2024-03-28T16:49:14.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94809702378816] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3623a6a980 20 remove_watcher() i=1 2024-03-28T16:49:14.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.211+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94809702381424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1fdc0 con 0x563a9aa15cd0 2024-03-28T16:49:14.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.215+0000 7f3620d84640 1 -- 172.21.15.170:0/3574563423 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94809702381424] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3618004530 con 0x563a9aa15cd0 2024-03-28T16:49:14.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.215+0000 7f3623a6a980 20 remove_watcher() i=2 2024-03-28T16:49:14.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.215+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94809702384976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1b770 con 0x7f3618024070 2024-03-28T16:49:14.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.215+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94809702384976] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x7f3618024070 2024-03-28T16:49:14.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.215+0000 7f3623a6a980 20 remove_watcher() i=3 2024-03-28T16:49:14.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.215+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94809702389056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1c550 con 0x563a9aa0cb60 2024-03-28T16:49:14.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.219+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94809702389056] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f361c03f040 con 0x563a9aa0cb60 2024-03-28T16:49:14.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.219+0000 7f3623a6a980 20 remove_watcher() i=4 2024-03-28T16:49:14.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.219+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94809702391312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1b770 con 0x563a9aa110d0 2024-03-28T16:49:14.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.219+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94809702391312] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.219+0000 7f3623a6a980 20 remove_watcher() i=5 2024-03-28T16:49:14.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.219+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94809702394784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1c550 con 0x563a9aa110d0 2024-03-28T16:49:14.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.223+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94809702394784] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.223+0000 7f3623a6a980 20 remove_watcher() i=6 2024-03-28T16:49:14.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.223+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94809702397696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x563a9aa1de10 con 0x563a9aa110d0 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.223+0000 7f3621585640 1 -- 172.21.15.170:0/3574563423 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94809702397696] v29'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3614113040 con 0x563a9aa110d0 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.223+0000 7f3623a6a980 20 remove_watcher() i=7 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9aa110d0 msgr2=0x563a9aa134c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9aa110d0 0x563a9aa134c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9aa0cb60 msgr2=0x563a9aa0f020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9aa0cb60 0x563a9aa0f020 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3618024070 msgr2=0x7f3618032980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3618024070 0x7f3618032980 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x563a9aa15cd0 msgr2=0x563a9aa18190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x563a9aa15cd0 0x563a9aa18190 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34ec0342b0 msgr2=0x7f34ec036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34ec0342b0 0x7f34ec036770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36180062e0 tx=0x7f3618010000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3621d86640 1 -- 172.21.15.170:0/3574563423 reap_dead start 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 msgr2=0x563a9aa017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9aa017b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f36140d1e90 tx=0x7f36140f2040 comp rx=0 tx=0).stop 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 shutdown_connections 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3574563423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9fa8e0 0x563a9aa017b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 >> 172.21.15.170:0/3574563423 conn(0x563a9a9f6c20 msgr2=0x563a9a9f4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 shutdown_connections 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3574563423 wait complete. 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34fc0342b0 msgr2=0x7f34fc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34fc0342b0 0x7f34fc036770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x563a9a9d8330 tx=0x7f361800f000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9edaa0 msgr2=0x563a9a9edec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9edaa0 0x563a9a9edec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f36140b5fd0 tx=0x7f36140bfe00 comp rx=0 tx=0).stop 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 shutdown_connections 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34fc0342b0 0x7f34fc036770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/1239434844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a9edaa0 0x563a9a9edec0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 >> 172.21.15.170:0/1239434844 conn(0x563a9a9e28b0 msgr2=0x563a9a9e33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 shutdown_connections 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/1239434844 wait complete. 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9a9d9160 msgr2=0x563a9a9db570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x563a9a9d9160 0x563a9a9db570 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9a8dfa50 msgr2=0x563a9a9d6f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x563a9a8dfa50 0x563a9a9d6f30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x563a9a9dd120 msgr2=0x563a9a9df510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x563a9a9dd120 0x563a9a9df510 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 msgr2=0x7f3508036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f36180012f0 tx=0x7f3618006700 comp rx=0 tx=0).stop 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 msgr2=0x563a9a9d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 --2- 172.21.15.170:0/3382677430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563a9a8df130 0x563a9a9d0350 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f361401b2c0 tx=0x7f3614018020 comp rx=0 tx=0).stop 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3621d86640 1 -- 172.21.15.170:0/3382677430 reap_dead start 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 shutdown_connections 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 >> 172.21.15.170:0/3382677430 conn(0x563a9a852850 msgr2=0x563a9a9c3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 shutdown_connections 2024-03-28T16:49:14.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.227+0000 7f3623a6a980 1 -- 172.21.15.170:0/3382677430 wait complete. 2024-03-28T16:49:14.238 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:14.238 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:14.238 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:14.238 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:14.313 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:14.313 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:14.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dad8c8980 1 Processor -- start 2024-03-28T16:49:14.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dad8c8980 1 -- start start 2024-03-28T16:49:14.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dad8c8980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5616e8f15130 0x5616e8f15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dad8c8980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5616e8e5f520 con 0x5616e8f15a50 2024-03-28T16:49:14.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dad8c8980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5616e8e260f0 con 0x5616e8f15130 2024-03-28T16:49:14.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dab3e3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5616e8f15130 0x5616e8f15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dab3e3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5616e8f15130 0x5616e8f15510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55338/0 (socket says 172.21.15.170:55338) 2024-03-28T16:49:14.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dab3e3640 1 -- 172.21.15.170:0/1609744211 learned_addr learned my addr 172.21.15.170:0/1609744211 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2daabe2640 1 --1- 172.21.15.170:0/1609744211 >> v1:172.21.15.119:6789/0 conn(0x5616e8f15a50 0x5616e8f16400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60866/0 (socket says 172.21.15.170:60866) 2024-03-28T16:49:14.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dab3e3640 1 -- 172.21.15.170:0/1609744211 >> v1:172.21.15.119:6789/0 conn(0x5616e8f15a50 legacy=0x5616e8f16400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.315+0000 7f2dab3e3640 1 -- 172.21.15.170:0/1609744211 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e8f16c30 con 0x5616e8f15130 2024-03-28T16:49:14.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/1609744211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8f15510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00a5c0 tx=0x7f2d9c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6322d237f9a51a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2daa3e1640 1 -- 172.21.15.170:0/1609744211 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0aa470 con 0x5616e8f15130 2024-03-28T16:49:14.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2daa3e1640 1 -- 172.21.15.170:0/1609744211 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0aa5d0 con 0x5616e8f15130 2024-03-28T16:49:14.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2daa3e1640 1 -- 172.21.15.170:0/1609744211 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0aa890 con 0x5616e8f15130 2024-03-28T16:49:14.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1609744211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 msgr2=0x5616e8f15510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/1609744211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8f15510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00a5c0 tx=0x7f2d9c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:14.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1609744211 shutdown_connections 2024-03-28T16:49:14.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/1609744211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8f15510 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1609744211 >> 172.21.15.170:0/1609744211 conn(0x5616e8e88850 msgr2=0x5616e8e88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1609744211 shutdown_connections 2024-03-28T16:49:14.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.319+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1609744211 wait complete. 2024-03-28T16:49:14.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 Processor -- start 2024-03-28T16:49:14.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 -- start start 2024-03-28T16:49:14.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8fff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8e23620 con 0x5616e8f15130 2024-03-28T16:49:14.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8fff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8fff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55340/0 (socket says 172.21.15.170:55340) 2024-03-28T16:49:14.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dab3e3640 1 -- 172.21.15.170:0/1488278983 learned_addr learned my addr 172.21.15.170:0/1488278983 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dab3e3640 1 -- 172.21.15.170:0/1488278983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c005df0 con 0x5616e8f15130 2024-03-28T16:49:14.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/1488278983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8fff600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0ac040 tx=0x7f2d9c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58658cfb048092d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2c9960a640 1 -- 172.21.15.170:0/1488278983 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0aa9b0 con 0x5616e8f15130 2024-03-28T16:49:14.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2c9960a640 1 -- 172.21.15.170:0/1488278983 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0aab30 con 0x5616e8f15130 2024-03-28T16:49:14.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2c9960a640 1 -- 172.21.15.170:0/1488278983 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0aae10 con 0x5616e8f15130 2024-03-28T16:49:14.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1488278983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 msgr2=0x5616e8fff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/1488278983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8fff600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0ac040 tx=0x7f2d9c003030 comp rx=0 tx=0).stop 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1488278983 shutdown_connections 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/1488278983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e8fff600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1488278983 >> 172.21.15.170:0/1488278983 conn(0x5616e8e88850 msgr2=0x5616e8ffb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1488278983 shutdown_connections 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.335+0000 7f2dad8c8980 1 -- 172.21.15.170:0/1488278983 wait complete. 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 Processor -- start 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 -- start start 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e9006370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8e1ee00 con 0x5616e8f15130 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e9006370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e9006370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55350/0 (socket says 172.21.15.170:55350) 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dab3e3640 1 -- 172.21.15.170:0/427339395 learned_addr learned my addr 172.21.15.170:0/427339395 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dab3e3640 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e8f16c30 con 0x5616e8f15130 2024-03-28T16:49:14.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e9006370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c01b820 tx=0x7f2d9c0180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2c97e07640 1 -- 172.21.15.170:0/427339395 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0c6650 con 0x5616e8f15130 2024-03-28T16:49:14.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5616e90068b0 con 0x5616e8f15130 2024-03-28T16:49:14.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2c97e07640 1 -- 172.21.15.170:0/427339395 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0c67b0 con 0x5616e8f15130 2024-03-28T16:49:14.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2c97e07640 1 -- 172.21.15.170:0/427339395 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0c6a90 con 0x5616e8f15130 2024-03-28T16:49:14.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616e9006ce0 con 0x5616e8f15130 2024-03-28T16:49:14.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2c97e07640 1 -- 172.21.15.170:0/427339395 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2d9c0c6cc0 con 0x5616e8f15130 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2c97e07640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c900342b0 0x7f2c90036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2c97e07640 1 -- 172.21.15.170:0/427339395 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f2d9c0e4ca0 con 0x5616e8f15130 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2daabe2640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c900342b0 0x7f2c90036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e8f15a50 0x5616e900cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.339+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e900d530 con 0x5616e8f15a50 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dabbe4640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e8f15a50 0x5616e900cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2daabe2640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c900342b0 0x7f2c90036770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2da0000f70 tx=0x7f2da0001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dabbe4640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e8f15a50 0x5616e900cff0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dabbe4640 1 -- 172.21.15.170:0/427339395 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e8f15a50 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e900d890 con 0x5616e8f15a50 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dabbe4640 1 -- 172.21.15.170:0/427339395 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e8f15a50 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e900f5b0 0x5616e90119a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9011ee0 con 0x5616e900f5b0 2024-03-28T16:49:14.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e900f5b0 0x5616e90119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e900f5b0 0x5616e90119a0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dab3e3640 1 -- 172.21.15.170:0/427339395 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f2d9c0b0020 con 0x5616e900f5b0 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e900e3f0 con 0x5616e8f15a50 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dabbe4640 1 -- 172.21.15.170:0/427339395 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e8f15a50 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5616e9013250 0x5616e9015640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9015c90 con 0x5616e9013250 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2daabe2640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5616e9013250 0x5616e9015640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.343+0000 7f2daabe2640 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5616e9013250 0x5616e9015640 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2daabe2640 1 -- 172.21.15.170:0/427339395 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f2da000f6a0 con 0x5616e9013250 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dad8c8980 1 Processor -- start 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dad8c8980 1 -- start start 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dad8c8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e9023bc0 0x5616e9023fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dad8c8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8db7e30 con 0x5616e9023bc0 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e9023bc0 0x5616e9023fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e9023bc0 0x5616e9023fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55364/0 (socket says 172.21.15.170:55364) 2024-03-28T16:49:14.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3199751173 learned_addr learned my addr 172.21.15.170:0/3199751173 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3199751173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e9024520 con 0x5616e9023bc0 2024-03-28T16:49:14.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e9023bc0 0x5616e9023fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0bfff0 tx=0x7f2d9c0e3320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2c95602640 1 -- 172.21.15.170:0/3199751173 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0e0030 con 0x5616e9023bc0 2024-03-28T16:49:14.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5616e9024770 con 0x5616e9023bc0 2024-03-28T16:49:14.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2c95602640 1 -- 172.21.15.170:0/3199751173 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0c5720 con 0x5616e9023bc0 2024-03-28T16:49:14.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2c95602640 1 -- 172.21.15.170:0/3199751173 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0c59e0 con 0x5616e9023bc0 2024-03-28T16:49:14.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616e90252c0 con 0x5616e9023bc0 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2c95602640 1 -- 172.21.15.170:0/3199751173 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2d9c0c73f0 con 0x5616e9023bc0 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2c95602640 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c840342b0 0x7f2c84036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.347+0000 7f2c95602640 1 -- 172.21.15.170:0/3199751173 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f2d9c0e1070 con 0x5616e9023bc0 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2daabe2640 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c840342b0 0x7f2c84036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 Processor -- start 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- start start 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2daabe2640 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c840342b0 0x7f2c84036770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5616e900e750 tx=0x7f2da000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e8e89e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e8dab790 con 0x5616e8e89a90 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e8e89e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e8e89e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55374/0 (socket says 172.21.15.170:55374) 2024-03-28T16:49:14.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 -- 172.21.15.170:0/4286762228 learned_addr learned my addr 172.21.15.170:0/4286762228 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 -- 172.21.15.170:0/4286762228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e8e8a460 con 0x5616e8e89a90 2024-03-28T16:49:14.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/4286762228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e8e89e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0ee040 tx=0x7f2d9c0c76a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e055c716be4692 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2c8effd640 1 -- 172.21.15.170:0/4286762228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c107070 con 0x5616e8e89a90 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2c8effd640 1 -- 172.21.15.170:0/4286762228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0fd040 con 0x5616e8e89a90 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2c8effd640 1 -- 172.21.15.170:0/4286762228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0ec400 con 0x5616e8e89a90 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- 172.21.15.170:0/4286762228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 msgr2=0x5616e8e89e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/4286762228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e8e89e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0ee040 tx=0x7f2d9c0c76a0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- 172.21.15.170:0/4286762228 shutdown_connections 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/4286762228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e8e89e90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- 172.21.15.170:0/4286762228 >> 172.21.15.170:0/4286762228 conn(0x5616e902cea0 msgr2=0x5616e902af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- 172.21.15.170:0/4286762228 shutdown_connections 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- 172.21.15.170:0/4286762228 wait complete. 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 Processor -- start 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- start start 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e903bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dad8c8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616e903c300 con 0x5616e8e89a90 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e903bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e903bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55384/0 (socket says 172.21.15.170:55384) 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.351+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 learned_addr learned my addr 172.21.15.170:0/3042116558 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616e90252c0 con 0x5616e8e89a90 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e903bdc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0d7fd0 tx=0x7f2d9c0ecf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2c8d7fa640 1 -- 172.21.15.170:0/3042116558 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c107040 con 0x5616e8e89a90 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2c8d7fa640 1 -- 172.21.15.170:0/3042116558 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0ec990 con 0x5616e8e89a90 2024-03-28T16:49:14.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5616e90252c0 con 0x5616e8e89a90 2024-03-28T16:49:14.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2c8d7fa640 1 -- 172.21.15.170:0/3042116558 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d9c0ec400 con 0x5616e8e89a90 2024-03-28T16:49:14.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616e90252c0 con 0x5616e8e89a90 2024-03-28T16:49:14.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2c8d7fa640 1 -- 172.21.15.170:0/3042116558 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2d9c0a7920 con 0x5616e8e89a90 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2c8d7fa640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c74034260 0x7f2c74036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2c8d7fa640 1 -- 172.21.15.170:0/3042116558 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f2d9c10daf0 con 0x5616e8e89a90 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2daabe2640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c74034260 0x7f2c74036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e9046f10 0x5616e90493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9049b20 con 0x5616e9046f10 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dabbe4640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e9046f10 0x5616e90493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2daabe2640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c74034260 0x7f2c74036720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c010 tx=0x7f2da0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dabbe4640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e9046f10 0x5616e90493d0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 realm 2024-03-28T16:49:14.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9049ef0 con 0x5616e9046f10 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.355+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904aaa0 con 0x5616e9046f10 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e904b450 0x5616e904d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904ddc0 con 0x5616e904b450 2024-03-28T16:49:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e904b450 0x5616e904d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dab3e3640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e904b450 0x5616e904d840 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 20 searching for the correct realm 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5616e904fc50 0x5616e9052110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90527e0 con 0x5616e904fc50 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2daabe2640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5616e904fc50 0x5616e9052110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2daabe2640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5616e904fc50 0x5616e9052110 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2daabe2640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f2da001fc60 0x7f2da003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90527e0 con 0x7f2da001fc60 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dabbe4640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f2da001fc60 0x7f2da003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.359+0000 7f2dabbe4640 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f2da001fc60 0x7f2da003a3c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x7f2da001fc60 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.363+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x7f2da001fc60 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.367+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x7f2da001fc60 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x7f2da001fc60 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904fc50 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.371+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e904b450 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x5616e9046f10 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904a740 con 0x7f2da001fc60 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:14.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904e5c0 con 0x5616e9046f10 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904e5c0 con 0x5616e9046f10 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904e5c0 con 0x7f2da001fc60 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:14.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 zone default found 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 4 Realm: () 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 zonegroup default 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904ee10 con 0x5616e904b450 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e904f1f0 con 0x5616e904fc50 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9054760 con 0x5616e904fc50 2024-03-28T16:49:14.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 started sync module instance, tier type = 2024-03-28T16:49:14.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:14.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.375+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9053ee0 con 0x7f2da001fc60 2024-03-28T16:49:14.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.379+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.379+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94656398703168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9054290 con 0x7f2da001fc60 2024-03-28T16:49:14.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.379+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90554c0 con 0x5616e9046f10 2024-03-28T16:49:14.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.379+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94656398703168] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.379+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.379+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94656398707584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9055480 con 0x5616e9046f10 2024-03-28T16:49:14.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.379+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9055f30 con 0x5616e904fc50 2024-03-28T16:49:14.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94656398707584] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94656398711344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90562e0 con 0x5616e904fc50 2024-03-28T16:49:14.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9056c50 con 0x7f2da001fc60 2024-03-28T16:49:14.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94656398715104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90574a0 con 0x7f2da001fc60 2024-03-28T16:49:14.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9057e10 con 0x5616e9046f10 2024-03-28T16:49:14.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94656398711344] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.383+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94656398715104] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.387+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.387+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94656398716768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9057380 con 0x5616e9046f10 2024-03-28T16:49:14.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.387+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90582b0 con 0x5616e904b450 2024-03-28T16:49:14.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.387+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94656398716768] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.387+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.387+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94656398719344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9058610 con 0x5616e904b450 2024-03-28T16:49:14.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.387+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9058e10 con 0x5616e904b450 2024-03-28T16:49:14.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94656398719344] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94656398723440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9059230 con 0x5616e904b450 2024-03-28T16:49:14.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9059be0 con 0x5616e904b450 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94656398723440] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94656398726912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9059dc0 con 0x5616e904b450 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 20 add_watcher() i=0 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 20 add_watcher() i=1 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 20 add_watcher() i=2 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 20 add_watcher() i=3 2024-03-28T16:49:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 20 add_watcher() i=4 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 20 add_watcher() i=5 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.391+0000 7f2dad8c8980 20 add_watcher() i=6 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94656398726912] v29'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 20 add_watcher() i=7 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e905d180 con 0x5616e9046f10 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94656398728368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e905a5e0 con 0x5616e9046f10 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94656398728368] v29'58 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2c61fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 5 note: GC not initialized 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2c60fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 20 Filter name: none 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2bda7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e907eaf0 con 0x5616e9046f10 2024-03-28T16:49:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.395+0000 7f2dad8c8980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:14.399 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2c61fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94656398728368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e905f6b0 con 0x5616e9046f10 2024-03-28T16:49:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94656398728368] v29'59 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94656398703168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9059cb0 con 0x7f2da001fc60 2024-03-28T16:49:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94656398703168] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2dad8c8980 20 remove_watcher() i=0 2024-03-28T16:49:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2dad8c8980 2 removed watcher, disabling cache 2024-03-28T16:49:14.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.399+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94656398707584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9053a40 con 0x5616e9046f10 2024-03-28T16:49:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94656398707584] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dad8c8980 20 remove_watcher() i=1 2024-03-28T16:49:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94656398711344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9053a40 con 0x5616e904fc50 2024-03-28T16:49:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2daabe2640 1 -- 172.21.15.170:0/3042116558 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94656398711344] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da0033040 con 0x5616e904fc50 2024-03-28T16:49:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dad8c8980 20 remove_watcher() i=2 2024-03-28T16:49:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94656398715104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9055a30 con 0x7f2da001fc60 2024-03-28T16:49:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94656398715104] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x7f2da001fc60 2024-03-28T16:49:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dad8c8980 20 remove_watcher() i=3 2024-03-28T16:49:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.403+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94656398716768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90568e0 con 0x5616e9046f10 2024-03-28T16:49:14.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.407+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94656398716768] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2da400a4d0 con 0x5616e9046f10 2024-03-28T16:49:14.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.407+0000 7f2dad8c8980 20 remove_watcher() i=4 2024-03-28T16:49:14.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.407+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94656398719344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9055a30 con 0x5616e904b450 2024-03-28T16:49:14.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.407+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94656398719344] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.407+0000 7f2dad8c8980 20 remove_watcher() i=5 2024-03-28T16:49:14.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.407+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94656398723440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e9056f60 con 0x5616e904b450 2024-03-28T16:49:14.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94656398723440] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 20 remove_watcher() i=6 2024-03-28T16:49:14.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94656398726912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616e90568e0 con 0x5616e904b450 2024-03-28T16:49:14.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dab3e3640 1 -- 172.21.15.170:0/3042116558 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94656398726912] v29'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2d9c109070 con 0x5616e904b450 2024-03-28T16:49:14.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 20 remove_watcher() i=7 2024-03-28T16:49:14.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e904b450 msgr2=0x5616e904d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e904b450 0x5616e904d840 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e9046f10 msgr2=0x5616e90493d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e9046f10 0x5616e90493d0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f2da001fc60 msgr2=0x7f2da003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f2da001fc60 0x7f2da003a3c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5616e904fc50 msgr2=0x5616e9052110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.411+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5616e904fc50 0x5616e9052110 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c74034260 msgr2=0x7f2c74036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c74034260 0x7f2c74036720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2da000c010 tx=0x7f2da0010000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 msgr2=0x5616e903bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e903bdc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0d7fd0 tx=0x7f2d9c0ecf30 comp rx=0 tx=0).stop 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dabbe4640 1 -- 172.21.15.170:0/3042116558 reap_dead start 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 shutdown_connections 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3042116558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8e89a90 0x5616e903bdc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 >> 172.21.15.170:0/3042116558 conn(0x5616e902cea0 msgr2=0x5616e902a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 shutdown_connections 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3042116558 wait complete. 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c840342b0 msgr2=0x7f2c84036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c840342b0 0x7f2c84036770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5616e900e750 tx=0x7f2da000f000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e9023bc0 msgr2=0x5616e9023fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e9023bc0 0x5616e9023fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0bfff0 tx=0x7f2d9c0e3320 comp rx=0 tx=0).stop 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 shutdown_connections 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c840342b0 0x7f2c84036770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/3199751173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e9023bc0 0x5616e9023fe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 >> 172.21.15.170:0/3199751173 conn(0x5616e90189d0 msgr2=0x5616e90194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 shutdown_connections 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/3199751173 wait complete. 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e900f5b0 msgr2=0x5616e90119a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616e900f5b0 0x5616e90119a0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e8f15a50 msgr2=0x5616e900cff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5616e8f15a50 0x5616e900cff0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5616e9013250 msgr2=0x5616e9015640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5616e9013250 0x5616e9015640 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c900342b0 msgr2=0x7f2c90036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2c900342b0 0x7f2c90036770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2da0000f70 tx=0x7f2da0001380 comp rx=0 tx=0).stop 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 msgr2=0x5616e9006370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 --2- 172.21.15.170:0/427339395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616e8f15130 0x5616e9006370 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c01b820 tx=0x7f2d9c0180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dabbe4640 1 -- 172.21.15.170:0/427339395 reap_dead start 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 shutdown_connections 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 >> 172.21.15.170:0/427339395 conn(0x5616e8e88850 msgr2=0x5616e8ff9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 shutdown_connections 2024-03-28T16:49:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.415+0000 7f2dad8c8980 1 -- 172.21.15.170:0/427339395 wait complete. 2024-03-28T16:49:14.426 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:14.426 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-03-28T16:49:14.426 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-03-28T16:49:14.426 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-03-28T16:49:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:14.500 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:14.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3b6709980 1 Processor -- start 2024-03-28T16:49:14.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3b6709980 1 -- start start 2024-03-28T16:49:14.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3b6709980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ef1b6aa630 0x55ef1b6aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3b6709980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ef1b5f0520 con 0x55ef1b6a6130 2024-03-28T16:49:14.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3b6709980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ef1b5b70f0 con 0x55ef1b6aa630 2024-03-28T16:49:14.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3abfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ef1b6aa630 0x55ef1b6aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3abfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ef1b6aa630 0x55ef1b6aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55390/0 (socket says 172.21.15.170:55390) 2024-03-28T16:49:14.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3b4224640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55ef1b6a6130 0x55ef1b6a6510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60868/0 (socket says 172.21.15.170:60868) 2024-03-28T16:49:14.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3abfff640 1 -- 172.21.15.170:0/2461319381 learned_addr learned my addr 172.21.15.170:0/2461319381 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3abfff640 1 -- 172.21.15.170:0/2461319381 >> v1:172.21.15.119:6789/0 conn(0x55ef1b6a6130 legacy=0x55ef1b6a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3abfff640 1 -- 172.21.15.170:0/2461319381 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1b6adf70 con 0x55ef1b6aa630 2024-03-28T16:49:14.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.503+0000 7fe3abfff640 1 --2- 172.21.15.170:0/2461319381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6aa630 0x55ef1b6aca20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00a5c0 tx=0x7fe39c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fa7cecde55cb17 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b3a23640 1 -- 172.21.15.170:0/2461319381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe39c0aa470 con 0x55ef1b6aa630 2024-03-28T16:49:14.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b3a23640 1 -- 172.21.15.170:0/2461319381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c0aa5d0 con 0x55ef1b6aa630 2024-03-28T16:49:14.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b3a23640 1 -- 172.21.15.170:0/2461319381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe39c0aa890 con 0x55ef1b6aa630 2024-03-28T16:49:14.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b6709980 1 -- 172.21.15.170:0/2461319381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6aa630 msgr2=0x55ef1b6aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b6709980 1 --2- 172.21.15.170:0/2461319381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6aa630 0x55ef1b6aca20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00a5c0 tx=0x7fe39c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:14.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b6709980 1 -- 172.21.15.170:0/2461319381 shutdown_connections 2024-03-28T16:49:14.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b6709980 1 --2- 172.21.15.170:0/2461319381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6aa630 0x55ef1b6aca20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b6709980 1 -- 172.21.15.170:0/2461319381 >> 172.21.15.170:0/2461319381 conn(0x55ef1b619850 msgr2=0x55ef1b619c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b6709980 1 -- 172.21.15.170:0/2461319381 shutdown_connections 2024-03-28T16:49:14.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.507+0000 7fe3b6709980 1 -- 172.21.15.170:0/2461319381 wait complete. 2024-03-28T16:49:14.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 Processor -- start 2024-03-28T16:49:14.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 -- start start 2024-03-28T16:49:14.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b78ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1b5b4620 con 0x55ef1b6a6130 2024-03-28T16:49:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b78ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b78ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55400/0 (socket says 172.21.15.170:55400) 2024-03-28T16:49:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b4224640 1 -- 172.21.15.170:0/3374351604 learned_addr learned my addr 172.21.15.170:0/3374351604 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b4224640 1 -- 172.21.15.170:0/3374351604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c005df0 con 0x55ef1b6a6130 2024-03-28T16:49:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b4224640 1 --2- 172.21.15.170:0/3374351604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b78ffe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4001cc0 tx=0x7fe3a400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3703e26ce7dee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe2a260c640 1 -- 172.21.15.170:0/3374351604 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4014070 con 0x55ef1b6a6130 2024-03-28T16:49:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe2a260c640 1 -- 172.21.15.170:0/3374351604 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a400f590 con 0x55ef1b6a6130 2024-03-28T16:49:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe2a260c640 1 -- 172.21.15.170:0/3374351604 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4007040 con 0x55ef1b6a6130 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 -- 172.21.15.170:0/3374351604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 msgr2=0x55ef1b78ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 --2- 172.21.15.170:0/3374351604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b78ffe0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4001cc0 tx=0x7fe3a400b610 comp rx=0 tx=0).stop 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 -- 172.21.15.170:0/3374351604 shutdown_connections 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 --2- 172.21.15.170:0/3374351604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b78ffe0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 -- 172.21.15.170:0/3374351604 >> 172.21.15.170:0/3374351604 conn(0x55ef1b619850 msgr2=0x55ef1b78bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 -- 172.21.15.170:0/3374351604 shutdown_connections 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 -- 172.21.15.170:0/3374351604 wait complete. 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.523+0000 7fe3b6709980 1 Processor -- start 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b6709980 1 -- start start 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b6709980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b796e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b6709980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1b5afe00 con 0x55ef1b6a6130 2024-03-28T16:49:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b796e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b796e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55404/0 (socket says 172.21.15.170:55404) 2024-03-28T16:49:14.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b4224640 1 -- 172.21.15.170:0/1445108758 learned_addr learned my addr 172.21.15.170:0/1445108758 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b4224640 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1b5f0520 con 0x55ef1b6a6130 2024-03-28T16:49:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b4224640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b796e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55ef1b790e70 tx=0x7fe3a4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe2a0e09640 1 -- 172.21.15.170:0/1445108758 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4014030 con 0x55ef1b6a6130 2024-03-28T16:49:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe2a0e09640 1 -- 172.21.15.170:0/1445108758 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a401c840 con 0x55ef1b6a6130 2024-03-28T16:49:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe2a0e09640 1 -- 172.21.15.170:0/1445108758 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a401cb40 con 0x55ef1b6a6130 2024-03-28T16:49:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef1b797370 con 0x55ef1b6a6130 2024-03-28T16:49:14.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef1b7977a0 con 0x55ef1b6a6130 2024-03-28T16:49:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe2a0e09640 1 -- 172.21.15.170:0/1445108758 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe3a401cca0 con 0x55ef1b6a6130 2024-03-28T16:49:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe2a0e09640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe29c0342b0 0x7fe29c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe2a0e09640 1 -- 172.21.15.170:0/1445108758 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe3a4006070 con 0x55ef1b6a6130 2024-03-28T16:49:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3abfff640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe29c0342b0 0x7fe29c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b6aa630 0x55ef1b79dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b79dff0 con 0x55ef1b6aa630 2024-03-28T16:49:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3b4a25640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b6aa630 0x55ef1b79dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.527+0000 7fe3abfff640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe29c0342b0 0x7fe29c036770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0ae000 tx=0x7fe39c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b4a25640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b6aa630 0x55ef1b79dab0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b4a25640 1 -- 172.21.15.170:0/1445108758 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b6aa630 2024-03-28T16:49:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b79e350 con 0x55ef1b6aa630 2024-03-28T16:49:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b4a25640 1 -- 172.21.15.170:0/1445108758 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b6aa630 2024-03-28T16:49:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7a0070 0x55ef1b7a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7a29a0 con 0x55ef1b7a0070 2024-03-28T16:49:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b4224640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7a0070 0x55ef1b7a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b4224640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7a0070 0x55ef1b7a2460 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b4224640 1 -- 172.21.15.170:0/1445108758 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fe3a4016070 con 0x55ef1b7a0070 2024-03-28T16:49:14.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b79eee0 con 0x55ef1b6aa630 2024-03-28T16:49:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b4a25640 1 -- 172.21.15.170:0/1445108758 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b6aa630 2024-03-28T16:49:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ef1b7a3d40 0x55ef1b7a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7a67d0 con 0x55ef1b7a3d40 2024-03-28T16:49:14.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3abfff640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ef1b7a3d40 0x55ef1b7a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.531+0000 7fe3abfff640 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ef1b7a3d40 0x55ef1b7a6130 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3abfff640 1 -- 172.21.15.170:0/1445108758 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7a3d40 2024-03-28T16:49:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b6709980 1 Processor -- start 2024-03-28T16:49:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b6709980 1 -- start start 2024-03-28T16:49:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b6709980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7b4700 0x55ef1b7b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b6709980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1b548e30 con 0x55ef1b7b4700 2024-03-28T16:49:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7b4700 0x55ef1b7b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7b4700 0x55ef1b7b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55416/0 (socket says 172.21.15.170:55416) 2024-03-28T16:49:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b4224640 1 -- 172.21.15.170:0/4165793970 learned_addr learned my addr 172.21.15.170:0/4165793970 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b4224640 1 -- 172.21.15.170:0/4165793970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1b7b5060 con 0x55ef1b7b4700 2024-03-28T16:49:14.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b4224640 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7b4700 0x55ef1b7b4b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4000f70 tx=0x7fe3a4025850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe299ffb640 1 -- 172.21.15.170:0/4165793970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4017bb0 con 0x55ef1b7b4700 2024-03-28T16:49:14.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe299ffb640 1 -- 172.21.15.170:0/4165793970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a4017d10 con 0x55ef1b7b4700 2024-03-28T16:49:14.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef1b7b52b0 con 0x55ef1b7b4700 2024-03-28T16:49:14.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe299ffb640 1 -- 172.21.15.170:0/4165793970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4018400 con 0x55ef1b7b4700 2024-03-28T16:49:14.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef1b7b5e00 con 0x55ef1b7b4700 2024-03-28T16:49:14.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe299ffb640 1 -- 172.21.15.170:0/4165793970 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe3a4049bd0 con 0x55ef1b7b4700 2024-03-28T16:49:14.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe299ffb640 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe28c0342b0 0x7fe28c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe299ffb640 1 -- 172.21.15.170:0/4165793970 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe3a403db00 con 0x55ef1b7b4700 2024-03-28T16:49:14.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.535+0000 7fe3abfff640 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe28c0342b0 0x7fe28c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 Processor -- start 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- start start 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3abfff640 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe28c0342b0 0x7fe28c036770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55ef1b79b540 tx=0x7fe39c005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1b53c790 con 0x55ef1b7c15f0 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55432/0 (socket says 172.21.15.170:55432) 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 -- 172.21.15.170:0/1263403380 learned_addr learned my addr 172.21.15.170:0/1263403380 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 -- 172.21.15.170:0/1263403380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1b7c1fe0 con 0x55ef1b7c15f0 2024-03-28T16:49:14.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 --2- 172.21.15.170:0/1263403380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c1a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe3a404b000 tx=0x7fe3a40264d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1ebfa112f42b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe2937fe640 1 -- 172.21.15.170:0/1263403380 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4011e20 con 0x55ef1b7c15f0 2024-03-28T16:49:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe2937fe640 1 -- 172.21.15.170:0/1263403380 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a4019cb0 con 0x55ef1b7c15f0 2024-03-28T16:49:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe2937fe640 1 -- 172.21.15.170:0/1263403380 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4041210 con 0x55ef1b7c15f0 2024-03-28T16:49:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- 172.21.15.170:0/1263403380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 msgr2=0x55ef1b7c1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1263403380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c1a10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe3a404b000 tx=0x7fe3a40264d0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- 172.21.15.170:0/1263403380 shutdown_connections 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1263403380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c1a10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- 172.21.15.170:0/1263403380 >> 172.21.15.170:0/1263403380 conn(0x55ef1b7bd980 msgr2=0x55ef1b7bb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- 172.21.15.170:0/1263403380 shutdown_connections 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- 172.21.15.170:0/1263403380 wait complete. 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 Processor -- start 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- start start 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b6709980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1b7c8a80 con 0x55ef1b7c15f0 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55442/0 (socket says 172.21.15.170:55442) 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 learned_addr learned my addr 172.21.15.170:0/924524649 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.539+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef1b7b5e00 con 0x55ef1b7c15f0 2024-03-28T16:49:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.543+0000 7fe3b4224640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c8540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4048050 tx=0x7fe3a400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.543+0000 7fe291ffb640 1 -- 172.21.15.170:0/924524649 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4064070 con 0x55ef1b7c15f0 2024-03-28T16:49:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.543+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef1b7b5e00 con 0x55ef1b7c15f0 2024-03-28T16:49:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.543+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef1b7b5e00 con 0x55ef1b7c15f0 2024-03-28T16:49:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.543+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.543+0000 7fe291ffb640 1 -- 172.21.15.170:0/924524649 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a4011e20 con 0x55ef1b7c15f0 2024-03-28T16:49:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.543+0000 7fe291ffb640 1 -- 172.21.15.170:0/924524649 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3a4041210 con 0x55ef1b7c15f0 2024-03-28T16:49:14.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe291ffb640 1 -- 172.21.15.170:0/924524649 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe3a4041ba0 con 0x55ef1b7c15f0 2024-03-28T16:49:14.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe291ffb640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2840342b0 0x7fe284036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe291ffb640 1 -- 172.21.15.170:0/924524649 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe3a4076170 con 0x55ef1b7c15f0 2024-03-28T16:49:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3abfff640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2840342b0 0x7fe284036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b7d3640 0x55ef1b7d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6250 con 0x55ef1b7d3640 2024-03-28T16:49:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b4a25640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b7d3640 0x55ef1b7d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3abfff640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2840342b0 0x7fe284036770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe39c01b380 tx=0x7fe39c0c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b4a25640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b7d3640 0x55ef1b7d5b00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 realm 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6670 con 0x55ef1b7d3640 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d7220 con 0x55ef1b7d3640 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7d7bd0 0x55ef1b7d9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7da660 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.547+0000 7fe3b4224640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7d7bd0 0x55ef1b7d9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4224640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7d7bd0 0x55ef1b7d9fc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b6709980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:14.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b6709980 20 searching for the correct realm 2024-03-28T16:49:14.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ef1b7dc7d0 0x55ef1b7dec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7df310 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3abfff640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ef1b7dc7d0 0x55ef1b7dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3abfff640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ef1b7dc7d0 0x55ef1b7dec90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3abfff640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe39c0e0040 0x7fe39c0d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7df310 con 0x7fe39c0e0040 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4a25640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe39c0e0040 0x7fe39c0d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4a25640 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe39c0e0040 0x7fe39c0d3400 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.551+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x7fe39c0e0040 2024-03-28T16:49:14.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x7fe39c0e0040 2024-03-28T16:49:14.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.555+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x7fe39c0e0040 2024-03-28T16:49:14.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.559+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x7fe39c0e0040 2024-03-28T16:49:14.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x55ef1b7d3640 2024-03-28T16:49:14.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7d6ec0 con 0x7fe39c0e0040 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.563+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7dabd0 con 0x55ef1b7d3640 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7dabd0 con 0x55ef1b7d3640 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7dabd0 con 0x7fe39c0e0040 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 zone default found 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 4 Realm: () 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 zonegroup default 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7db420 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e1130 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e1490 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 started sync module instance, tier type = 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e0a10 con 0x7fe39c0e0040 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94485446788464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e0990 con 0x7fe39c0e0040 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.567+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e2130 con 0x55ef1b7d3640 2024-03-28T16:49:14.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.571+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94485446788464] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.571+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.571+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94485446793200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e0990 con 0x55ef1b7d3640 2024-03-28T16:49:14.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.571+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e26c0 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.571+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94485446793200] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94485446795808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e2a70 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e33e0 con 0x7fe39c0e0040 2024-03-28T16:49:14.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94485446799472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e3c30 con 0x7fe39c0e0040 2024-03-28T16:49:14.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e45e0 con 0x55ef1b7d3640 2024-03-28T16:49:14.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94485446795808] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94485446799472] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94485446794256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e3710 con 0x55ef1b7d3640 2024-03-28T16:49:14.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.575+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e4e60 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.579+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94485446794256] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.579+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.579+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94485446805952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e3710 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.579+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e5660 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.579+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94485446805952] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94485446803712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e5e60 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e67d0 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94485446812976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e6f50 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=0 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=1 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=2 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=3 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=4 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=5 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94485446803712] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=6 2024-03-28T16:49:14.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94485446812976] v29'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 20 add_watcher() i=7 2024-03-28T16:49:14.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:14.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.583+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e59c0 con 0x55ef1b7d3640 2024-03-28T16:49:14.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94485446814192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e59c0 con 0x55ef1b7d3640 2024-03-28T16:49:14.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94485446814192] v29'60 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe26efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 5 note: GC not initialized 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe26dfdb640 20 reqs_thread_entry: start 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 20 Filter name: none 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe1d77fe640 20 reqs_thread_entry: start 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b80be90 con 0x55ef1b7d3640 2024-03-28T16:49:14.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:14.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:14.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:14.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b80e3b0 con 0x7fe39c0e0040 2024-03-28T16:49:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.587+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b811b70 con 0x55ef1b7d3640 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v29'3 uv3 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b6709980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b6709980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b6709980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94485446989456 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b80e850 con 0x55ef1b7d3640 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94485446794256 notify 124554051586 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7fe3ac043990 con 0x55ef1b7d3640 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe2937fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051586 cookie 94485446794256 notifier 4319 bl.length()=533 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe2937fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(65 notify.4 [notify cookie 94485446989456 out=8b] v0'0 uv39 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe2937fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe2937fe640 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fe1ec003ea0 con 0x55ef1b7d3640 2024-03-28T16:49:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94485446989456 notify 124554051586 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fe3ac043b20 con 0x55ef1b7d3640 2024-03-28T16:49:14.592 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe26efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94485446814192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b80adf0 con 0x55ef1b7d3640 2024-03-28T16:49:14.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94485446814192] v29'61 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.591+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94485446788464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e69b0 con 0x7fe39c0e0040 2024-03-28T16:49:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.595+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94485446788464] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.595+0000 7fe3b6709980 20 remove_watcher() i=0 2024-03-28T16:49:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.595+0000 7fe3b6709980 2 removed watcher, disabling cache 2024-03-28T16:49:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.595+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94485446793200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e0570 con 0x55ef1b7d3640 2024-03-28T16:49:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.595+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94485446793200] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.595+0000 7fe3b6709980 20 remove_watcher() i=1 2024-03-28T16:49:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.595+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94485446795808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e0570 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.599+0000 7fe3abfff640 1 -- 172.21.15.170:0/924524649 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94485446795808] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe39c0b0020 con 0x55ef1b7dc7d0 2024-03-28T16:49:14.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.599+0000 7fe3b6709980 20 remove_watcher() i=2 2024-03-28T16:49:14.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.599+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94485446799472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e2220 con 0x7fe39c0e0040 2024-03-28T16:49:14.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.599+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94485446799472] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x7fe39c0e0040 2024-03-28T16:49:14.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.599+0000 7fe3b6709980 20 remove_watcher() i=3 2024-03-28T16:49:14.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.599+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94485446794256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e3070 con 0x55ef1b7d3640 2024-03-28T16:49:14.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.603+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94485446794256] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3ac00a4d0 con 0x55ef1b7d3640 2024-03-28T16:49:14.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.603+0000 7fe3b6709980 20 remove_watcher() i=4 2024-03-28T16:49:14.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.603+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94485446805952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e3070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.603+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94485446805952] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.603+0000 7fe3b6709980 20 remove_watcher() i=5 2024-03-28T16:49:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.603+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94485446803712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e3070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94485446803712] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 20 remove_watcher() i=6 2024-03-28T16:49:14.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94485446812976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ef1b7e4100 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b4224640 1 -- 172.21.15.170:0/924524649 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94485446812976] v29'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe3a405a070 con 0x55ef1b7d7bd0 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 20 remove_watcher() i=7 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7d7bd0 msgr2=0x55ef1b7d9fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7d7bd0 0x55ef1b7d9fc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b7d3640 msgr2=0x55ef1b7d5b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b7d3640 0x55ef1b7d5b00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe39c0e0040 msgr2=0x7fe39c0d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe39c0e0040 0x7fe39c0d3400 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ef1b7dc7d0 msgr2=0x55ef1b7dec90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ef1b7dc7d0 0x55ef1b7dec90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2840342b0 msgr2=0x7fe284036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2840342b0 0x7fe284036770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe39c01b380 tx=0x7fe39c0c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:14.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b4a25640 1 -- 172.21.15.170:0/924524649 reap_dead start 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 msgr2=0x55ef1b7c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c8540 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4048050 tx=0x7fe3a400ee90 comp rx=0 tx=0).stop 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 shutdown_connections 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 --2- 172.21.15.170:0/924524649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7c15f0 0x55ef1b7c8540 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 >> 172.21.15.170:0/924524649 conn(0x55ef1b7bd980 msgr2=0x55ef1b7bafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 shutdown_connections 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.607+0000 7fe3b6709980 1 -- 172.21.15.170:0/924524649 wait complete. 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe28c0342b0 msgr2=0x7fe28c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe28c0342b0 0x7fe28c036770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55ef1b79b540 tx=0x7fe39c005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7b4700 msgr2=0x55ef1b7b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7b4700 0x55ef1b7b4b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4000f70 tx=0x7fe3a4025850 comp rx=0 tx=0).stop 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 shutdown_connections 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe28c0342b0 0x7fe28c036770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/4165793970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b7b4700 0x55ef1b7b4b20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 >> 172.21.15.170:0/4165793970 conn(0x55ef1b7a9510 msgr2=0x55ef1b7aa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 shutdown_connections 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/4165793970 wait complete. 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7a0070 msgr2=0x55ef1b7a2460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ef1b7a0070 0x55ef1b7a2460 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b6aa630 msgr2=0x55ef1b79dab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ef1b6aa630 0x55ef1b79dab0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ef1b7a3d40 msgr2=0x55ef1b7a6130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ef1b7a3d40 0x55ef1b7a6130 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe29c0342b0 msgr2=0x7fe29c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe29c0342b0 0x7fe29c036770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0ae000 tx=0x7fe39c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 msgr2=0x55ef1b796e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 --2- 172.21.15.170:0/1445108758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ef1b6a6130 0x55ef1b796e30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55ef1b790e70 tx=0x7fe3a4000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b4a25640 1 -- 172.21.15.170:0/1445108758 reap_dead start 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 shutdown_connections 2024-03-28T16:49:14.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 >> 172.21.15.170:0/1445108758 conn(0x55ef1b619850 msgr2=0x55ef1b78a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 shutdown_connections 2024-03-28T16:49:14.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.611+0000 7fe3b6709980 1 -- 172.21.15.170:0/1445108758 wait complete. 2024-03-28T16:49:14.622 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:14.623 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:14.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:14.623 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:14.703 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:14.703 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5f960980 1 Processor -- start 2024-03-28T16:49:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5f960980 1 -- start start 2024-03-28T16:49:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5f960980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e1397f0440 0x55e1397f2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5f960980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e139738520 con 0x55e1397ebf80 2024-03-28T16:49:14.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5f960980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e1396ff0f0 con 0x55e1397f0440 2024-03-28T16:49:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5cc7a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e1397f0440 0x55e1397f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5cc7a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e1397f0440 0x55e1397f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55454/0 (socket says 172.21.15.170:55454) 2024-03-28T16:49:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5d47b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55e1397ebf80 0x55e1397ec360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60870/0 (socket says 172.21.15.170:60870) 2024-03-28T16:49:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/3141852021 learned_addr learned my addr 172.21.15.170:0/3141852021 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/3141852021 >> v1:172.21.15.119:6789/0 conn(0x55e1397ebf80 legacy=0x55e1397ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.703+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/3141852021 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1397f3d80 con 0x55e1397f0440 2024-03-28T16:49:14.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/3141852021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397f0440 0x55e1397f2830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400a5c0 tx=0x7fbd5401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669fe9dffee8ec43 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd4ffff640 1 -- 172.21.15.170:0/3141852021 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd540aa470 con 0x55e1397f0440 2024-03-28T16:49:14.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd4ffff640 1 -- 172.21.15.170:0/3141852021 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd540aa5d0 con 0x55e1397f0440 2024-03-28T16:49:14.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd4ffff640 1 -- 172.21.15.170:0/3141852021 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd540aa890 con 0x55e1397f0440 2024-03-28T16:49:14.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5f960980 1 -- 172.21.15.170:0/3141852021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397f0440 msgr2=0x55e1397f2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5f960980 1 --2- 172.21.15.170:0/3141852021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397f0440 0x55e1397f2830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400a5c0 tx=0x7fbd5401b340 comp rx=0 tx=0).stop 2024-03-28T16:49:14.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5f960980 1 -- 172.21.15.170:0/3141852021 shutdown_connections 2024-03-28T16:49:14.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5f960980 1 --2- 172.21.15.170:0/3141852021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397f0440 0x55e1397f2830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5f960980 1 -- 172.21.15.170:0/3141852021 >> 172.21.15.170:0/3141852021 conn(0x55e1397e7850 msgr2=0x55e1397e9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5f960980 1 -- 172.21.15.170:0/3141852021 shutdown_connections 2024-03-28T16:49:14.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.707+0000 7fbd5f960980 1 -- 172.21.15.170:0/3141852021 wait complete. 2024-03-28T16:49:14.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.723+0000 7fbd5f960980 1 Processor -- start 2024-03-28T16:49:14.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.723+0000 7fbd5f960980 1 -- start start 2024-03-28T16:49:14.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.723+0000 7fbd5f960980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398d5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.723+0000 7fbd5f960980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1396fc620 con 0x55e1397ebf80 2024-03-28T16:49:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.723+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.723+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398d5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55462/0 (socket says 172.21.15.170:55462) 2024-03-28T16:49:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.723+0000 7fbd5d47b640 1 -- 172.21.15.170:0/3375798618 learned_addr learned my addr 172.21.15.170:0/3375798618 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5d47b640 1 -- 172.21.15.170:0/3375798618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54005df0 con 0x55e1397ebf80 2024-03-28T16:49:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/3375798618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398d5df0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001d40 tx=0x7fbd5000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9058474065fcf9c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbc4b606640 1 -- 172.21.15.170:0/3375798618 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd50014070 con 0x55e1397ebf80 2024-03-28T16:49:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbc4b606640 1 -- 172.21.15.170:0/3375798618 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5000f590 con 0x55e1397ebf80 2024-03-28T16:49:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbc4b606640 1 -- 172.21.15.170:0/3375798618 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd50007040 con 0x55e1397ebf80 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- 172.21.15.170:0/3375798618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 msgr2=0x55e1398d5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 --2- 172.21.15.170:0/3375798618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398d5df0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001d40 tx=0x7fbd5000b610 comp rx=0 tx=0).stop 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- 172.21.15.170:0/3375798618 shutdown_connections 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 --2- 172.21.15.170:0/3375798618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398d5df0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- 172.21.15.170:0/3375798618 >> 172.21.15.170:0/3375798618 conn(0x55e1397e7850 msgr2=0x55e1398d1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- 172.21.15.170:0/3375798618 shutdown_connections 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- 172.21.15.170:0/3375798618 wait complete. 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 Processor -- start 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- start start 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1396f7e00 con 0x55e1397ebf80 2024-03-28T16:49:14.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398dcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55476/0 (socket says 172.21.15.170:55476) 2024-03-28T16:49:14.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5d47b640 1 -- 172.21.15.170:0/2855622270 learned_addr learned my addr 172.21.15.170:0/2855622270 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5d47b640 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e139738520 con 0x55e1397ebf80 2024-03-28T16:49:14.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398dcc40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b650 tx=0x7fbd50000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbc49e03640 1 -- 172.21.15.170:0/2855622270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd50014030 con 0x55e1397ebf80 2024-03-28T16:49:14.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1398dd180 con 0x55e1397ebf80 2024-03-28T16:49:14.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbc49e03640 1 -- 172.21.15.170:0/2855622270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd50001530 con 0x55e1397ebf80 2024-03-28T16:49:14.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbc49e03640 1 -- 172.21.15.170:0/2855622270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd5000eb10 con 0x55e1397ebf80 2024-03-28T16:49:14.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.727+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1398dd5b0 con 0x55e1397ebf80 2024-03-28T16:49:14.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbc49e03640 1 -- 172.21.15.170:0/2855622270 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbd5000ed00 con 0x55e1397ebf80 2024-03-28T16:49:14.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbc49e03640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc440342b0 0x7fbc44036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbc49e03640 1 -- 172.21.15.170:0/2855622270 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbd50006070 con 0x55e1397ebf80 2024-03-28T16:49:14.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc440342b0 0x7fbc44036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1397f0440 0x55e1398e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1398e3e00 con 0x55e1397f0440 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5dc7c640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1397f0440 0x55e1398e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc440342b0 0x7fbc44036770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd540ae000 tx=0x7fbd540a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5dc7c640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1397f0440 0x55e1398e38c0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/2855622270 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e1397f0440 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1398e4160 con 0x55e1397f0440 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/2855622270 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e1397f0440 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1398e5e80 0x55e1398e8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1398e87b0 con 0x55e1398e5e80 2024-03-28T16:49:14.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1398e5e80 0x55e1398e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.731+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1398e5e80 0x55e1398e8270 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5d47b640 1 -- 172.21.15.170:0/2855622270 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fbd50016070 con 0x55e1398e5e80 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1398e4cf0 con 0x55e1397f0440 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/2855622270 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e1397f0440 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1398e9b90 0x55e1398ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1398ec620 con 0x55e1398e9b90 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1398e9b90 0x55e1398ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1398e9b90 0x55e1398ebf80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/2855622270 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1398e9b90 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 Processor -- start 2024-03-28T16:49:14.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 -- start start 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1398fa550 0x55e1398fa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e139690e30 con 0x55e1398fa550 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1398fa550 0x55e1398fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1398fa550 0x55e1398fa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55492/0 (socket says 172.21.15.170:55492) 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5d47b640 1 -- 172.21.15.170:0/217808353 learned_addr learned my addr 172.21.15.170:0/217808353 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5d47b640 1 -- 172.21.15.170:0/217808353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1398faeb0 con 0x55e1398fa550 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1398fa550 0x55e1398fa970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd50017820 tx=0x7fbd50025a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbc42ffd640 1 -- 172.21.15.170:0/217808353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd50039070 con 0x55e1398fa550 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbc42ffd640 1 -- 172.21.15.170:0/217808353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd50017ce0 con 0x55e1398fa550 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1398fb100 con 0x55e1398fa550 2024-03-28T16:49:14.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbc42ffd640 1 -- 172.21.15.170:0/217808353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd5001c440 con 0x55e1398fa550 2024-03-28T16:49:14.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.735+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1398fbc50 con 0x55e1398fa550 2024-03-28T16:49:14.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbc42ffd640 1 -- 172.21.15.170:0/217808353 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbd50049b10 con 0x55e1398fa550 2024-03-28T16:49:14.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbc42ffd640 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc340342b0 0x7fbc34036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbc42ffd640 1 -- 172.21.15.170:0/217808353 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbd5003fac0 con 0x55e1398fa550 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc340342b0 0x7fbc34036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 Processor -- start 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- start start 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc340342b0 0x7fbc34036770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55e1398e5050 tx=0x7fbd54005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1397629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e139684790 con 0x55e1397625b0 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1397629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1397629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55498/0 (socket says 172.21.15.170:55498) 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5d47b640 1 -- 172.21.15.170:0/2505812816 learned_addr learned my addr 172.21.15.170:0/2505812816 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5d47b640 1 -- 172.21.15.170:0/2505812816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e139760e00 con 0x55e1397625b0 2024-03-28T16:49:14.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/2505812816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1397629b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbd50025aa0 tx=0x7fbd500034b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8939aad22d4ed4c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbc40ff9640 1 -- 172.21.15.170:0/2505812816 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd5001ae30 con 0x55e1397625b0 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbc40ff9640 1 -- 172.21.15.170:0/2505812816 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5001c920 con 0x55e1397625b0 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbc40ff9640 1 -- 172.21.15.170:0/2505812816 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd5001dd80 con 0x55e1397625b0 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- 172.21.15.170:0/2505812816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 msgr2=0x55e1397629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2505812816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1397629b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbd50025aa0 tx=0x7fbd500034b0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- 172.21.15.170:0/2505812816 shutdown_connections 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2505812816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1397629b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- 172.21.15.170:0/2505812816 >> 172.21.15.170:0/2505812816 conn(0x55e139903810 msgr2=0x55e139901870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- 172.21.15.170:0/2505812816 shutdown_connections 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- 172.21.15.170:0/2505812816 wait complete. 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 Processor -- start 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- start start 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1399104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.739+0000 7fbd5f960980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e139910a10 con 0x55e1397625b0 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1399104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5d47b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1399104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55510/0 (socket says 172.21.15.170:55510) 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 learned_addr learned my addr 172.21.15.170:0/1466544223 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1398fbc50 con 0x55e1397625b0 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1399104d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd5001cad0 tx=0x7fbd50037e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbc32ffd640 1 -- 172.21.15.170:0/1466544223 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd50065070 con 0x55e1397625b0 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbc32ffd640 1 -- 172.21.15.170:0/1466544223 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5001ae30 con 0x55e1397625b0 2024-03-28T16:49:14.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e139910c10 con 0x55e1397625b0 2024-03-28T16:49:14.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbc32ffd640 1 -- 172.21.15.170:0/1466544223 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd5001bb90 con 0x55e1397625b0 2024-03-28T16:49:14.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e139910c10 con 0x55e1397625b0 2024-03-28T16:49:14.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbc32ffd640 1 -- 172.21.15.170:0/1466544223 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbd5001bcf0 con 0x55e1397625b0 2024-03-28T16:49:14.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbc32ffd640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc280342b0 0x7fbc28036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbc32ffd640 1 -- 172.21.15.170:0/1466544223 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbd500211d0 con 0x55e1397625b0 2024-03-28T16:49:14.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc280342b0 0x7fbc28036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e13991b770 0x55e13991dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991e350 con 0x55e13991b770 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5dc7c640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e13991b770 0x55e13991dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc280342b0 0x7fbc28036770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c10 tx=0x7fbd540c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5dc7c640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e13991b770 0x55e13991dc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 20 realm 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.743+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991e720 con 0x55e13991b770 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:14.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991f300 con 0x55e13991b770 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e13991fcb0 0x55e1399220a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1399225e0 con 0x55e13991fcb0 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e13991fcb0 0x55e1399220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5d47b640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e13991fcb0 0x55e1399220a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:14.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 20 searching for the correct realm 2024-03-28T16:49:14.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1399244f0 0x55e139926980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139927050 con 0x55e1399244f0 2024-03-28T16:49:14.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1399244f0 0x55e139926980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1399244f0 0x55e139926980 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5cc7a640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbd540e0040 0x7fbd540d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139927050 con 0x7fbd540e0040 2024-03-28T16:49:14.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.747+0000 7fbd5dc7c640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbd540e0040 0x7fbd540d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbd540e0040 0x7fbd540d3400 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x7fbd540e0040 2024-03-28T16:49:14.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.751+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x7fbd540e0040 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.755+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x7fbd540e0040 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x7fbd540e0040 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e1399244f0 2024-03-28T16:49:14.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991fcb0 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x55e13991b770 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13991efa0 con 0x7fbd540e0040 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:14.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139922e10 con 0x55e13991b770 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.759+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139922e10 con 0x55e13991b770 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139922e10 con 0x7fbd540e0040 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 zone default found 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 4 Realm: () 2024-03-28T16:49:14.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 zonegroup default 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139923660 con 0x55e13991fcb0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139923a40 con 0x55e1399244f0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139928f20 con 0x55e1399244f0 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 started sync module instance, tier type = 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139928750 con 0x7fbd540e0040 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94425821905584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139928b00 con 0x7fbd540e0040 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139929c80 con 0x55e13991b770 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.763+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94425821905584] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.767+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.767+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94425821909824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139929c80 con 0x55e13991b770 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.767+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992a720 con 0x55e1399244f0 2024-03-28T16:49:14.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.767+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94425821909824] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.767+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.767+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94425821913600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992aad0 con 0x55e1399244f0 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.767+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992b440 con 0x7fbd540e0040 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94425821917360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992bb30 con 0x7fbd540e0040 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992c5e0 con 0x55e13991b770 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94425821913600] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94425821917360] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94425821921504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992c5a0 con 0x55e13991b770 2024-03-28T16:49:14.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992c9c0 con 0x55e13991fcb0 2024-03-28T16:49:14.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.771+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94425821921504] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.775+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.775+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94425821910880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992cd70 con 0x55e13991fcb0 2024-03-28T16:49:14.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.775+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992d6e0 con 0x55e13991fcb0 2024-03-28T16:49:14.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.775+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.775+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94425821928000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992e300 con 0x55e13991fcb0 2024-03-28T16:49:14.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.775+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992ec70 con 0x55e13991fcb0 2024-03-28T16:49:14.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.775+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94425821910880] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94425821928000] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94425821918608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992e7d0 con 0x55e13991fcb0 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=0 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=1 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=2 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=3 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=4 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=5 2024-03-28T16:49:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=6 2024-03-28T16:49:14.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94425821918608] v29'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 add_watcher() i=7 2024-03-28T16:49:14.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:14.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992e7d0 con 0x55e13991b770 2024-03-28T16:49:14.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94425821929056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992e7d0 con 0x55e13991b770 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94425821929056] v29'62 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbc137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.779+0000 7fbd5f960980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 5 note: GC not initialized 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbc127e4640 20 reqs_thread_entry: start 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 20 Filter name: none 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbb83fff640 20 reqs_thread_entry: start 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139953630 con 0x55e13991b770 2024-03-28T16:49:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:14.783 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbc137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94425821929056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e139934200 con 0x55e13991b770 2024-03-28T16:49:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94425821929056] v29'63 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94425821905584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992de60 con 0x7fbd540e0040 2024-03-28T16:49:14.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94425821905584] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 20 remove_watcher() i=0 2024-03-28T16:49:14.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 2 removed watcher, disabling cache 2024-03-28T16:49:14.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.783+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94425821909824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1399282b0 con 0x55e13991b770 2024-03-28T16:49:14.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.787+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94425821909824] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.787+0000 7fbd5f960980 20 remove_watcher() i=1 2024-03-28T16:49:14.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.787+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94425821913600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1399282b0 con 0x55e1399244f0 2024-03-28T16:49:14.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.787+0000 7fbd5cc7a640 1 -- 172.21.15.170:0/1466544223 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94425821913600] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd540b0020 con 0x55e1399244f0 2024-03-28T16:49:14.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.787+0000 7fbd5f960980 20 remove_watcher() i=2 2024-03-28T16:49:14.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.787+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94425821917360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992a200 con 0x7fbd540e0040 2024-03-28T16:49:14.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.791+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94425821917360] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x7fbd540e0040 2024-03-28T16:49:14.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.791+0000 7fbd5f960980 20 remove_watcher() i=3 2024-03-28T16:49:14.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.791+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94425821921504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992b0b0 con 0x55e13991b770 2024-03-28T16:49:14.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.791+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94425821921504] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd5800a4d0 con 0x55e13991b770 2024-03-28T16:49:14.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.791+0000 7fbd5f960980 20 remove_watcher() i=4 2024-03-28T16:49:14.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.791+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94425821910880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e13992b0b0 con 0x55e13991fcb0 2024-03-28T16:49:14.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94425821910880] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5f960980 20 remove_watcher() i=5 2024-03-28T16:49:14.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94425821928000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1399297e0 con 0x55e13991fcb0 2024-03-28T16:49:14.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94425821928000] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5f960980 20 remove_watcher() i=6 2024-03-28T16:49:14.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94425821918608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1399297e0 con 0x55e13991fcb0 2024-03-28T16:49:14.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5d47b640 1 -- 172.21.15.170:0/1466544223 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94425821918608] v29'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbd50016070 con 0x55e13991fcb0 2024-03-28T16:49:14.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.795+0000 7fbd5f960980 20 remove_watcher() i=7 2024-03-28T16:49:14.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e13991fcb0 msgr2=0x55e1399220a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e13991fcb0 0x55e1399220a0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e13991b770 msgr2=0x55e13991dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e13991b770 0x55e13991dc00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbd540e0040 msgr2=0x7fbd540d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbd540e0040 0x7fbd540d3400 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1399244f0 msgr2=0x55e139926980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1399244f0 0x55e139926980 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc280342b0 msgr2=0x7fbc28036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc280342b0 0x7fbc28036770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c10 tx=0x7fbd540c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 msgr2=0x55e1399104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1399104d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbd5001cad0 tx=0x7fbd50037e90 comp rx=0 tx=0).stop 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/1466544223 reap_dead start 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 shutdown_connections 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/1466544223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397625b0 0x55e1399104d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 >> 172.21.15.170:0/1466544223 conn(0x55e139903810 msgr2=0x55e139900e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 shutdown_connections 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/1466544223 wait complete. 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc340342b0 msgr2=0x7fbc34036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc340342b0 0x7fbc34036770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55e1398e5050 tx=0x7fbd54005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1398fa550 msgr2=0x55e1398fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1398fa550 0x55e1398fa970 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd50017820 tx=0x7fbd50025a10 comp rx=0 tx=0).stop 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 shutdown_connections 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc340342b0 0x7fbc34036770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/217808353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1398fa550 0x55e1398fa970 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 >> 172.21.15.170:0/217808353 conn(0x55e1398ef360 msgr2=0x55e1398efe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 shutdown_connections 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/217808353 wait complete. 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1398e5e80 msgr2=0x55e1398e8270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1398e5e80 0x55e1398e8270 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1397f0440 msgr2=0x55e1398e38c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1397f0440 0x55e1398e38c0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1398e9b90 msgr2=0x55e1398ebf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1398e9b90 0x55e1398ebf80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc440342b0 msgr2=0x7fbc44036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbc440342b0 0x7fbc44036770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd540ae000 tx=0x7fbd540a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 msgr2=0x55e1398dcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 --2- 172.21.15.170:0/2855622270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1397ebf80 0x55e1398dcc40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000b650 tx=0x7fbd50000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5dc7c640 1 -- 172.21.15.170:0/2855622270 reap_dead start 2024-03-28T16:49:14.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 shutdown_connections 2024-03-28T16:49:14.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 >> 172.21.15.170:0/2855622270 conn(0x55e1397e7850 msgr2=0x55e1398d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 shutdown_connections 2024-03-28T16:49:14.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.799+0000 7fbd5f960980 1 -- 172.21.15.170:0/2855622270 wait complete. 2024-03-28T16:49:14.808 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:14.808 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-03-28T16:49:14.808 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-03-28T16:49:14.809 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-03-28T16:49:14.879 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:14.879 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:14.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65460d4980 1 Processor -- start 2024-03-28T16:49:14.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65460d4980 1 -- start start 2024-03-28T16:49:14.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65460d4980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d689f3fda0 0x55d689fcdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65460d4980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d689f174c0 con 0x55d689f410e0 2024-03-28T16:49:14.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65460d4980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d689ede0f0 con 0x55d689f3fda0 2024-03-28T16:49:14.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65433ee640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d689f3fda0 0x55d689fcdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f6543bef640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55d689f410e0 0x55d689f3f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60884/0 (socket says 172.21.15.170:60884) 2024-03-28T16:49:14.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f6543bef640 1 -- 172.21.15.170:0/4273573781 learned_addr learned my addr 172.21.15.170:0/4273573781 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:14.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65433ee640 1 -- 172.21.15.170:0/4273573781 >> v1:172.21.15.119:6789/0 conn(0x55d689f410e0 legacy=0x55d689f3f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.879+0000 7f65433ee640 1 -- 172.21.15.170:0/4273573781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d689f401b0 con 0x55d689f3fda0 2024-03-28T16:49:14.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f65433ee640 1 --2- 172.21.15.170:0/4273573781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d689fcdad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f653800a480 tx=0x7f653801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db468d1c7cad75c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f6542bed640 1 -- 172.21.15.170:0/4273573781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65380aa530 con 0x55d689f3fda0 2024-03-28T16:49:14.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f6542bed640 1 -- 172.21.15.170:0/4273573781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65380a6050 con 0x55d689f3fda0 2024-03-28T16:49:14.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f6542bed640 1 -- 172.21.15.170:0/4273573781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65380aa7e0 con 0x55d689f3fda0 2024-03-28T16:49:14.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f65460d4980 1 -- 172.21.15.170:0/4273573781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 msgr2=0x55d689fcdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f65460d4980 1 --2- 172.21.15.170:0/4273573781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d689fcdad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f653800a480 tx=0x7f653801b280 comp rx=0 tx=0).stop 2024-03-28T16:49:14.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f65460d4980 1 -- 172.21.15.170:0/4273573781 shutdown_connections 2024-03-28T16:49:14.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f65460d4980 1 --2- 172.21.15.170:0/4273573781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d689fcdad0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f65460d4980 1 -- 172.21.15.170:0/4273573781 >> 172.21.15.170:0/4273573781 conn(0x55d689fc67b0 msgr2=0x55d689fc8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.883+0000 7f65460d4980 1 -- 172.21.15.170:0/4273573781 shutdown_connections 2024-03-28T16:49:14.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.887+0000 7f65460d4980 1 -- 172.21.15.170:0/4273573781 wait complete. 2024-03-28T16:49:14.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 Processor -- start 2024-03-28T16:49:14.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- start start 2024-03-28T16:49:14.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689edb620 con 0x55d689f3fda0 2024-03-28T16:49:14.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0bb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55520/0 (socket says 172.21.15.170:55520) 2024-03-28T16:49:14.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6543bef640 1 -- 172.21.15.170:0/1394763795 learned_addr learned my addr 172.21.15.170:0/1394763795 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6543bef640 1 -- 172.21.15.170:0/1394763795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538005d30 con 0x55d689f3fda0 2024-03-28T16:49:14.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6543bef640 1 --2- 172.21.15.170:0/1394763795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0bb800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6534001c90 tx=0x7f653400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bca69651ca96bc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6431e0b640 1 -- 172.21.15.170:0/1394763795 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f653400e690 con 0x55d689f3fda0 2024-03-28T16:49:14.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6431e0b640 1 -- 172.21.15.170:0/1394763795 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400e7f0 con 0x55d689f3fda0 2024-03-28T16:49:14.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6431e0b640 1 -- 172.21.15.170:0/1394763795 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f653400eaf0 con 0x55d689f3fda0 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- 172.21.15.170:0/1394763795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 msgr2=0x55d68a0bb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 --2- 172.21.15.170:0/1394763795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0bb800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6534001c90 tx=0x7f653400b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- 172.21.15.170:0/1394763795 shutdown_connections 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 --2- 172.21.15.170:0/1394763795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0bb800 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- 172.21.15.170:0/1394763795 >> 172.21.15.170:0/1394763795 conn(0x55d689fc67b0 msgr2=0x55d68a0b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- 172.21.15.170:0/1394763795 shutdown_connections 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- 172.21.15.170:0/1394763795 wait complete. 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 Processor -- start 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- start start 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0c2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f65460d4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689ed6e00 con 0x55d689f3fda0 2024-03-28T16:49:14.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.899+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0c2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55536/0 (socket says 172.21.15.170:55536) 2024-03-28T16:49:14.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f6543bef640 1 -- 172.21.15.170:0/372181124 learned_addr learned my addr 172.21.15.170:0/372181124 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f6543bef640 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d689f174c0 con 0x55d689f3fda0 2024-03-28T16:49:14.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f6543bef640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0c2770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6534001ee0 tx=0x7f6534000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f642bfff640 1 -- 172.21.15.170:0/372181124 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f653401f070 con 0x55d689f3fda0 2024-03-28T16:49:14.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d68a0c2cb0 con 0x55d689f3fda0 2024-03-28T16:49:14.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f642bfff640 1 -- 172.21.15.170:0/372181124 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653401a4d0 con 0x55d689f3fda0 2024-03-28T16:49:14.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d68a0c30e0 con 0x55d689f3fda0 2024-03-28T16:49:14.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f642bfff640 1 -- 172.21.15.170:0/372181124 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f653401a770 con 0x55d689f3fda0 2024-03-28T16:49:14.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f642bfff640 1 -- 172.21.15.170:0/372181124 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f653400a040 con 0x55d689f3fda0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f642bfff640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f642c0342b0 0x7f642c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f642bfff640 1 -- 172.21.15.170:0/372181124 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f6534049730 con 0x55d689f3fda0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65433ee640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f642c0342b0 0x7f642c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d689f410e0 0x55d68a0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a0c9890 con 0x55d689f410e0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65443f0640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d689f410e0 0x55d68a0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65433ee640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f642c0342b0 0x7f642c036770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f65380ac000 tx=0x7f65380a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65443f0640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d689f410e0 0x55d68a0c9350 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65443f0640 1 -- 172.21.15.170:0/372181124 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d689f410e0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a0c9bf0 con 0x55d689f410e0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65443f0640 1 -- 172.21.15.170:0/372181124 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d689f410e0 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a0cb910 0x55d68a0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.903+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a0ce240 con 0x55d68a0cb910 2024-03-28T16:49:14.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f6543bef640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a0cb910 0x55d68a0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f6543bef640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a0cb910 0x55d68a0cdd00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f6543bef640 1 -- 172.21.15.170:0/372181124 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f6534014070 con 0x55d68a0cb910 2024-03-28T16:49:14.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a0ca780 con 0x55d689f410e0 2024-03-28T16:49:14.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65443f0640 1 -- 172.21.15.170:0/372181124 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d689f410e0 2024-03-28T16:49:14.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d68a0cf5e0 0x55d68a0d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a0d2070 con 0x55d68a0cf5e0 2024-03-28T16:49:14.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65433ee640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d68a0cf5e0 0x55d68a0d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65433ee640 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d68a0cf5e0 0x55d68a0d19d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65433ee640 1 -- 172.21.15.170:0/372181124 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a0cf5e0 2024-03-28T16:49:14.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65460d4980 1 Processor -- start 2024-03-28T16:49:14.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65460d4980 1 -- start start 2024-03-28T16:49:14.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65460d4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0dffa0 0x55d68a0e03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f65460d4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689e6fe30 con 0x55d68a0dffa0 2024-03-28T16:49:14.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0dffa0 0x55d68a0e03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0dffa0 0x55d68a0e03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55540/0 (socket says 172.21.15.170:55540) 2024-03-28T16:49:14.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f6543bef640 1 -- 172.21.15.170:0/2919126694 learned_addr learned my addr 172.21.15.170:0/2919126694 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.907+0000 7f6543bef640 1 -- 172.21.15.170:0/2919126694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d68a0e0900 con 0x55d68a0dffa0 2024-03-28T16:49:14.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6543bef640 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0dffa0 0x55d68a0e03c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6534047c70 tx=0x7f6534049460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f64297fa640 1 -- 172.21.15.170:0/2919126694 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65340186b0 con 0x55d68a0dffa0 2024-03-28T16:49:14.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d68a0e0b50 con 0x55d68a0dffa0 2024-03-28T16:49:14.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f64297fa640 1 -- 172.21.15.170:0/2919126694 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400d3e0 con 0x55d68a0dffa0 2024-03-28T16:49:14.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f64297fa640 1 -- 172.21.15.170:0/2919126694 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f653400d6a0 con 0x55d68a0dffa0 2024-03-28T16:49:14.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d68a0e16a0 con 0x55d68a0dffa0 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f64297fa640 1 -- 172.21.15.170:0/2919126694 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f653400d800 con 0x55d68a0dffa0 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f64297fa640 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f641c0342b0 0x7f641c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f64297fa640 1 -- 172.21.15.170:0/2919126694 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f6534046c00 con 0x55d68a0dffa0 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65433ee640 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f641c0342b0 0x7f641c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65460d4980 1 Processor -- start 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65460d4980 1 -- start start 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65433ee640 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f641c0342b0 0x7f641c036770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55d68a0c6de0 tx=0x7f6538005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65460d4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0ed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f65460d4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d689e63790 con 0x55d68a0ecee0 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0ed300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55548/0 (socket says 172.21.15.170:55548) 2024-03-28T16:49:14.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6543bef640 1 -- 172.21.15.170:0/3991645785 learned_addr learned my addr 172.21.15.170:0/3991645785 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6543bef640 1 -- 172.21.15.170:0/3991645785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d68a0ed8d0 con 0x55d68a0ecee0 2024-03-28T16:49:14.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6543bef640 1 --2- 172.21.15.170:0/3991645785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0ed300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f653402ba30 tx=0x7f653402b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68067d06e3dc6bf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6422ffd640 1 -- 172.21.15.170:0/3991645785 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f653402b470 con 0x55d68a0ecee0 2024-03-28T16:49:14.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6422ffd640 1 -- 172.21.15.170:0/3991645785 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653400f070 con 0x55d68a0ecee0 2024-03-28T16:49:14.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.911+0000 7f6422ffd640 1 -- 172.21.15.170:0/3991645785 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6534017c90 con 0x55d68a0ecee0 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/3991645785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 msgr2=0x55d68a0ed300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 --2- 172.21.15.170:0/3991645785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0ed300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f653402ba30 tx=0x7f653402b780 comp rx=0 tx=0).stop 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/3991645785 shutdown_connections 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 --2- 172.21.15.170:0/3991645785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0ed300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/3991645785 >> 172.21.15.170:0/3991645785 conn(0x55d68a0e9220 msgr2=0x55d68a0e7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/3991645785 shutdown_connections 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/3991645785 wait complete. 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 Processor -- start 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- start start 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a0f42a0 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f6543bef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55550/0 (socket says 172.21.15.170:55550) 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 learned_addr learned my addr 172.21.15.170:0/373950091 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d68a0e16a0 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f6543bef640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0f3d60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6534023dc0 tx=0x7f653401cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f64217fa640 1 -- 172.21.15.170:0/373950091 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f653400f1e0 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d68a0e16a0 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f64217fa640 1 -- 172.21.15.170:0/373950091 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6534063040 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f64217fa640 1 -- 172.21.15.170:0/373950091 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6534024780 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d68a0f4680 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f64217fa640 1 -- 172.21.15.170:0/373950091 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f65340248e0 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f64217fa640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f640c0342b0 0x7f640c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f64217fa640 1 -- 172.21.15.170:0/373950091 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f6534076810 con 0x55d68a0ecee0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65433ee640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f640c0342b0 0x7f640c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d68a0ff000 0x55d68a1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a101c10 con 0x55d68a0ff000 2024-03-28T16:49:14.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65443f0640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d68a0ff000 0x55d68a1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.915+0000 7f65433ee640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f640c0342b0 0x7f640c036770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6538000c10 tx=0x7f65380a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65443f0640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d68a0ff000 0x55d68a1014c0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 realm 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a101fe0 con 0x55d68a0ff000 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102bc0 con 0x55d68a0ff000 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a103570 0x55d68a105960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a105f70 con 0x55d68a103570 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f6543bef640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a103570 0x55d68a105960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f6543bef640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a103570 0x55d68a105960 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 20 searching for the correct realm 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d68a108150 0x55d68a10a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10ace0 con 0x55d68a108150 2024-03-28T16:49:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65433ee640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d68a108150 0x55d68a10a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65433ee640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d68a108150 0x55d68a10a610 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65433ee640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65380da040 0x7f65380da930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10ace0 con 0x7f65380da040 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.919+0000 7f65443f0640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65380da040 0x7f65380da930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65380da040 0x7f65380da930 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x7f65380da040 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.923+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x7f65380da040 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x7f65380da040 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.927+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x7f65380da040 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a108150 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a103570 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x55d68a0ff000 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a102860 con 0x7f65380da040 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a106550 con 0x55d68a0ff000 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a106550 con 0x55d68a0ff000 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:14.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.931+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a106520 con 0x7f65380da040 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 zone default found 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 4 Realm: () 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 zonegroup default 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a106da0 con 0x55d68a103570 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a1071a0 con 0x55d68a108150 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10cc20 con 0x55d68a108150 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 started sync module instance, tier type = 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10c3e0 con 0x7f65380da040 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94379927715648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10c790 con 0x7f65380da040 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d980 con 0x55d68a0ff000 2024-03-28T16:49:14.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.935+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94379927715648] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94379927720000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d900 con 0x55d68a0ff000 2024-03-28T16:49:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10e3d0 con 0x55d68a108150 2024-03-28T16:49:14.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94379927720000] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94379927721056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d880 con 0x55d68a108150 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10f0c0 con 0x7f65380da040 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94379927727408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10f910 con 0x7f65380da040 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.939+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a1102c0 con 0x55d68a0ff000 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94379927727408] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94379927721056] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94379927725952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10f420 con 0x55d68a0ff000 2024-03-28T16:49:14.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a110b10 con 0x55d68a103570 2024-03-28T16:49:14.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94379927725952] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94379927733872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10f420 con 0x55d68a103570 2024-03-28T16:49:14.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.943+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a111310 con 0x55d68a103570 2024-03-28T16:49:14.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94379927733872] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94379927731680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a111b60 con 0x55d68a103570 2024-03-28T16:49:14.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a1123e0 con 0x55d68a103570 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94379927741344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a112c60 con 0x55d68a103570 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 20 add_watcher() i=0 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 20 add_watcher() i=1 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 20 add_watcher() i=2 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 20 add_watcher() i=3 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 20 add_watcher() i=4 2024-03-28T16:49:14.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 20 add_watcher() i=5 2024-03-28T16:49:14.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94379927731680] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.947+0000 7f65460d4980 20 add_watcher() i=6 2024-03-28T16:49:14.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94379927741344] v29'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 add_watcher() i=7 2024-03-28T16:49:14.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:14.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a112c60 con 0x55d68a0ff000 2024-03-28T16:49:14.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94379927742400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a112be0 con 0x55d68a0ff000 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94379927742400] v29'64 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f63f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 5 note: GC not initialized 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f63f8fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 Filter name: none 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f63727fc640 20 reqs_thread_entry: start 2024-03-28T16:49:14.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a137c50 con 0x55d68a0ff000 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:14.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.951+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a138150 con 0x55d68a103570 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~459 in=459b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a13da80 con 0x55d68a0ff000 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~459] v29'4 uv4 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94379927917984 in=647b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a13a610 con 0x55d68a0ff000 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94379927725952 notify 124554051587 ret 0) v3 ==== 677+0+0 (crc 0 0 0) 0x7f653c04a040 con 0x55d68a0ff000 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f6422ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051587 cookie 94379927725952 notifier 4333 bl.length()=635 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f6422ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(65 notify.4 [notify cookie 94379927917984 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f6422ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f6422ffd640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6408003ea0 con 0x55d68a0ff000 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94379927917984 notify 124554051587 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f653c018ab0 con 0x55d68a0ff000 2024-03-28T16:49:14.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.955+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a13d5a0 con 0x55d68a103570 2024-03-28T16:49:14.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94379927917984 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a13a610 con 0x55d68a108150 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== watch-notify(notify (1) cookie 94379927721056 notify 124554051584 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7f65380d3070 con 0x55d68a108150 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f6422ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051584 cookie 94379927721056 notifier 4333 bl.length()=177 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f6422ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f6422ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(68 notify.2 [notify cookie 94379927917984 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f6422ffd640 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6408003ea0 con 0x55d68a108150 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== watch-notify(notify_complete (2) cookie 94379927917984 notify 124554051584 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55d689f401b0 con 0x55d68a108150 2024-03-28T16:49:14.959 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:14.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f63f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:14.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94379927742400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a136cd0 con 0x55d68a0ff000 2024-03-28T16:49:14.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94379927742400] v29'65 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94379927715648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a1127c0 con 0x7f65380da040 2024-03-28T16:49:14.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94379927715648] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 20 remove_watcher() i=0 2024-03-28T16:49:14.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 2 removed watcher, disabling cache 2024-03-28T16:49:14.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.959+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94379927720000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a1127c0 con 0x55d68a0ff000 2024-03-28T16:49:14.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.963+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94379927720000] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.963+0000 7f65460d4980 20 remove_watcher() i=1 2024-03-28T16:49:14.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.963+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94379927721056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d040 con 0x55d68a108150 2024-03-28T16:49:14.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.963+0000 7f65433ee640 1 -- 172.21.15.170:0/373950091 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94379927721056] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65380aaaf0 con 0x55d68a108150 2024-03-28T16:49:14.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.963+0000 7f65460d4980 20 remove_watcher() i=2 2024-03-28T16:49:14.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.963+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94379927727408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d4e0 con 0x7f65380da040 2024-03-28T16:49:14.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.967+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94379927727408] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x7f65380da040 2024-03-28T16:49:14.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.967+0000 7f65460d4980 20 remove_watcher() i=3 2024-03-28T16:49:14.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.967+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94379927725952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d4e0 con 0x55d68a0ff000 2024-03-28T16:49:14.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.967+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94379927725952] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653c00a4d0 con 0x55d68a0ff000 2024-03-28T16:49:14.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.967+0000 7f65460d4980 20 remove_watcher() i=4 2024-03-28T16:49:14.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.967+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94379927733872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d4e0 con 0x55d68a103570 2024-03-28T16:49:14.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.971+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94379927733872] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.971+0000 7f65460d4980 20 remove_watcher() i=5 2024-03-28T16:49:14.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.971+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94379927731680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d4e0 con 0x55d68a103570 2024-03-28T16:49:14.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.971+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94379927731680] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.971+0000 7f65460d4980 20 remove_watcher() i=6 2024-03-28T16:49:14.972 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.971+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94379927741344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d68a10d4e0 con 0x55d68a103570 2024-03-28T16:49:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f6543bef640 1 -- 172.21.15.170:0/373950091 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94379927741344] v29'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f653406d020 con 0x55d68a103570 2024-03-28T16:49:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 20 remove_watcher() i=7 2024-03-28T16:49:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a103570 msgr2=0x55d68a105960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a103570 0x55d68a105960 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d68a0ff000 msgr2=0x55d68a1014c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d68a0ff000 0x55d68a1014c0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65380da040 msgr2=0x7f65380da930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65380da040 0x7f65380da930 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d68a108150 msgr2=0x55d68a10a610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d68a108150 0x55d68a10a610 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f640c0342b0 msgr2=0x7f640c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f640c0342b0 0x7f640c036770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6538000c10 tx=0x7f65380a7000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 msgr2=0x55d68a0f3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/373950091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0ecee0 0x55d68a0f3d60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6534023dc0 tx=0x7f653401cf40 comp rx=0 tx=0).stop 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 reap_dead start 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65443f0640 1 -- 172.21.15.170:0/373950091 reap_dead start 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 shutdown_connections 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 >> 172.21.15.170:0/373950091 conn(0x55d68a0e9220 msgr2=0x55d68a0e6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 shutdown_connections 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/373950091 wait complete. 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f641c0342b0 msgr2=0x7f641c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f641c0342b0 0x7f641c036770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55d68a0c6de0 tx=0x7f6538005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0dffa0 msgr2=0x55d68a0e03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0dffa0 0x55d68a0e03c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6534047c70 tx=0x7f6534049460 comp rx=0 tx=0).stop 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 shutdown_connections 2024-03-28T16:49:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f641c0342b0 0x7f641c036770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/2919126694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d68a0dffa0 0x55d68a0e03c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 >> 172.21.15.170:0/2919126694 conn(0x55d68a0d4db0 msgr2=0x55d68a0d58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 shutdown_connections 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/2919126694 wait complete. 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a0cb910 msgr2=0x55d68a0cdd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d68a0cb910 0x55d68a0cdd00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d689f410e0 msgr2=0x55d68a0c9350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d689f410e0 0x55d68a0c9350 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d68a0cf5e0 msgr2=0x55d68a0d19d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d68a0cf5e0 0x55d68a0d19d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f642c0342b0 msgr2=0x7f642c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f642c0342b0 0x7f642c036770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f65380ac000 tx=0x7f65380a5000 comp rx=0 tx=0).stop 2024-03-28T16:49:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 msgr2=0x55d68a0c2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 --2- 172.21.15.170:0/372181124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d689f3fda0 0x55d68a0c2770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6534001ee0 tx=0x7f6534000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65443f0640 1 -- 172.21.15.170:0/372181124 reap_dead start 2024-03-28T16:49:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 shutdown_connections 2024-03-28T16:49:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 >> 172.21.15.170:0/372181124 conn(0x55d689fc67b0 msgr2=0x55d68a0b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 shutdown_connections 2024-03-28T16:49:14.980 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:14.975+0000 7f65460d4980 1 -- 172.21.15.170:0/372181124 wait complete. 2024-03-28T16:49:14.983 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:14.984 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:14.984 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:14.984 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:15.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b7d58980 1 Processor -- start 2024-03-28T16:49:15.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b7d58980 1 -- start start 2024-03-28T16:49:15.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b7d58980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55fd4add7310 0x55fd4add9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b7d58980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fd4ad1f520 con 0x55fd4ad478f0 2024-03-28T16:49:15.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b7d58980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fd4ace60f0 con 0x55fd4add7310 2024-03-28T16:49:15.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b5072640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55fd4add7310 0x55fd4add9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b5072640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55fd4add7310 0x55fd4add9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55552/0 (socket says 172.21.15.170:55552) 2024-03-28T16:49:15.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b5873640 1 --1- 172.21.15.170:0/701727542 >> v1:172.21.15.119:6789/0 conn(0x55fd4ad478f0 0x55fd4add5bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60886/0 (socket says 172.21.15.170:60886) 2024-03-28T16:49:15.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b5072640 1 -- 172.21.15.170:0/701727542 learned_addr learned my addr 172.21.15.170:0/701727542 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b5072640 1 -- 172.21.15.170:0/701727542 >> v1:172.21.15.119:6789/0 conn(0x55fd4ad478f0 legacy=0x55fd4add5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.059+0000 7fb2b5072640 1 -- 172.21.15.170:0/701727542 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4addace0 con 0x55fd4add7310 2024-03-28T16:49:15.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b5072640 1 --2- 172.21.15.170:0/701727542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4add7310 0x55fd4add9700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000a4e0 tx=0x7fb2a001b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fd61a93c3b0d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b4871640 1 -- 172.21.15.170:0/701727542 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a00aa540 con 0x55fd4add7310 2024-03-28T16:49:15.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b4871640 1 -- 172.21.15.170:0/701727542 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a00a6070 con 0x55fd4add7310 2024-03-28T16:49:15.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b4871640 1 -- 172.21.15.170:0/701727542 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a00aa850 con 0x55fd4add7310 2024-03-28T16:49:15.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b7d58980 1 -- 172.21.15.170:0/701727542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4add7310 msgr2=0x55fd4add9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/701727542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4add7310 0x55fd4add9700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000a4e0 tx=0x7fb2a001b090 comp rx=0 tx=0).stop 2024-03-28T16:49:15.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b7d58980 1 -- 172.21.15.170:0/701727542 shutdown_connections 2024-03-28T16:49:15.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/701727542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4add7310 0x55fd4add9700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b7d58980 1 -- 172.21.15.170:0/701727542 >> 172.21.15.170:0/701727542 conn(0x55fd4adcd710 msgr2=0x55fd4adcfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b7d58980 1 -- 172.21.15.170:0/701727542 shutdown_connections 2024-03-28T16:49:15.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.063+0000 7fb2b7d58980 1 -- 172.21.15.170:0/701727542 wait complete. 2024-03-28T16:49:15.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.075+0000 7fb2b7d58980 1 Processor -- start 2024-03-28T16:49:15.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.075+0000 7fb2b7d58980 1 -- start start 2024-03-28T16:49:15.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.075+0000 7fb2b7d58980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aebce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.075+0000 7fb2b7d58980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4ace3620 con 0x55fd4ad478f0 2024-03-28T16:49:15.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.075+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aebce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.075+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aebce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55560/0 (socket says 172.21.15.170:55560) 2024-03-28T16:49:15.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.075+0000 7fb2b5873640 1 -- 172.21.15.170:0/4149811963 learned_addr learned my addr 172.21.15.170:0/4149811963 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b5873640 1 -- 172.21.15.170:0/4149811963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a000a1c0 con 0x55fd4ad478f0 2024-03-28T16:49:15.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b5873640 1 --2- 172.21.15.170:0/4149811963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aebce60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8001cc0 tx=0x7fb2a800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cb6df5be12ff04 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb1a2e15640 1 -- 172.21.15.170:0/4149811963 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a8014070 con 0x55fd4ad478f0 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb1a2e15640 1 -- 172.21.15.170:0/4149811963 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a800f590 con 0x55fd4ad478f0 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb1a2e15640 1 -- 172.21.15.170:0/4149811963 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a8007040 con 0x55fd4ad478f0 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4149811963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 msgr2=0x55fd4aebce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/4149811963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aebce60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8001cc0 tx=0x7fb2a800b610 comp rx=0 tx=0).stop 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4149811963 shutdown_connections 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/4149811963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aebce60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4149811963 >> 172.21.15.170:0/4149811963 conn(0x55fd4adcd710 msgr2=0x55fd4aeb89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4149811963 shutdown_connections 2024-03-28T16:49:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4149811963 wait complete. 2024-03-28T16:49:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 Processor -- start 2024-03-28T16:49:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- start start 2024-03-28T16:49:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aec3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4acdee00 con 0x55fd4ad478f0 2024-03-28T16:49:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aec3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aec3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55566/0 (socket says 172.21.15.170:55566) 2024-03-28T16:49:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b5873640 1 -- 172.21.15.170:0/658366363 learned_addr learned my addr 172.21.15.170:0/658366363 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b5873640 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4ad1f520 con 0x55fd4ad478f0 2024-03-28T16:49:15.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b5873640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aec3cb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55fd4aebdcf0 tx=0x7fb2a8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb1a1612640 1 -- 172.21.15.170:0/658366363 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a8014030 con 0x55fd4ad478f0 2024-03-28T16:49:15.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fd4aec41f0 con 0x55fd4ad478f0 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb1a1612640 1 -- 172.21.15.170:0/658366363 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a801c840 con 0x55fd4ad478f0 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb1a1612640 1 -- 172.21.15.170:0/658366363 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a801cb40 con 0x55fd4ad478f0 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.079+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd4aec4620 con 0x55fd4ad478f0 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb1a1612640 1 -- 172.21.15.170:0/658366363 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb2a801cca0 con 0x55fd4ad478f0 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb1a1612640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb19c034300 0x7fb19c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb1a1612640 1 -- 172.21.15.170:0/658366363 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb2a8006070 con 0x55fd4ad478f0 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b5072640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb19c034300 0x7fb19c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4add7310 0x55fd4aeca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4aecae70 con 0x55fd4add7310 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b6074640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4add7310 0x55fd4aeca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b5072640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb19c034300 0x7fb19c0367c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00ab000 tx=0x7fb2a00a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b6074640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4add7310 0x55fd4aeca930 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b6074640 1 -- 172.21.15.170:0/658366363 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4add7310 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4aecb1d0 con 0x55fd4add7310 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b6074640 1 -- 172.21.15.170:0/658366363 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4add7310 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4aeccef0 0x55fd4aecf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4aecf820 con 0x55fd4aeccef0 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b5873640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4aeccef0 0x55fd4aecf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.083+0000 7fb2b5873640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4aeccef0 0x55fd4aecf2e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5873640 1 -- 172.21.15.170:0/658366363 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fb2a8016070 con 0x55fd4aeccef0 2024-03-28T16:49:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4aecbd30 con 0x55fd4add7310 2024-03-28T16:49:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b6074640 1 -- 172.21.15.170:0/658366363 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4add7310 2024-03-28T16:49:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fd4aed0b90 0x55fd4aed2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4aed3620 con 0x55fd4aed0b90 2024-03-28T16:49:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5072640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fd4aed0b90 0x55fd4aed2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5072640 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fd4aed0b90 0x55fd4aed2f80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5072640 1 -- 172.21.15.170:0/658366363 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4aed0b90 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 Processor -- start 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 -- start start 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aee1530 0x55fd4aee1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4ac77e30 con 0x55fd4aee1530 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aee1530 0x55fd4aee1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aee1530 0x55fd4aee1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55568/0 (socket says 172.21.15.170:55568) 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5873640 1 -- 172.21.15.170:0/2322249918 learned_addr learned my addr 172.21.15.170:0/2322249918 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5873640 1 -- 172.21.15.170:0/2322249918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4aee1e90 con 0x55fd4aee1530 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b5873640 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aee1530 0x55fd4aee1950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8000f70 tx=0x7fb2a80187d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb19a7fc640 1 -- 172.21.15.170:0/2322249918 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a8017c90 con 0x55fd4aee1530 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb19a7fc640 1 -- 172.21.15.170:0/2322249918 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a8017df0 con 0x55fd4aee1530 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fd4aee20e0 con 0x55fd4aee1530 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb19a7fc640 1 -- 172.21.15.170:0/2322249918 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a80419d0 con 0x55fd4aee1530 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.087+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd4aee2c30 con 0x55fd4aee1530 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb19a7fc640 1 -- 172.21.15.170:0/2322249918 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb2a8049bc0 con 0x55fd4aee1530 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb19a7fc640 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb18c0342b0 0x7fb18c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb19a7fc640 1 -- 172.21.15.170:0/2322249918 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb2a803db00 con 0x55fd4aee1530 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5072640 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb18c0342b0 0x7fb18c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 Processor -- start 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- start start 2024-03-28T16:49:15.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aeee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4ac6b790 con 0x55fd4aeee490 2024-03-28T16:49:15.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5072640 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb18c0342b0 0x7fb18c036770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55fd4aec8360 tx=0x7fb2a0005e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aeee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aeee8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55570/0 (socket says 172.21.15.170:55570) 2024-03-28T16:49:15.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 -- 172.21.15.170:0/4143150909 learned_addr learned my addr 172.21.15.170:0/4143150909 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 -- 172.21.15.170:0/4143150909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4aeeee80 con 0x55fd4aeee490 2024-03-28T16:49:15.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 --2- 172.21.15.170:0/4143150909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aeee8b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800ffd0 tx=0x7fb2a80195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff572fff6ccf27e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb18bfff640 1 -- 172.21.15.170:0/4143150909 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a8045da0 con 0x55fd4aeee490 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb18bfff640 1 -- 172.21.15.170:0/4143150909 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a8031040 con 0x55fd4aeee490 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb18bfff640 1 -- 172.21.15.170:0/4143150909 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a8041210 con 0x55fd4aeee490 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4143150909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 msgr2=0x55fd4aeee8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/4143150909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aeee8b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800ffd0 tx=0x7fb2a80195a0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4143150909 shutdown_connections 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/4143150909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aeee8b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4143150909 >> 172.21.15.170:0/4143150909 conn(0x55fd4aeea7d0 msgr2=0x55fd4aee8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4143150909 shutdown_connections 2024-03-28T16:49:15.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- 172.21.15.170:0/4143150909 wait complete. 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 Processor -- start 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- start start 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aef53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b7d58980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd4aef58e0 con 0x55fd4aeee490 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aef53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aef53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55576/0 (socket says 172.21.15.170:55576) 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 learned_addr learned my addr 172.21.15.170:0/3071601534 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.091+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd4aee2c30 con 0x55fd4aeee490 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b5873640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aef53a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80195e0 tx=0x7fb2a8031320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb18a7fc640 1 -- 172.21.15.170:0/3071601534 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a80311b0 con 0x55fd4aeee490 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb18a7fc640 1 -- 172.21.15.170:0/3071601534 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a8062040 con 0x55fd4aeee490 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fd4aee2c30 con 0x55fd4aeee490 2024-03-28T16:49:15.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb18a7fc640 1 -- 172.21.15.170:0/3071601534 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2a8041210 con 0x55fd4aeee490 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd4aee2c30 con 0x55fd4aeee490 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb18a7fc640 1 -- 172.21.15.170:0/3071601534 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb2a801b3d0 con 0x55fd4aeee490 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb18a7fc640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb1800342b0 0x7fb180036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb18a7fc640 1 -- 172.21.15.170:0/3071601534 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb2a8075810 con 0x55fd4aeee490 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b5072640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb1800342b0 0x7fb180036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4af00480 0x55fd4af02940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03090 con 0x55fd4af00480 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b6074640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4af00480 0x55fd4af02940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b5072640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb1800342b0 0x7fb180036770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00aafd0 tx=0x7fb2a00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b6074640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4af00480 0x55fd4af02940 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 realm 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03460 con 0x55fd4af00480 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af04010 con 0x55fd4af00480 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4af049c0 0x55fd4af06db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.095+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af07330 con 0x55fd4af049c0 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5873640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4af049c0 0x55fd4af06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5873640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4af049c0 0x55fd4af06db0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b7d58980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:15.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b7d58980 20 searching for the correct realm 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fd4af09230 0x55fd4af0b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0bd50 con 0x55fd4af09230 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5072640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fd4af09230 0x55fd4af0b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5072640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fd4af09230 0x55fd4af0b680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5072640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2a00cd050 0x7fb2a00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0bd50 con 0x7fb2a00cd050 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b6074640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2a00cd050 0x7fb2a00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b6074640 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2a00cd050 0x7fb2a00cd430 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.099+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x7fb2a00cd050 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x7fb2a00cd050 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.103+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x7fb2a00cd050 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.107+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x7fb2a00cd050 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af09230 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af049c0 2024-03-28T16:49:15.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x55fd4af00480 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af03cb0 con 0x7fb2a00cd050 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af07ba0 con 0x55fd4af00480 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af07ba0 con 0x55fd4af00480 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af07b70 con 0x7fb2a00cd050 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 zone default found 2024-03-28T16:49:15.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 4 Realm: () 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 zonegroup default 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.111+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af083f0 con 0x55fd4af049c0 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af087d0 con 0x55fd4af09230 2024-03-28T16:49:15.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0dba0 con 0x55fd4af09230 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 20 started sync module instance, tier type = 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0d450 con 0x7fb2a00cd050 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94546372382640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0d800 con 0x7fb2a00cd050 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0e900 con 0x55fd4af00480 2024-03-28T16:49:15.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.115+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94546372382640] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94546372386752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0e880 con 0x55fd4af00480 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0f0c0 con 0x55fd4af09230 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94546372386752] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94546372388080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0f470 con 0x55fd4af09230 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0fde0 con 0x7fb2a00cd050 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94546372393552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af10490 con 0x7fb2a00cd050 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.119+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af10f60 con 0x55fd4af00480 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94546372388080] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94546372393552] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94546372397632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af10f20 con 0x55fd4af00480 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af10060 con 0x55fd4af049c0 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94546372397632] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94546372389200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af117b0 con 0x55fd4af049c0 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.123+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af11fb0 con 0x55fd4af049c0 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94546372389200] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94546372399888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af127b0 con 0x55fd4af049c0 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af13120 con 0x55fd4af049c0 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94546372408448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af13d40 con 0x55fd4af049c0 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 20 add_watcher() i=0 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 20 add_watcher() i=1 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 20 add_watcher() i=2 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 20 add_watcher() i=3 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 20 add_watcher() i=4 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 20 add_watcher() i=5 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94546372399888] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.127+0000 7fb2b7d58980 20 add_watcher() i=6 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94546372408448] v29'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 20 add_watcher() i=7 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af123e0 con 0x55fd4af00480 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94546372409504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af145d0 con 0x55fd4af00480 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94546372409504] v29'66 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb16afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 5 note: GC not initialized 2024-03-28T16:49:15.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb169fe3640 20 reqs_thread_entry: start 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 20 Filter name: none 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb0db7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.131+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af386d0 con 0x55fd4af00480 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b7d58980 20 rados_obj.operate() r=0 bl.length=459 2024-03-28T16:49:15.137 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb16afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94546372409504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af376d0 con 0x55fd4af00480 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94546372409504] v29'67 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94546372382640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af138a0 con 0x7fb2a00cd050 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94546372382640] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b7d58980 20 remove_watcher() i=0 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b7d58980 2 removed watcher, disabling cache 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.135+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94546372386752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af138a0 con 0x55fd4af00480 2024-03-28T16:49:15.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94546372386752] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b7d58980 20 remove_watcher() i=1 2024-03-28T16:49:15.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94546372388080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0dfc0 con 0x55fd4af09230 2024-03-28T16:49:15.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b5072640 1 -- 172.21.15.170:0/3071601534 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94546372388080] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a00aaa00 con 0x55fd4af09230 2024-03-28T16:49:15.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b7d58980 20 remove_watcher() i=2 2024-03-28T16:49:15.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94546372393552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af138a0 con 0x7fb2a00cd050 2024-03-28T16:49:15.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94546372393552] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x7fb2a00cd050 2024-03-28T16:49:15.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b7d58980 20 remove_watcher() i=3 2024-03-28T16:49:15.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.139+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94546372397632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0fa50 con 0x55fd4af00480 2024-03-28T16:49:15.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.143+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94546372397632] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2b00083f0 con 0x55fd4af00480 2024-03-28T16:49:15.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.143+0000 7fb2b7d58980 20 remove_watcher() i=4 2024-03-28T16:49:15.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.143+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94546372389200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af138a0 con 0x55fd4af049c0 2024-03-28T16:49:15.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.143+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94546372389200] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.143+0000 7fb2b7d58980 20 remove_watcher() i=5 2024-03-28T16:49:15.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.143+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94546372399888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0e990 con 0x55fd4af049c0 2024-03-28T16:49:15.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.147+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94546372399888] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.147+0000 7fb2b7d58980 20 remove_watcher() i=6 2024-03-28T16:49:15.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.147+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94546372408448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fd4af0fa50 con 0x55fd4af049c0 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.147+0000 7fb2b5873640 1 -- 172.21.15.170:0/3071601534 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94546372408448] v29'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2a806c020 con 0x55fd4af049c0 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 20 remove_watcher() i=7 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4af049c0 msgr2=0x55fd4af06db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4af049c0 0x55fd4af06db0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4af00480 msgr2=0x55fd4af02940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4af00480 0x55fd4af02940 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2a00cd050 msgr2=0x7fb2a00cd430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2a00cd050 0x7fb2a00cd430 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fd4af09230 msgr2=0x55fd4af0b680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fd4af09230 0x55fd4af0b680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb1800342b0 msgr2=0x7fb180036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb1800342b0 0x7fb180036770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00aafd0 tx=0x7fb2a00a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 msgr2=0x55fd4aef53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b6074640 1 -- 172.21.15.170:0/3071601534 reap_dead start 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aef53a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80195e0 tx=0x7fb2a8031320 comp rx=0 tx=0).stop 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 shutdown_connections 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/3071601534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aeee490 0x55fd4aef53a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 >> 172.21.15.170:0/3071601534 conn(0x55fd4aeea7d0 msgr2=0x55fd4aee7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 shutdown_connections 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/3071601534 wait complete. 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb18c0342b0 msgr2=0x7fb18c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb18c0342b0 0x7fb18c036770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x55fd4aec8360 tx=0x7fb2a0005e50 comp rx=0 tx=0).stop 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aee1530 msgr2=0x55fd4aee1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aee1530 0x55fd4aee1950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8000f70 tx=0x7fb2a80187d0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 shutdown_connections 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb18c0342b0 0x7fb18c036770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/2322249918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4aee1530 0x55fd4aee1950 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 >> 172.21.15.170:0/2322249918 conn(0x55fd4aed6340 msgr2=0x55fd4aed6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 shutdown_connections 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/2322249918 wait complete. 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4aeccef0 msgr2=0x55fd4aecf2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fd4aeccef0 0x55fd4aecf2e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4add7310 msgr2=0x55fd4aeca930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fd4add7310 0x55fd4aeca930 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fd4aed0b90 msgr2=0x55fd4aed2f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fd4aed0b90 0x55fd4aed2f80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb19c034300 msgr2=0x7fb19c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb19c034300 0x7fb19c0367c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00ab000 tx=0x7fb2a00a5000 comp rx=0 tx=0).stop 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 msgr2=0x55fd4aec3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 --2- 172.21.15.170:0/658366363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fd4ad478f0 0x55fd4aec3cb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55fd4aebdcf0 tx=0x7fb2a8000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b6074640 1 -- 172.21.15.170:0/658366363 reap_dead start 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 shutdown_connections 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 >> 172.21.15.170:0/658366363 conn(0x55fd4adcd710 msgr2=0x55fd4aeb7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 shutdown_connections 2024-03-28T16:49:15.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.151+0000 7fb2b7d58980 1 -- 172.21.15.170:0/658366363 wait complete. 2024-03-28T16:49:15.160 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:15.160 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-03-28T16:49:15.160 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-03-28T16:49:15.160 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-03-28T16:49:15.228 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:15.229 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f9065685980 1 Processor -- start 2024-03-28T16:49:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f9065685980 1 -- start start 2024-03-28T16:49:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f9065685980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55887c515370 0x55887c517760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f9065685980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55887c45e4c0 con 0x55887c510e10 2024-03-28T16:49:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f9065685980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55887c4250f0 con 0x55887c515370 2024-03-28T16:49:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f906299f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55887c515370 0x55887c517760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f90631a0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55887c510e10 0x55887c5111f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60902/0 (socket says 172.21.15.170:60902) 2024-03-28T16:49:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f906299f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55887c515370 0x55887c517760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55592/0 (socket says 172.21.15.170:55592) 2024-03-28T16:49:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f90631a0640 1 -- 172.21.15.170:0/3484424450 learned_addr learned my addr 172.21.15.170:0/3484424450 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f906299f640 1 -- 172.21.15.170:0/3484424450 >> v1:172.21.15.119:6789/0 conn(0x55887c510e10 legacy=0x55887c5111f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f906299f640 1 -- 172.21.15.170:0/3484424450 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887c518cb0 con 0x55887c515370 2024-03-28T16:49:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f906299f640 1 --2- 172.21.15.170:0/3484424450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c515370 0x55887c517760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f905800a480 tx=0x7f905801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13214c6111b60c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.231+0000 7f906219e640 1 -- 172.21.15.170:0/3484424450 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90580ab540 con 0x55887c515370 2024-03-28T16:49:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f906219e640 1 -- 172.21.15.170:0/3484424450 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90580ab6a0 con 0x55887c515370 2024-03-28T16:49:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f906219e640 1 -- 172.21.15.170:0/3484424450 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90580ab960 con 0x55887c515370 2024-03-28T16:49:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f9065685980 1 -- 172.21.15.170:0/3484424450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c515370 msgr2=0x55887c517760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f9065685980 1 --2- 172.21.15.170:0/3484424450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c515370 0x55887c517760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f905800a480 tx=0x7f905801b280 comp rx=0 tx=0).stop 2024-03-28T16:49:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f9065685980 1 -- 172.21.15.170:0/3484424450 shutdown_connections 2024-03-28T16:49:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f9065685980 1 --2- 172.21.15.170:0/3484424450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c515370 0x55887c517760 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f9065685980 1 -- 172.21.15.170:0/3484424450 >> 172.21.15.170:0/3484424450 conn(0x55887c50c6e0 msgr2=0x55887c50eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f9065685980 1 -- 172.21.15.170:0/3484424450 shutdown_connections 2024-03-28T16:49:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.235+0000 7f9065685980 1 -- 172.21.15.170:0/3484424450 wait complete. 2024-03-28T16:49:15.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 Processor -- start 2024-03-28T16:49:15.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 -- start start 2024-03-28T16:49:15.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c5f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887c422620 con 0x55887c510e10 2024-03-28T16:49:15.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c5f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c5f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55602/0 (socket says 172.21.15.170:55602) 2024-03-28T16:49:15.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f90631a0640 1 -- 172.21.15.170:0/3435148023 learned_addr learned my addr 172.21.15.170:0/3435148023 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f90631a0640 1 -- 172.21.15.170:0/3435148023 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058005d30 con 0x55887c510e10 2024-03-28T16:49:15.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f90631a0640 1 --2- 172.21.15.170:0/3435148023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c5f7db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9054001c90 tx=0x7f905400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9203653371c6fcb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f8f5160a640 1 -- 172.21.15.170:0/3435148023 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f905400e690 con 0x55887c510e10 2024-03-28T16:49:15.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f8f5160a640 1 -- 172.21.15.170:0/3435148023 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905400e7f0 con 0x55887c510e10 2024-03-28T16:49:15.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f8f5160a640 1 -- 172.21.15.170:0/3435148023 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f905400eaf0 con 0x55887c510e10 2024-03-28T16:49:15.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 -- 172.21.15.170:0/3435148023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 msgr2=0x55887c5f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 --2- 172.21.15.170:0/3435148023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c5f7db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9054001c90 tx=0x7f905400b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 -- 172.21.15.170:0/3435148023 shutdown_connections 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 --2- 172.21.15.170:0/3435148023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c5f7db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 -- 172.21.15.170:0/3435148023 >> 172.21.15.170:0/3435148023 conn(0x55887c50c6e0 msgr2=0x55887c4859d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 -- 172.21.15.170:0/3435148023 shutdown_connections 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.247+0000 7f9065685980 1 -- 172.21.15.170:0/3435148023 wait complete. 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 Processor -- start 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 -- start start 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c604fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887c41de00 con 0x55887c510e10 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c604fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c604fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55604/0 (socket says 172.21.15.170:55604) 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f90631a0640 1 -- 172.21.15.170:0/2262259853 learned_addr learned my addr 172.21.15.170:0/2262259853 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f90631a0640 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887c45e4c0 con 0x55887c510e10 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f90631a0640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c604fc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9054001ee0 tx=0x7f9054000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f8f4fe07640 1 -- 172.21.15.170:0/2262259853 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f905401f070 con 0x55887c510e10 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55887c605500 con 0x55887c510e10 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f8f4fe07640 1 -- 172.21.15.170:0/2262259853 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905401a4d0 con 0x55887c510e10 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55887c605930 con 0x55887c510e10 2024-03-28T16:49:15.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f8f4fe07640 1 -- 172.21.15.170:0/2262259853 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f905401a770 con 0x55887c510e10 2024-03-28T16:49:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f8f4fe07640 1 -- 172.21.15.170:0/2262259853 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f905400a040 con 0x55887c510e10 2024-03-28T16:49:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f8f4fe07640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f480342b0 0x7f8f48036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f8f4fe07640 1 -- 172.21.15.170:0/2262259853 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f9054049670 con 0x55887c510e10 2024-03-28T16:49:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f906299f640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f480342b0 0x7f8f48036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c515370 0x55887c60bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c60c180 con 0x55887c515370 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f90639a1640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c515370 0x55887c60bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f906299f640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f480342b0 0x7f8f48036770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f90580ad000 tx=0x7f90580a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.251+0000 7f90639a1640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c515370 0x55887c60bc40 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f90639a1640 1 -- 172.21.15.170:0/2262259853 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c515370 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c60c4e0 con 0x55887c515370 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f90639a1640 1 -- 172.21.15.170:0/2262259853 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c515370 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c60e200 0x55887c6105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c610b30 con 0x55887c60e200 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f90631a0640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c60e200 0x55887c6105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f90631a0640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c60e200 0x55887c6105f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f90631a0640 1 -- 172.21.15.170:0/2262259853 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f9054014070 con 0x55887c60e200 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c60d040 con 0x55887c515370 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f90639a1640 1 -- 172.21.15.170:0/2262259853 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c515370 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55887c611ea0 0x55887c614290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6148f0 con 0x55887c611ea0 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f906299f640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55887c611ea0 0x55887c614290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f906299f640 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55887c611ea0 0x55887c614290 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f906299f640 1 -- 172.21.15.170:0/2262259853 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c611ea0 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 Processor -- start 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 -- start start 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c622820 0x55887c622c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.255+0000 7f9065685980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887c3b6e30 con 0x55887c622820 2024-03-28T16:49:15.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c622820 0x55887c622c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c622820 0x55887c622c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55618/0 (socket says 172.21.15.170:55618) 2024-03-28T16:49:15.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 -- 172.21.15.170:0/3151938137 learned_addr learned my addr 172.21.15.170:0/3151938137 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 -- 172.21.15.170:0/3151938137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887c623180 con 0x55887c622820 2024-03-28T16:49:15.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c622820 0x55887c622c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9054047c70 tx=0x7f905401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f8f4d602640 1 -- 172.21.15.170:0/3151938137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9054046040 con 0x55887c622820 2024-03-28T16:49:15.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f8f4d602640 1 -- 172.21.15.170:0/3151938137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9054017910 con 0x55887c622820 2024-03-28T16:49:15.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55887c623370 con 0x55887c622820 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f8f4d602640 1 -- 172.21.15.170:0/3151938137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9054017bb0 con 0x55887c622820 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55887c623370 con 0x55887c622820 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f8f4d602640 1 -- 172.21.15.170:0/3151938137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9054047d50 con 0x55887c622820 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f8f4d602640 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f3c0342b0 0x7f8f3c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f8f4d602640 1 -- 172.21.15.170:0/3151938137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f9054047110 con 0x55887c622820 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f906299f640 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f3c0342b0 0x7f8f3c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f9065685980 1 Processor -- start 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f9065685980 1 -- start start 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f906299f640 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f3c0342b0 0x7f8f3c036770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55887c609670 tx=0x7f9058005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f9065685980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c62f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f9065685980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887c3aa790 con 0x55887c62f3a0 2024-03-28T16:49:15.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c62f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c62f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55624/0 (socket says 172.21.15.170:55624) 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 -- 172.21.15.170:0/1077725853 learned_addr learned my addr 172.21.15.170:0/1077725853 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.259+0000 7f90631a0640 1 -- 172.21.15.170:0/1077725853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55887c62fd90 con 0x55887c62f3a0 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f90631a0640 1 --2- 172.21.15.170:0/1077725853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c62f7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f905402da90 tx=0x7f9054023720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c754f78a193e88d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f8f46ffd640 1 -- 172.21.15.170:0/1077725853 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9054059070 con 0x55887c62f3a0 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f8f46ffd640 1 -- 172.21.15.170:0/1077725853 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9054041a00 con 0x55887c62f3a0 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f8f46ffd640 1 -- 172.21.15.170:0/1077725853 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9054017d10 con 0x55887c62f3a0 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- 172.21.15.170:0/1077725853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 msgr2=0x55887c62f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 --2- 172.21.15.170:0/1077725853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c62f7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f905402da90 tx=0x7f9054023720 comp rx=0 tx=0).stop 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- 172.21.15.170:0/1077725853 shutdown_connections 2024-03-28T16:49:15.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 --2- 172.21.15.170:0/1077725853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c62f7c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- 172.21.15.170:0/1077725853 >> 172.21.15.170:0/1077725853 conn(0x55887c62b730 msgr2=0x55887c629790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- 172.21.15.170:0/1077725853 shutdown_connections 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- 172.21.15.170:0/1077725853 wait complete. 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 Processor -- start 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- start start 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c636250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887c636790 con 0x55887c62f3a0 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c636250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f90631a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c636250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55634/0 (socket says 172.21.15.170:55634) 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 learned_addr learned my addr 172.21.15.170:0/2324271969 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905403d030 con 0x55887c62f3a0 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f90631a0640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c636250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9054041bb0 tx=0x7f9054046d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f8f457fa640 1 -- 172.21.15.170:0/2324271969 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f905400fe40 con 0x55887c62f3a0 2024-03-28T16:49:15.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f8f457fa640 1 -- 172.21.15.170:0/2324271969 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9054046820 con 0x55887c62f3a0 2024-03-28T16:49:15.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55887c636990 con 0x55887c62f3a0 2024-03-28T16:49:15.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f8f457fa640 1 -- 172.21.15.170:0/2324271969 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9054049b40 con 0x55887c62f3a0 2024-03-28T16:49:15.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.263+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55887c636990 con 0x55887c62f3a0 2024-03-28T16:49:15.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f8f457fa640 1 -- 172.21.15.170:0/2324271969 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9054049ca0 con 0x55887c62f3a0 2024-03-28T16:49:15.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f8f457fa640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f2c034300 0x7f8f2c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f8f457fa640 1 -- 172.21.15.170:0/2324271969 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f905406bab0 con 0x55887c62f3a0 2024-03-28T16:49:15.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f906299f640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f2c034300 0x7f8f2c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c6416e0 0x55887c643b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6442d0 con 0x55887c6416e0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90639a1640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c6416e0 0x55887c643b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f906299f640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f2c034300 0x7f8f2c0367c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f905801b2c0 tx=0x7f90580a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90639a1640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c6416e0 0x55887c643b80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 realm 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6446a0 con 0x55887c6416e0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:15.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c645250 con 0x55887c6416e0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c645c00 0x55887c647ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c648640 con 0x55887c645c00 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90631a0640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c645c00 0x55887c647ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90631a0640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c645c00 0x55887c647ff0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 20 searching for the correct realm 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55887c64a820 0x55887c64cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64d360 con 0x55887c64a820 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f906299f640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55887c64a820 0x55887c64cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.267+0000 7f906299f640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55887c64a820 0x55887c64cce0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f906299f640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f90580da040 0x7f90580cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64d360 con 0x7f90580da040 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f90580da040 0x7f90580cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f90580da040 0x7f90580cc420 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.271+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x7f90580da040 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x7f90580da040 2024-03-28T16:49:15.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.275+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x7f90580da040 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x7f90580da040 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c64a820 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.279+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c645c00 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x55887c6416e0 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c644ef0 con 0x7f90580da040 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c648c20 con 0x55887c6416e0 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c648c20 con 0x55887c6416e0 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c648c20 con 0x7f90580da040 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 zone default found 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 4 Realm: () 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:15.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 zonegroup default 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c649470 con 0x55887c645c00 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6498c0 con 0x55887c64a820 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64f2f0 con 0x55887c64a820 2024-03-28T16:49:15.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 started sync module instance, tier type = 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.283+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64ea60 con 0x7f90580da040 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.287+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.287+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94044690900416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64ee10 con 0x7f90580da040 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.287+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c650050 con 0x55887c6416e0 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.287+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94044690900416] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.287+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.287+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94044690904848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64ffd0 con 0x55887c6416e0 2024-03-28T16:49:15.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.287+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c650a80 con 0x55887c64a820 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94044690908544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6512d0 con 0x55887c64a820 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c651ca0 con 0x7f90580da040 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94044690904848] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94044690913184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c652050 con 0x7f90580da040 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6529c0 con 0x55887c6416e0 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94044690908544] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94044690913184] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94044690905904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c652520 con 0x55887c6416e0 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.291+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c653210 con 0x55887c645c00 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94044690905904] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94044690918768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c652520 con 0x55887c645c00 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c653a10 con 0x55887c645c00 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94044690918768] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94044690910768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c654210 con 0x55887c645c00 2024-03-28T16:49:15.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.295+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c654b80 con 0x55887c645c00 2024-03-28T16:49:15.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94044690925552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6552d0 con 0x55887c645c00 2024-03-28T16:49:15.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=0 2024-03-28T16:49:15.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=1 2024-03-28T16:49:15.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=2 2024-03-28T16:49:15.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=3 2024-03-28T16:49:15.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=4 2024-03-28T16:49:15.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=5 2024-03-28T16:49:15.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94044690910768] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=6 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94044690925552] v29'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 20 add_watcher() i=7 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c653d70 con 0x55887c6416e0 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.299+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94044690926960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c653d70 con 0x55887c6416e0 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94044690926960] v29'68 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8f19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 5 note: GC not initialized 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8f18fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 20 Filter name: none 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8e927fc640 20 reqs_thread_entry: start 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c67a3d0 con 0x55887c6416e0 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 20 rados_obj.operate() r=0 bl.length=459 2024-03-28T16:49:15.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:15.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:15.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6800b0 con 0x55887c6416e0 2024-03-28T16:49:15.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v29'5 uv5 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:15.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94044691102672 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c67cd90 con 0x55887c6416e0 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94044690905904 notify 124554051588 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7f905c04a040 con 0x55887c6416e0 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8f46ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051588 cookie 94044690905904 notifier 4347 bl.length()=533 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8f46ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8f46ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(64 notify.4 [notify cookie 94044691102672 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8f46ffd640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f8f28003ea0 con 0x55887c6416e0 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94044691102672 notify 124554051588 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f905c043b80 con 0x55887c6416e0 2024-03-28T16:49:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify cookie 94044691102672 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c67a8d0 con 0x55887c64a820 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== watch-notify(notify (1) cookie 94044690908544 notify 124554051585 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55887c518cb0 con 0x55887c64a820 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.303+0000 7f8f467fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051585 cookie 94044690908544 notifier 4347 bl.length()=170 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 notify.2 [notify cookie 94044691102672 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f8f467fc640 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f8e980032c0 con 0x55887c64a820 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== watch-notify(notify_complete (2) cookie 94044691102672 notify 124554051585 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f90580db430 con 0x55887c64a820 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(67 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c67ac80 con 0x55887c645c00 2024-03-28T16:49:15.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(68 p5YnriCv1nAtykxBrupQ [delete] v29'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.311 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f8f19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94044690926960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6793b0 con 0x55887c6416e0 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94044690926960] v29'69 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.307+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94044690900416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c654d30 con 0x7f90580da040 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.311+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94044690900416] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.311+0000 7f9065685980 20 remove_watcher() i=0 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.311+0000 7f9065685980 2 removed watcher, disabling cache 2024-03-28T16:49:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.311+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94044690904848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64e5c0 con 0x55887c6416e0 2024-03-28T16:49:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.311+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94044690904848] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.311+0000 7f9065685980 20 remove_watcher() i=1 2024-03-28T16:49:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.311+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94044690908544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64e5c0 con 0x55887c64a820 2024-03-28T16:49:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f906299f640 1 -- 172.21.15.170:0/2324271969 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94044690908544] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f90580abbe0 con 0x55887c64a820 2024-03-28T16:49:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f9065685980 20 remove_watcher() i=2 2024-03-28T16:49:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94044690913184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64e5c0 con 0x7f90580da040 2024-03-28T16:49:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94044690913184] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x7f90580da040 2024-03-28T16:49:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f9065685980 20 remove_watcher() i=3 2024-03-28T16:49:15.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94044690905904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c6517a0 con 0x55887c6416e0 2024-03-28T16:49:15.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94044690905904] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f905c00a4d0 con 0x55887c6416e0 2024-03-28T16:49:15.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f9065685980 20 remove_watcher() i=4 2024-03-28T16:49:15.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.315+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94044690918768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c64fbb0 con 0x55887c645c00 2024-03-28T16:49:15.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.319+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94044690918768] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.319+0000 7f9065685980 20 remove_watcher() i=5 2024-03-28T16:49:15.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.319+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94044690910768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c652d70 con 0x55887c645c00 2024-03-28T16:49:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94044690910768] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 20 remove_watcher() i=6 2024-03-28T16:49:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94044690925552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55887c650e30 con 0x55887c645c00 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f90631a0640 1 -- 172.21.15.170:0/2324271969 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94044690925552] v29'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9054014070 con 0x55887c645c00 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 20 remove_watcher() i=7 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c645c00 msgr2=0x55887c647ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c645c00 0x55887c647ff0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c6416e0 msgr2=0x55887c643b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c6416e0 0x55887c643b80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f90580da040 msgr2=0x7f90580cc420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f90580da040 0x7f90580cc420 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55887c64a820 msgr2=0x55887c64cce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55887c64a820 0x55887c64cce0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f2c034300 msgr2=0x7f8f2c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f2c034300 0x7f8f2c0367c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f905801b2c0 tx=0x7f90580a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 msgr2=0x55887c636250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2324271969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c62f3a0 0x55887c636250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9054041bb0 tx=0x7f9054046d50 comp rx=0 tx=0).stop 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f90639a1640 1 -- 172.21.15.170:0/2324271969 reap_dead start 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 shutdown_connections 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 >> 172.21.15.170:0/2324271969 conn(0x55887c62b730 msgr2=0x55887c628d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 shutdown_connections 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2324271969 wait complete. 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f3c0342b0 msgr2=0x7f8f3c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f3c0342b0 0x7f8f3c036770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55887c609670 tx=0x7f9058005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c622820 msgr2=0x55887c622c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c622820 0x55887c622c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9054047c70 tx=0x7f905401aca0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 shutdown_connections 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f3c0342b0 0x7f8f3c036770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/3151938137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c622820 0x55887c622c40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 >> 172.21.15.170:0/3151938137 conn(0x55887c617630 msgr2=0x55887c618140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 shutdown_connections 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/3151938137 wait complete. 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c60e200 msgr2=0x55887c6105f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55887c60e200 0x55887c6105f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c515370 msgr2=0x55887c60bc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55887c515370 0x55887c60bc40 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55887c611ea0 msgr2=0x55887c614290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55887c611ea0 0x55887c614290 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f480342b0 msgr2=0x7f8f48036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8f480342b0 0x7f8f48036770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f90580ad000 tx=0x7f90580a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 msgr2=0x55887c604fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 --2- 172.21.15.170:0/2262259853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55887c510e10 0x55887c604fc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9054001ee0 tx=0x7f9054000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f90639a1640 1 -- 172.21.15.170:0/2262259853 reap_dead start 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 shutdown_connections 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 >> 172.21.15.170:0/2262259853 conn(0x55887c50c6e0 msgr2=0x55887c5f6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 shutdown_connections 2024-03-28T16:49:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.323+0000 7f9065685980 1 -- 172.21.15.170:0/2262259853 wait complete. 2024-03-28T16:49:15.333 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:15.334 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-03-28T16:49:15.334 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-03-28T16:49:15.334 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-03-28T16:49:15.412 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:15.412 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:15.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.411+0000 7f4cdde29980 1 Processor -- start 2024-03-28T16:49:15.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.411+0000 7f4cdde29980 1 -- start start 2024-03-28T16:49:15.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.411+0000 7f4cdde29980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d762c84360 0x55d762c86750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.411+0000 7f4cdde29980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d762bcc4c0 con 0x55d762bf48c0 2024-03-28T16:49:15.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.411+0000 7f4cdde29980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d762b930f0 con 0x55d762c84360 2024-03-28T16:49:15.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdb143640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d762c84360 0x55d762c86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdb944640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55d762bf48c0 0x55d762c82c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60914/0 (socket says 172.21.15.170:60914) 2024-03-28T16:49:15.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdb143640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d762c84360 0x55d762c86750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55644/0 (socket says 172.21.15.170:55644) 2024-03-28T16:49:15.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdb944640 1 -- 172.21.15.170:0/1373499942 learned_addr learned my addr 172.21.15.170:0/1373499942 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:15.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdb143640 1 -- 172.21.15.170:0/1373499942 >> v1:172.21.15.119:6789/0 conn(0x55d762bf48c0 legacy=0x55d762c82c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdb143640 1 -- 172.21.15.170:0/1373499942 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d762c87cd0 con 0x55d762c84360 2024-03-28T16:49:15.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdb143640 1 --2- 172.21.15.170:0/1373499942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762c84360 0x55d762c86750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400a480 tx=0x7f4cc401b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9bcc32de8874ec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cda942640 1 -- 172.21.15.170:0/1373499942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc40ab540 con 0x55d762c84360 2024-03-28T16:49:15.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cda942640 1 -- 172.21.15.170:0/1373499942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc40ab6a0 con 0x55d762c84360 2024-03-28T16:49:15.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cda942640 1 -- 172.21.15.170:0/1373499942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cc40ab960 con 0x55d762c84360 2024-03-28T16:49:15.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdde29980 1 -- 172.21.15.170:0/1373499942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762c84360 msgr2=0x55d762c86750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1373499942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762c84360 0x55d762c86750 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400a480 tx=0x7f4cc401b280 comp rx=0 tx=0).stop 2024-03-28T16:49:15.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdde29980 1 -- 172.21.15.170:0/1373499942 shutdown_connections 2024-03-28T16:49:15.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1373499942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762c84360 0x55d762c86750 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdde29980 1 -- 172.21.15.170:0/1373499942 >> 172.21.15.170:0/1373499942 conn(0x55d762c7a720 msgr2=0x55d762c7cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.415+0000 7f4cdde29980 1 -- 172.21.15.170:0/1373499942 shutdown_connections 2024-03-28T16:49:15.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.419+0000 7f4cdde29980 1 -- 172.21.15.170:0/1373499942 wait complete. 2024-03-28T16:49:15.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 Processor -- start 2024-03-28T16:49:15.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 -- start start 2024-03-28T16:49:15.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d69e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d762b90620 con 0x55d762bf48c0 2024-03-28T16:49:15.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d69e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d69e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55648/0 (socket says 172.21.15.170:55648) 2024-03-28T16:49:15.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdb944640 1 -- 172.21.15.170:0/2193178818 learned_addr learned my addr 172.21.15.170:0/2193178818 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdb944640 1 -- 172.21.15.170:0/2193178818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4005d30 con 0x55d762bf48c0 2024-03-28T16:49:15.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdb944640 1 --2- 172.21.15.170:0/2193178818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d69e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc006fb0 tx=0x7f4ccc001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6fecae927b6c3e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4bc9e0b640 1 -- 172.21.15.170:0/2193178818 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc004530 con 0x55d762bf48c0 2024-03-28T16:49:15.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4bc9e0b640 1 -- 172.21.15.170:0/2193178818 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc004690 con 0x55d762bf48c0 2024-03-28T16:49:15.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4bc9e0b640 1 -- 172.21.15.170:0/2193178818 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc0049b0 con 0x55d762bf48c0 2024-03-28T16:49:15.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 -- 172.21.15.170:0/2193178818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 msgr2=0x55d762d69e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 --2- 172.21.15.170:0/2193178818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d69e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc006fb0 tx=0x7f4ccc001940 comp rx=0 tx=0).stop 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 -- 172.21.15.170:0/2193178818 shutdown_connections 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 --2- 172.21.15.170:0/2193178818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d69e80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 -- 172.21.15.170:0/2193178818 >> 172.21.15.170:0/2193178818 conn(0x55d762c7a720 msgr2=0x55d762d65a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 -- 172.21.15.170:0/2193178818 shutdown_connections 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.431+0000 7f4cdde29980 1 -- 172.21.15.170:0/2193178818 wait complete. 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 Processor -- start 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 -- start start 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d70cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d762b8be00 con 0x55d762bf48c0 2024-03-28T16:49:15.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d70cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d70cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55656/0 (socket says 172.21.15.170:55656) 2024-03-28T16:49:15.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdb944640 1 -- 172.21.15.170:0/3973186034 learned_addr learned my addr 172.21.15.170:0/3973186034 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdb944640 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d762bcc4c0 con 0x55d762bf48c0 2024-03-28T16:49:15.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdb944640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d70cd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55d762d6ad10 tx=0x7f4ccc000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4bc3fff640 1 -- 172.21.15.170:0/3973186034 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc013760 con 0x55d762bf48c0 2024-03-28T16:49:15.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4bc3fff640 1 -- 172.21.15.170:0/3973186034 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc0138c0 con 0x55d762bf48c0 2024-03-28T16:49:15.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d762d71210 con 0x55d762bf48c0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4bc3fff640 1 -- 172.21.15.170:0/3973186034 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc013b60 con 0x55d762bf48c0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d762d71640 con 0x55d762bf48c0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4bc3fff640 1 -- 172.21.15.170:0/3973186034 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f4ccc028460 con 0x55d762bf48c0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4bc3fff640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bc40342b0 0x7f4bc4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4bc3fff640 1 -- 172.21.15.170:0/3973186034 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f4ccc0493a0 con 0x55d762bf48c0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdb143640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bc40342b0 0x7f4bc4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762c84360 0x55d762d77950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762d77e90 con 0x55d762c84360 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.435+0000 7f4cdc145640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762c84360 0x55d762d77950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdb143640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bc40342b0 0x7f4bc4036770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ad000 tx=0x7f4cc40a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdc145640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762c84360 0x55d762d77950 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdc145640 1 -- 172.21.15.170:0/3973186034 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762c84360 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762d781f0 con 0x55d762c84360 2024-03-28T16:49:15.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdc145640 1 -- 172.21.15.170:0/3973186034 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762c84360 2024-03-28T16:49:15.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762d79f10 0x55d762d7c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762d7c840 con 0x55d762d79f10 2024-03-28T16:49:15.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdb944640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762d79f10 0x55d762d7c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdb944640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762d79f10 0x55d762d7c300 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdb944640 1 -- 172.21.15.170:0/3973186034 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762d79f10 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762d78d50 con 0x55d762c84360 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdc145640 1 -- 172.21.15.170:0/3973186034 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762c84360 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d762d7dbb0 0x55d762d7ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762d80640 con 0x55d762d7dbb0 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdb143640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d762d7dbb0 0x55d762d7ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.439+0000 7f4cdb143640 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d762d7dbb0 0x55d762d7ffa0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdb143640 1 -- 172.21.15.170:0/3973186034 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762d7dbb0 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 Processor -- start 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 -- start start 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d8e550 0x55d762d8e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d762b24e30 con 0x55d762d8e550 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d8e550 0x55d762d8e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d8e550 0x55d762d8e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55672/0 (socket says 172.21.15.170:55672) 2024-03-28T16:49:15.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdb944640 1 -- 172.21.15.170:0/583452169 learned_addr learned my addr 172.21.15.170:0/583452169 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdb944640 1 -- 172.21.15.170:0/583452169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d762d8eeb0 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdb944640 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d8e550 0x55d762d8e970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0479a0 tx=0x7f4ccc01e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4bc17fa640 1 -- 172.21.15.170:0/583452169 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc03a070 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d762d8f100 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4bc17fa640 1 -- 172.21.15.170:0/583452169 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc010570 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4bc17fa640 1 -- 172.21.15.170:0/583452169 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc011760 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d762d8fc50 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4bc17fa640 1 -- 172.21.15.170:0/583452169 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f4ccc0118c0 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4bc17fa640 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bb40342b0 0x7f4bb4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4bc17fa640 1 -- 172.21.15.170:0/583452169 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f4ccc040b10 con 0x55d762d8e550 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdb143640 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bb40342b0 0x7f4bb4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 Processor -- start 2024-03-28T16:49:15.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.443+0000 7f4cdde29980 1 -- start start 2024-03-28T16:49:15.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb143640 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bb40342b0 0x7f4bb4036770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d762d75370 tx=0x7f4cc4005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762d9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d762b18790 con 0x55d762d9b370 2024-03-28T16:49:15.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762d9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762d9b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55676/0 (socket says 172.21.15.170:55676) 2024-03-28T16:49:15.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 -- 172.21.15.170:0/1355975592 learned_addr learned my addr 172.21.15.170:0/1355975592 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 -- 172.21.15.170:0/1355975592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d762d9bd40 con 0x55d762d9b370 2024-03-28T16:49:15.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 --2- 172.21.15.170:0/1355975592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762d9b770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc007a50 tx=0x7f4ccc007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ed9ed8c6f949c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4bbaffd640 1 -- 172.21.15.170:0/1355975592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc059070 con 0x55d762d9b370 2024-03-28T16:49:15.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4bbaffd640 1 -- 172.21.15.170:0/1355975592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc027700 con 0x55d762d9b370 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4bbaffd640 1 -- 172.21.15.170:0/1355975592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc0370d0 con 0x55d762d9b370 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- 172.21.15.170:0/1355975592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 msgr2=0x55d762d9b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1355975592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762d9b770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc007a50 tx=0x7f4ccc007b40 comp rx=0 tx=0).stop 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- 172.21.15.170:0/1355975592 shutdown_connections 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1355975592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762d9b770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- 172.21.15.170:0/1355975592 >> 172.21.15.170:0/1355975592 conn(0x55d762d976b0 msgr2=0x55d762d95710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- 172.21.15.170:0/1355975592 shutdown_connections 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- 172.21.15.170:0/1355975592 wait complete. 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 Processor -- start 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- start start 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762da2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d762da26c0 con 0x55d762d9b370 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762da2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762da2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55686/0 (socket says 172.21.15.170:55686) 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 learned_addr learned my addr 172.21.15.170:0/1605632917 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d762d8eeb0 con 0x55d762d9b370 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdb944640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762da2180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc027870 tx=0x7f4ccc008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4bb97fa640 1 -- 172.21.15.170:0/1605632917 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc00c150 con 0x55d762d9b370 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d762d8eeb0 con 0x55d762d9b370 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4bb97fa640 1 -- 172.21.15.170:0/1605632917 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc0278e0 con 0x55d762d9b370 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4bb97fa640 1 -- 172.21.15.170:0/1605632917 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ccc028a50 con 0x55d762d9b370 2024-03-28T16:49:15.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.447+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d762d8eeb0 con 0x55d762d9b370 2024-03-28T16:49:15.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4bb97fa640 1 -- 172.21.15.170:0/1605632917 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f4ccc047a80 con 0x55d762d9b370 2024-03-28T16:49:15.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4bb97fa640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ba4034260 0x7f4ba4036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4bb97fa640 1 -- 172.21.15.170:0/1605632917 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f4ccc074af0 con 0x55d762d9b370 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdb143640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ba4034260 0x7f4ba4036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762dad430 0x55d762daf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0020 con 0x55d762dad430 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdc145640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762dad430 0x55d762daf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdb143640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ba4034260 0x7f4ba4036720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40abfd0 tx=0x7f4cc40bff90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdc145640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762dad430 0x55d762daf8d0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 realm 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db03f0 con 0x55d762dad430 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0fa0 con 0x55d762dad430 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762db1950 0x55d762db3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db43e0 con 0x55d762db1950 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdb944640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762db1950 0x55d762db3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.451+0000 7f4cdb944640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762db1950 0x55d762db3d40 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdde29980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdde29980 20 searching for the correct realm 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d762db65a0 0x55d762db8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db90e0 con 0x55d762db65a0 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb143640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d762db65a0 0x55d762db8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb143640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d762db65a0 0x55d762db8a60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb143640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4cc40cc050 0x7f4cc40cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db90e0 con 0x7f4cc40cc050 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4cc40cc050 0x7f4cc40cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4cc40cc050 0x7f4cc40cb3b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.455+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x7f4cc40cc050 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x7f4cc40cc050 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.459+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x7f4cc40cc050 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.463+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x7f4cc40cc050 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db65a0 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762db1950 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x55d762dad430 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db0c40 con 0x7f4cc40cc050 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:15.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db49a0 con 0x55d762dad430 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db49a0 con 0x55d762dad430 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db4950 con 0x7f4cc40cc050 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 zone default found 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 4 Realm: () 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 zonegroup default 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db51f0 con 0x55d762db1950 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762db5650 con 0x55d762db65a0 2024-03-28T16:49:15.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.467+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbafb0 con 0x55d762db65a0 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 20 started sync module instance, tier type = 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dba7e0 con 0x7f4cc40cc050 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94383564890944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dba760 con 0x7f4cc40cc050 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbbd10 con 0x55d762dad430 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94383564890944] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94383564895184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbbc90 con 0x55d762dad430 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.471+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbc490 con 0x55d762db65a0 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94383564895184] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94383564897696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbcc90 con 0x55d762db65a0 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbb7f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94383564902752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbda20 con 0x7f4cc40cc050 2024-03-28T16:49:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbe390 con 0x55d762dad430 2024-03-28T16:49:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94383564897696] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.475+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94383564902752] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.479+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.479+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94383564903808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbc900 con 0x55d762dad430 2024-03-28T16:49:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.479+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbe830 con 0x55d762db1950 2024-03-28T16:49:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.479+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94383564903808] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.479+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.479+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94383564906224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbeb90 con 0x55d762db1950 2024-03-28T16:49:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.479+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbf390 con 0x55d762db1950 2024-03-28T16:49:15.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94383564906224] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94383564907280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbfb90 con 0x55d762db1950 2024-03-28T16:49:15.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dc03f0 con 0x55d762db1950 2024-03-28T16:49:15.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94383564914416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dc0c40 con 0x55d762db1950 2024-03-28T16:49:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=0 2024-03-28T16:49:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=1 2024-03-28T16:49:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=2 2024-03-28T16:49:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=3 2024-03-28T16:49:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=4 2024-03-28T16:49:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=5 2024-03-28T16:49:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94383564907280] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=6 2024-03-28T16:49:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94383564914416] v29'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 20 add_watcher() i=7 2024-03-28T16:49:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.483+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbf6f0 con 0x55d762dad430 2024-03-28T16:49:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94383564916640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbf6f0 con 0x55d762dad430 2024-03-28T16:49:15.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94383564916640] v29'70 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4b91fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 5 note: GC not initialized 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4b90fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 Filter name: none 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4b0a7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762de5b90 con 0x55d762dad430 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762de6090 con 0x55d762dad430 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762deba50 con 0x55d762dad430 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~513] v29'6 uv6 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.487+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94383565092208 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762de7540 con 0x55d762dad430 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== watch-notify(notify (1) cookie 94383564903808 notify 124554051589 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x7f4cd400ac70 con 0x55d762dad430 2024-03-28T16:49:15.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051589 cookie 94383564903808 notifier 4354 bl.length()=689 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(65 notify.4 [notify cookie 94383565092208 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f4ba0003ea0 con 0x55d762dad430 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== watch-notify(notify_complete (2) cookie 94383565092208 notify 124554051589 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f4cd400ae00 con 0x55d762dad430 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762deb570 con 0x55d762dad430 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdde29980 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdde29980 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdde29980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-03-28T16:49:15.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94383565092208 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762de7540 con 0x7f4cc40cc050 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== watch-notify(notify (1) cookie 94383564890944 notify 124554051586 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x7f4cd400ae00 con 0x7f4cc40cc050 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051586 cookie 94383564890944 notifier 4354 bl.length()=166 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.0 [notify cookie 94383565092208 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4bbaffd640 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f4ba0003ea0 con 0x7f4cc40cc050 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== watch-notify(notify_complete (2) cookie 94383565092208 notify 124554051586 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f4cd40425e0 con 0x7f4cc40cc050 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.497 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo1","secret_key":"RDQ3XR2rNcxnXTc1gjtoaVpKCn8yRHEZEoAgIpWm"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4b91fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.491+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94383564916640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762de4ca0 con 0x55d762dad430 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94383564916640] v29'71 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94383564890944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dc07a0 con 0x7f4cc40cc050 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94383564890944] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdde29980 20 remove_watcher() i=0 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdde29980 2 removed watcher, disabling cache 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94383564895184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dba340 con 0x55d762dad430 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94383564895184] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdde29980 20 remove_watcher() i=1 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.495+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94383564897696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dba340 con 0x55d762db65a0 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.499+0000 7f4cdb143640 1 -- 172.21.15.170:0/1605632917 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94383564897696] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cc40abc30 con 0x55d762db65a0 2024-03-28T16:49:15.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.499+0000 7f4cdde29980 20 remove_watcher() i=2 2024-03-28T16:49:15.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.499+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94383564902752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbbda0 con 0x7f4cc40cc050 2024-03-28T16:49:15.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.499+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94383564902752] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x7f4cc40cc050 2024-03-28T16:49:15.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.499+0000 7f4cdde29980 20 remove_watcher() i=3 2024-03-28T16:49:15.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.499+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94383564903808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbd160 con 0x55d762dad430 2024-03-28T16:49:15.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94383564903808] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4cd40083f0 con 0x55d762dad430 2024-03-28T16:49:15.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdde29980 20 remove_watcher() i=4 2024-03-28T16:49:15.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94383564906224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbd600 con 0x55d762db1950 2024-03-28T16:49:15.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94383564906224] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdde29980 20 remove_watcher() i=5 2024-03-28T16:49:15.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94383564907280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbdef0 con 0x55d762db1950 2024-03-28T16:49:15.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94383564907280] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdde29980 20 remove_watcher() i=6 2024-03-28T16:49:15.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.503+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94383564914416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d762dbb3d0 con 0x55d762db1950 2024-03-28T16:49:15.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdb944640 1 -- 172.21.15.170:0/1605632917 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94383564914416] v29'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4ccc024020 con 0x55d762db1950 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 20 remove_watcher() i=7 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762db1950 msgr2=0x55d762db3d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762db1950 0x55d762db3d40 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762dad430 msgr2=0x55d762daf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762dad430 0x55d762daf8d0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4cc40cc050 msgr2=0x7f4cc40cb3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4cc40cc050 0x7f4cc40cb3b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d762db65a0 msgr2=0x55d762db8a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d762db65a0 0x55d762db8a60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ba4034260 msgr2=0x7f4ba4036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ba4034260 0x7f4ba4036720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40abfd0 tx=0x7f4cc40bff90 comp rx=0 tx=0).stop 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 msgr2=0x55d762da2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/1605632917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d9b370 0x55d762da2180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc027870 tx=0x7f4ccc008eb0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 reap_dead start 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdc145640 1 -- 172.21.15.170:0/1605632917 reap_dead start 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 shutdown_connections 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 >> 172.21.15.170:0/1605632917 conn(0x55d762d976b0 msgr2=0x55d762d94d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 shutdown_connections 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/1605632917 wait complete. 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bb40342b0 msgr2=0x7f4bb4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bb40342b0 0x7f4bb4036770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55d762d75370 tx=0x7f4cc4005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d8e550 msgr2=0x55d762d8e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d8e550 0x55d762d8e970 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0479a0 tx=0x7f4ccc01e6c0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 shutdown_connections 2024-03-28T16:49:15.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bb40342b0 0x7f4bb4036770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/583452169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762d8e550 0x55d762d8e970 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 >> 172.21.15.170:0/583452169 conn(0x55d762d83360 msgr2=0x55d762d83e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 shutdown_connections 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/583452169 wait complete. 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762d79f10 msgr2=0x55d762d7c300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d762d79f10 0x55d762d7c300 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762c84360 msgr2=0x55d762d77950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d762c84360 0x55d762d77950 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d762d7dbb0 msgr2=0x55d762d7ffa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d762d7dbb0 0x55d762d7ffa0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bc40342b0 msgr2=0x7f4bc4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4bc40342b0 0x7f4bc4036770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ad000 tx=0x7f4cc40a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 msgr2=0x55d762d70cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 --2- 172.21.15.170:0/3973186034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d762bf48c0 0x55d762d70cd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55d762d6ad10 tx=0x7f4ccc000f50 comp rx=0 tx=0).stop 2024-03-28T16:49:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdc145640 1 -- 172.21.15.170:0/3973186034 reap_dead start 2024-03-28T16:49:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 shutdown_connections 2024-03-28T16:49:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 >> 172.21.15.170:0/3973186034 conn(0x55d762c7a720 msgr2=0x55d762d64520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 shutdown_connections 2024-03-28T16:49:15.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.507+0000 7f4cdde29980 1 -- 172.21.15.170:0/3973186034 wait complete. 2024-03-28T16:49:15.518 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'RDQ3XR2rNcxnXTc1gjtoaVpKCn8yRHEZEoAgIpWm'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:15.518 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-03-28T16:49:15.518 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-03-28T16:49:15.518 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-03-28T16:49:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:15.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bfa3c5980 1 Processor -- start 2024-03-28T16:49:15.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bfa3c5980 1 -- start start 2024-03-28T16:49:15.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bfa3c5980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x555970e11de0 0x555970e121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bfa3c5980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555970d5f4c0 con 0x555970e12700 2024-03-28T16:49:15.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bfa3c5980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x555970d260f0 con 0x555970e11de0 2024-03-28T16:49:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bf7ee0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x555970e11de0 0x555970e121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bf7ee0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x555970e11de0 0x555970e121c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55702/0 (socket says 172.21.15.170:55702) 2024-03-28T16:49:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/873025276 learned_addr learned my addr 172.21.15.170:0/873025276 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bf76df640 1 --1- 172.21.15.170:0/873025276 >> v1:172.21.15.119:6789/0 conn(0x555970e12700 0x555970e130c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60928/0 (socket says 172.21.15.170:60928) 2024-03-28T16:49:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/873025276 >> v1:172.21.15.119:6789/0 conn(0x555970e12700 legacy=0x555970e130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/873025276 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555970e138f0 con 0x555970e11de0 2024-03-28T16:49:15.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.595+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/873025276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970e121c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a4e0 tx=0x7f8be801b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ecc274d5f4760c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bf6ede640 1 -- 172.21.15.170:0/873025276 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80aa540 con 0x555970e11de0 2024-03-28T16:49:15.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bf6ede640 1 -- 172.21.15.170:0/873025276 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80a6070 con 0x555970e11de0 2024-03-28T16:49:15.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bf6ede640 1 -- 172.21.15.170:0/873025276 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80aa850 con 0x555970e11de0 2024-03-28T16:49:15.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/873025276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 msgr2=0x555970e121c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/873025276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970e121c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a4e0 tx=0x7f8be801b090 comp rx=0 tx=0).stop 2024-03-28T16:49:15.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/873025276 shutdown_connections 2024-03-28T16:49:15.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/873025276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970e121c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/873025276 >> 172.21.15.170:0/873025276 conn(0x555970e0d6b0 msgr2=0x555970e0fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/873025276 shutdown_connections 2024-03-28T16:49:15.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.599+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/873025276 wait complete. 2024-03-28T16:49:15.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bfa3c5980 1 Processor -- start 2024-03-28T16:49:15.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bfa3c5980 1 -- start start 2024-03-28T16:49:15.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bfa3c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970efc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bfa3c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555970d23620 con 0x555970e11de0 2024-03-28T16:49:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970efc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970efc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55710/0 (socket says 172.21.15.170:55710) 2024-03-28T16:49:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1432694910 learned_addr learned my addr 172.21.15.170:0/1432694910 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1432694910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be800a1c0 con 0x555970e11de0 2024-03-28T16:49:15.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.611+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/1432694910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970efc300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8be8017ca0 tx=0x7f8be80033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb59262d46511f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8ae5e0b640 1 -- 172.21.15.170:0/1432694910 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80aa780 con 0x555970e11de0 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8ae5e0b640 1 -- 172.21.15.170:0/1432694910 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80aa8e0 con 0x555970e11de0 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8ae5e0b640 1 -- 172.21.15.170:0/1432694910 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80aab80 con 0x555970e11de0 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1432694910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 msgr2=0x555970efc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1432694910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970efc300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8be8017ca0 tx=0x7f8be80033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1432694910 shutdown_connections 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1432694910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970efc300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1432694910 >> 172.21.15.170:0/1432694910 conn(0x555970e0d6b0 msgr2=0x555970ef7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1432694910 shutdown_connections 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1432694910 wait complete. 2024-03-28T16:49:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 Processor -- start 2024-03-28T16:49:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- start start 2024-03-28T16:49:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970d87600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555970d1ee00 con 0x555970e11de0 2024-03-28T16:49:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970d87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970d87600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55718/0 (socket says 172.21.15.170:55718) 2024-03-28T16:49:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/125784543 learned_addr learned my addr 172.21.15.170:0/125784543 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555970e138f0 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970d87600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8be801aec0 tx=0x7f8be8017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8adffff640 1 -- 172.21.15.170:0/125784543 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80c4070 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555970d87b40 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8adffff640 1 -- 172.21.15.170:0/125784543 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80b5650 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555970f07150 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.615+0000 7f8adffff640 1 -- 172.21.15.170:0/125784543 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80b58f0 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8adffff640 1 -- 172.21.15.170:0/125784543 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8be80a8070 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8adffff640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ae00342b0 0x7f8ae0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8adffff640 1 -- 172.21.15.170:0/125784543 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8be80e3570 con 0x555970e11de0 2024-03-28T16:49:15.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf76df640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ae00342b0 0x7f8ae0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970e12700 0x555970f0abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f0b190 con 0x555970e12700 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf86e1640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970e12700 0x555970f0abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf76df640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ae00342b0 0x7f8ae0036770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8be0000f70 tx=0x7f8be0001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf86e1640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970e12700 0x555970f0abc0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf86e1640 1 -- 172.21.15.170:0/125784543 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970e12700 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f082d0 con 0x555970e12700 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf86e1640 1 -- 172.21.15.170:0/125784543 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970e12700 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f0d3d0 0x555970f0f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f0fd00 con 0x555970f0d3d0 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f0d3d0 0x555970f0f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f0d3d0 0x555970f0f7c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/125784543 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f8be80b4a00 con 0x555970f0d3d0 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f0c1e0 con 0x555970e12700 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf86e1640 1 -- 172.21.15.170:0/125784543 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970e12700 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555970f11090 0x555970f13480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f13a90 con 0x555970f11090 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.619+0000 7f8bf76df640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555970f11090 0x555970f13480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf76df640 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555970f11090 0x555970f13480 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf76df640 1 -- 172.21.15.170:0/125784543 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f8be000f6a0 con 0x555970f11090 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bfa3c5980 1 Processor -- start 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bfa3c5980 1 -- start start 2024-03-28T16:49:15.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bfa3c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f219a0 0x555970f21dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bfa3c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555970cb7e30 con 0x555970f219a0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f219a0 0x555970f21dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f219a0 0x555970f21dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55726/0 (socket says 172.21.15.170:55726) 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/778770778 learned_addr learned my addr 172.21.15.170:0/778770778 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/778770778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555970f22300 con 0x555970f219a0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f219a0 0x555970f21dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8be80e1b70 tx=0x7f8be80a96b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8add7fa640 1 -- 172.21.15.170:0/778770778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80dd070 con 0x555970f219a0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8add7fa640 1 -- 172.21.15.170:0/778770778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80c8ca0 con 0x555970f219a0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555970f22550 con 0x555970f219a0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8add7fa640 1 -- 172.21.15.170:0/778770778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80d6040 con 0x555970f219a0 2024-03-28T16:49:15.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555970f230a0 con 0x555970f219a0 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8add7fa640 1 -- 172.21.15.170:0/778770778 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8be80d0b70 con 0x555970f219a0 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8add7fa640 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ad00342b0 0x7f8ad0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8add7fa640 1 -- 172.21.15.170:0/778770778 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8be80eeaf0 con 0x555970f219a0 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bf76df640 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ad00342b0 0x7f8ad0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.623+0000 7f8bfa3c5980 1 Processor -- start 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- start start 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf76df640 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ad00342b0 0x7f8ad0036770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x555970f0c540 tx=0x7f8be000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555970cab790 con 0x555970f044c0 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55730/0 (socket says 172.21.15.170:55730) 2024-03-28T16:49:15.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/2347489030 learned_addr learned my addr 172.21.15.170:0/2347489030 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/2347489030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555970f02d50 con 0x555970f044c0 2024-03-28T16:49:15.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/2347489030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f048e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8be80d7ad0 tx=0x7f8be80cb120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5c425754a16444 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8ad6ffd640 1 -- 172.21.15.170:0/2347489030 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80e3e40 con 0x555970f044c0 2024-03-28T16:49:15.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8ad6ffd640 1 -- 172.21.15.170:0/2347489030 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80f5070 con 0x555970f044c0 2024-03-28T16:49:15.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8ad6ffd640 1 -- 172.21.15.170:0/2347489030 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80c7940 con 0x555970f044c0 2024-03-28T16:49:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/2347489030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 msgr2=0x555970f048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/2347489030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f048e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8be80d7ad0 tx=0x7f8be80cb120 comp rx=0 tx=0).stop 2024-03-28T16:49:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/2347489030 shutdown_connections 2024-03-28T16:49:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/2347489030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f048e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/2347489030 >> 172.21.15.170:0/2347489030 conn(0x555970f2ac40 msgr2=0x555970f28ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/2347489030 shutdown_connections 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/2347489030 wait complete. 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 Processor -- start 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- start start 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bfa3c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555970f37ef0 con 0x555970f044c0 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55736/0 (socket says 172.21.15.170:55736) 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 learned_addr learned my addr 172.21.15.170:0/1230977937 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555970f230a0 con 0x555970f044c0 2024-03-28T16:49:15.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f379b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8be80d7fd0 tx=0x7f8be80deea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.627+0000 7f8ad57fa640 1 -- 172.21.15.170:0/1230977937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80e3180 con 0x555970f044c0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555970f380f0 con 0x555970f044c0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8ad57fa640 1 -- 172.21.15.170:0/1230977937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80e32e0 con 0x555970f044c0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8ad57fa640 1 -- 172.21.15.170:0/1230977937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8be80c7cc0 con 0x555970f044c0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555970f38520 con 0x555970f044c0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8ad57fa640 1 -- 172.21.15.170:0/1230977937 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8be8100040 con 0x555970f044c0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8ad57fa640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ac00342b0 0x7f8ac0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8ad57fa640 1 -- 172.21.15.170:0/1230977937 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8be80f1a00 con 0x555970f044c0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf76df640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ac00342b0 0x7f8ac0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970f42c10 0x555970f450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f45800 con 0x555970f42c10 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf86e1640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970f42c10 0x555970f450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf76df640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ac00342b0 0x7f8ac0036770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c010 tx=0x7f8be0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf86e1640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970f42c10 0x555970f450b0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 realm 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f45bd0 con 0x555970f42c10 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46420 con 0x555970f42c10 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f47490 0x555970f49880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f49ed0 con 0x555970f47490 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.631+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f47490 0x555970f49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf7ee0640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f47490 0x555970f49880 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bfa3c5980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bfa3c5980 20 searching for the correct realm 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555970f4c070 0x555970f4e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f4ebb0 con 0x555970f4c070 2024-03-28T16:49:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf76df640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555970f4c070 0x555970f4e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf76df640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555970f4c070 0x555970f4e530 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf76df640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8be001fc60 0x7f8be003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f4ebb0 con 0x7f8be001fc60 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8be001fc60 0x7f8be003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8be001fc60 0x7f8be003a3c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.635+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x7f8be001fc60 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x7f8be001fc60 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.639+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x7f8be001fc60 2024-03-28T16:49:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.643+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x7f8be001fc60 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f4c070 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f47490 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x555970f42c10 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f46780 con 0x7f8be001fc60 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f4a4b0 con 0x555970f42c10 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f4a4b0 con 0x555970f42c10 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f4a480 con 0x7f8be001fc60 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 zone default found 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 4 Realm: () 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 zonegroup default 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f4ad50 con 0x555970f47490 2024-03-28T16:49:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f4b150 con 0x555970f4c070 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f50b00 con 0x555970f4c070 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 started sync module instance, tier type = 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.647+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f502b0 con 0x7f8be001fc60 2024-03-28T16:49:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.651+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.651+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93842635554320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f50660 con 0x7f8be001fc60 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.651+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f51860 con 0x555970f42c10 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.651+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93842635554320] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.651+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.651+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93842635558688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f517e0 con 0x555970f42c10 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.651+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f51fe0 con 0x555970f4c070 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93842635561232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f52830 con 0x555970f4c070 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f53250 con 0x7f8be001fc60 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93842635558688] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93842635566384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f53600 con 0x7f8be001fc60 2024-03-28T16:49:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f53f70 con 0x555970f42c10 2024-03-28T16:49:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93842635561232] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.655+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93842635566384] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.659+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.659+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93842635559744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f53ad0 con 0x555970f42c10 2024-03-28T16:49:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.659+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f542d0 con 0x555970f47490 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.659+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.659+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93842635570176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f54b20 con 0x555970f47490 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.659+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f55490 con 0x555970f47490 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.659+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93842635559744] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93842635563920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f55c90 con 0x555970f47490 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f56600 con 0x555970f47490 2024-03-28T16:49:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 93842635570176] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93842635563920] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93842635577328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f55c10 con 0x555970f47490 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 20 add_watcher() i=0 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 20 add_watcher() i=1 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 20 add_watcher() i=2 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 20 add_watcher() i=3 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 20 add_watcher() i=4 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 20 add_watcher() i=5 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.663+0000 7f8bfa3c5980 20 add_watcher() i=6 2024-03-28T16:49:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93842635577328] v29'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 20 add_watcher() i=7 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f59630 con 0x555970f42c10 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93842635579744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f56a50 con 0x555970f42c10 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93842635579744] v29'72 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8aadfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 5 note: GC not initialized 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8aacfe1640 20 reqs_thread_entry: start 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 20 Filter name: none 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8a267fc640 20 reqs_thread_entry: start 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f7b4f0 con 0x555970f42c10 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv6 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 20 rados_obj.operate() r=0 bl.length=513 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.667+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f814a0 con 0x555970f42c10 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v29'7 uv7 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bfa3c5980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bfa3c5980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bfa3c5980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93842635755296 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f7e010 con 0x555970f42c10 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 93842635559744 notify 124554051590 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x7f8bf000aab0 con 0x555970f42c10 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8ad6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051590 cookie 93842635559744 notifier 4361 bl.length()=689 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8ad6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8ad6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8ad6ffd640 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f8abc003ea0 con 0x555970f42c10 2024-03-28T16:49:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(64 notify.4 [notify cookie 93842635755296 out=8b] v0'0 uv57 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 93842635755296 notify 124554051590 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f8bf000ac40 con 0x555970f42c10 2024-03-28T16:49:15.675 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8aadfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93842635579744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f5b900 con 0x555970f42c10 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93842635579744] v29'73 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.671+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93842635554320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f56160 con 0x7f8be001fc60 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.675+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93842635554320] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.675+0000 7f8bfa3c5980 20 remove_watcher() i=0 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.675+0000 7f8bfa3c5980 2 removed watcher, disabling cache 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.675+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93842635558688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f56160 con 0x555970f42c10 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.675+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93842635558688] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.675+0000 7f8bfa3c5980 20 remove_watcher() i=1 2024-03-28T16:49:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.675+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93842635561232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f50f20 con 0x555970f4c070 2024-03-28T16:49:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bf76df640 1 -- 172.21.15.170:0/1230977937 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93842635561232] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be0033040 con 0x555970f4c070 2024-03-28T16:49:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bfa3c5980 20 remove_watcher() i=2 2024-03-28T16:49:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93842635566384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f56160 con 0x7f8be001fc60 2024-03-28T16:49:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93842635566384] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x7f8be001fc60 2024-03-28T16:49:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bfa3c5980 20 remove_watcher() i=3 2024-03-28T16:49:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93842635559744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f52d30 con 0x555970f42c10 2024-03-28T16:49:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93842635559744] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8bf00083f0 con 0x555970f42c10 2024-03-28T16:49:15.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bfa3c5980 20 remove_watcher() i=4 2024-03-28T16:49:15.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.679+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93842635570176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f56160 con 0x555970f47490 2024-03-28T16:49:15.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.683+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93842635570176] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.683+0000 7f8bfa3c5980 20 remove_watcher() i=5 2024-03-28T16:49:15.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.683+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93842635563920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f56160 con 0x555970f47490 2024-03-28T16:49:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.683+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93842635563920] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.683+0000 7f8bfa3c5980 20 remove_watcher() i=6 2024-03-28T16:49:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.683+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93842635577328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555970f52390 con 0x555970f47490 2024-03-28T16:49:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bf7ee0640 1 -- 172.21.15.170:0/1230977937 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93842635577328] v29'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8be811d380 con 0x555970f47490 2024-03-28T16:49:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 20 remove_watcher() i=7 2024-03-28T16:49:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f47490 msgr2=0x555970f49880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f47490 0x555970f49880 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970f42c10 msgr2=0x555970f450b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970f42c10 0x555970f450b0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8be001fc60 msgr2=0x7f8be003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8be001fc60 0x7f8be003a3c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555970f4c070 msgr2=0x555970f4e530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555970f4c070 0x555970f4e530 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ac00342b0 msgr2=0x7f8ac0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ac00342b0 0x7f8ac0036770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c010 tx=0x7f8be0010000 comp rx=0 tx=0).stop 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 msgr2=0x555970f379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bf86e1640 1 -- 172.21.15.170:0/1230977937 reap_dead start 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f379b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8be80d7fd0 tx=0x7f8be80deea0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 shutdown_connections 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/1230977937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f044c0 0x555970f379b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 >> 172.21.15.170:0/1230977937 conn(0x555970f2ac40 msgr2=0x555970f282a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 shutdown_connections 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/1230977937 wait complete. 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ad00342b0 msgr2=0x7f8ad0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ad00342b0 0x7f8ad0036770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x555970f0c540 tx=0x7f8be000f000 comp rx=0 tx=0).stop 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f219a0 msgr2=0x555970f21dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f219a0 0x555970f21dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8be80e1b70 tx=0x7f8be80a96b0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 shutdown_connections 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ad00342b0 0x7f8ad0036770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/778770778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970f219a0 0x555970f21dc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 >> 172.21.15.170:0/778770778 conn(0x555970f167b0 msgr2=0x555970f172c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 shutdown_connections 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/778770778 wait complete. 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f0d3d0 msgr2=0x555970f0f7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555970f0d3d0 0x555970f0f7c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970e12700 msgr2=0x555970f0abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555970e12700 0x555970f0abc0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555970f11090 msgr2=0x555970f13480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555970f11090 0x555970f13480 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ae00342b0 msgr2=0x7f8ae0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8ae00342b0 0x7f8ae0036770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8be0000f70 tx=0x7f8be0001380 comp rx=0 tx=0).stop 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 msgr2=0x555970d87600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 --2- 172.21.15.170:0/125784543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555970e11de0 0x555970d87600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8be801aec0 tx=0x7f8be8017e30 comp rx=0 tx=0).stop 2024-03-28T16:49:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bf86e1640 1 -- 172.21.15.170:0/125784543 reap_dead start 2024-03-28T16:49:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 shutdown_connections 2024-03-28T16:49:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 >> 172.21.15.170:0/125784543 conn(0x555970e0d6b0 msgr2=0x555970ef69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 shutdown_connections 2024-03-28T16:49:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.687+0000 7f8bfa3c5980 1 -- 172.21.15.170:0/125784543 wait complete. 2024-03-28T16:49:15.697 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:15.697 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:15.697 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:15.698 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:15.769 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:15.769 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:15.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044d29f980 1 Processor -- start 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044d29f980 1 -- start start 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044d29f980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5595cf60d980 0x5595cf60bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044d29f980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5595cf5e3520 con 0x5595cf60c4c0 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044d29f980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5595cf5aa0f0 con 0x5595cf60d980 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044adba640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5595cf60d980 0x5595cf60bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044adba640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5595cf60d980 0x5595cf60bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55746/0 (socket says 172.21.15.170:55746) 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044adba640 1 -- 172.21.15.170:0/2062911546 learned_addr learned my addr 172.21.15.170:0/2062911546 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.771 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044a5b9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5595cf60c4c0 0x5595cf69ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60938/0 (socket says 172.21.15.170:60938) 2024-03-28T16:49:15.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044adba640 1 -- 172.21.15.170:0/2062911546 >> v1:172.21.15.119:6789/0 conn(0x5595cf60c4c0 legacy=0x5595cf69ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.771+0000 7f044adba640 1 -- 172.21.15.170:0/2062911546 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595cf60c8d0 con 0x5595cf60d980 2024-03-28T16:49:15.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044adba640 1 --2- 172.21.15.170:0/2062911546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60d980 0x5595cf60bf80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f043c00a5c0 tx=0x7f043c01b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34dae7014bb8814 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f0449db8640 1 -- 172.21.15.170:0/2062911546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0aa530 con 0x5595cf60d980 2024-03-28T16:49:15.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f0449db8640 1 -- 172.21.15.170:0/2062911546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c0a6050 con 0x5595cf60d980 2024-03-28T16:49:15.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f0449db8640 1 -- 172.21.15.170:0/2062911546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0aa7e0 con 0x5595cf60d980 2024-03-28T16:49:15.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044d29f980 1 -- 172.21.15.170:0/2062911546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60d980 msgr2=0x5595cf60bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044d29f980 1 --2- 172.21.15.170:0/2062911546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60d980 0x5595cf60bf80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f043c00a5c0 tx=0x7f043c01b2e0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044d29f980 1 -- 172.21.15.170:0/2062911546 shutdown_connections 2024-03-28T16:49:15.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044d29f980 1 --2- 172.21.15.170:0/2062911546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60d980 0x5595cf60bf80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044d29f980 1 -- 172.21.15.170:0/2062911546 >> 172.21.15.170:0/2062911546 conn(0x5595cf692850 msgr2=0x5595cf694c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044d29f980 1 -- 172.21.15.170:0/2062911546 shutdown_connections 2024-03-28T16:49:15.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.775+0000 7f044d29f980 1 -- 172.21.15.170:0/2062911546 wait complete. 2024-03-28T16:49:15.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044d29f980 1 Processor -- start 2024-03-28T16:49:15.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044d29f980 1 -- start start 2024-03-28T16:49:15.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044d29f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf783560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044d29f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cf5a7620 con 0x5595cf60c4c0 2024-03-28T16:49:15.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf783560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf783560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55752/0 (socket says 172.21.15.170:55752) 2024-03-28T16:49:15.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044adba640 1 -- 172.21.15.170:0/2283903767 learned_addr learned my addr 172.21.15.170:0/2283903767 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.787+0000 7f044adba640 1 -- 172.21.15.170:0/2283903767 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c005df0 con 0x5595cf60c4c0 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044adba640 1 --2- 172.21.15.170:0/2283903767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf783560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f043c018f70 tx=0x7f043c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea474b37b41ce6fd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f0338e09640 1 -- 172.21.15.170:0/2283903767 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0aa800 con 0x5595cf60c4c0 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f0338e09640 1 -- 172.21.15.170:0/2283903767 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c0aa960 con 0x5595cf60c4c0 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f0338e09640 1 -- 172.21.15.170:0/2283903767 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0aac20 con 0x5595cf60c4c0 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- 172.21.15.170:0/2283903767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 msgr2=0x5595cf783560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 --2- 172.21.15.170:0/2283903767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf783560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f043c018f70 tx=0x7f043c003030 comp rx=0 tx=0).stop 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- 172.21.15.170:0/2283903767 shutdown_connections 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 --2- 172.21.15.170:0/2283903767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf783560 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- 172.21.15.170:0/2283903767 >> 172.21.15.170:0/2283903767 conn(0x5595cf692850 msgr2=0x5595cf77f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- 172.21.15.170:0/2283903767 shutdown_connections 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- 172.21.15.170:0/2283903767 wait complete. 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 Processor -- start 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- start start 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf78a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cf5a2e00 con 0x5595cf60c4c0 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf78a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf78a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55760/0 (socket says 172.21.15.170:55760) 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044adba640 1 -- 172.21.15.170:0/365999673 learned_addr learned my addr 172.21.15.170:0/365999673 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044adba640 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595cf60c8d0 con 0x5595cf60c4c0 2024-03-28T16:49:15.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044adba640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf78a290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f043c01bb80 tx=0x7f043c0180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f0337606640 1 -- 172.21.15.170:0/365999673 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0c2070 con 0x5595cf60c4c0 2024-03-28T16:49:15.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5595cf78a7d0 con 0x5595cf60c4c0 2024-03-28T16:49:15.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595cf78aba0 con 0x5595cf60c4c0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f0337606640 1 -- 172.21.15.170:0/365999673 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c0bdaf0 con 0x5595cf60c4c0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.791+0000 7f0337606640 1 -- 172.21.15.170:0/365999673 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0cf450 con 0x5595cf60c4c0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f0337606640 1 -- 172.21.15.170:0/365999673 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f043c0b1070 con 0x5595cf60c4c0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f0337606640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03300342b0 0x7f0330036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f0337606640 1 -- 172.21.15.170:0/365999673 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f043c0e37b0 con 0x5595cf60c4c0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044a5b9640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03300342b0 0x7f0330036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf60d980 0x5595cf790e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7913b0 con 0x5595cf60d980 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044b5bb640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf60d980 0x5595cf790e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044a5b9640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03300342b0 0x7f0330036770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0434000f70 tx=0x7f0434001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044b5bb640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf60d980 0x5595cf790e70 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044b5bb640 1 -- 172.21.15.170:0/365999673 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf60d980 2024-03-28T16:49:15.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf791710 con 0x5595cf60d980 2024-03-28T16:49:15.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044b5bb640 1 -- 172.21.15.170:0/365999673 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf60d980 2024-03-28T16:49:15.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7930a0 0x5595cf795540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf795cd0 con 0x5595cf7930a0 2024-03-28T16:49:15.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044adba640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7930a0 0x5595cf795540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044adba640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7930a0 0x5595cf795540 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044adba640 1 -- 172.21.15.170:0/365999673 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7930a0 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf791f10 con 0x5595cf60d980 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.795+0000 7f044b5bb640 1 -- 172.21.15.170:0/365999673 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf60d980 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5595cf7970f0 0x5595cf7994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf799b40 con 0x5595cf7970f0 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044a5b9640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5595cf7970f0 0x5595cf7994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044a5b9640 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5595cf7970f0 0x5595cf7994e0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044a5b9640 1 -- 172.21.15.170:0/365999673 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f043400f6a0 con 0x5595cf7970f0 2024-03-28T16:49:15.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 Processor -- start 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 -- start start 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7a7a70 0x5595cf7a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cf53be30 con 0x5595cf7a7a70 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7a7a70 0x5595cf7a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7a7a70 0x5595cf7a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55770/0 (socket says 172.21.15.170:55770) 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044adba640 1 -- 172.21.15.170:0/2899736277 learned_addr learned my addr 172.21.15.170:0/2899736277 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044adba640 1 -- 172.21.15.170:0/2899736277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595cf7a83d0 con 0x5595cf7a7a70 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044adba640 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7a7a70 0x5595cf7a7e90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f043c000bf0 tx=0x7f043c0b4d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f0334e01640 1 -- 172.21.15.170:0/2899736277 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0d5040 con 0x5595cf7a7a70 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f0334e01640 1 -- 172.21.15.170:0/2899736277 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c0b4810 con 0x5595cf7a7a70 2024-03-28T16:49:15.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5595cf7a8620 con 0x5595cf7a7a70 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f0334e01640 1 -- 172.21.15.170:0/2899736277 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0e3b50 con 0x5595cf7a7a70 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595cf7a9170 con 0x5595cf7a7a70 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.799+0000 7f0334e01640 1 -- 172.21.15.170:0/2899736277 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f043c0d8040 con 0x5595cf7a7a70 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f0334e01640 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03240342b0 0x7f0324036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f0334e01640 1 -- 172.21.15.170:0/2899736277 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f043c0edaa0 con 0x5595cf7a7a70 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044a5b9640 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03240342b0 0x7f0324036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 Processor -- start 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- start start 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044a5b9640 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03240342b0 0x7f0324036770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5595cf792270 tx=0x7f043400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cf52f790 con 0x5595cf7b4a10 2024-03-28T16:49:15.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55772/0 (socket says 172.21.15.170:55772) 2024-03-28T16:49:15.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 -- 172.21.15.170:0/1280748838 learned_addr learned my addr 172.21.15.170:0/1280748838 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 -- 172.21.15.170:0/1280748838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595cf7b53e0 con 0x5595cf7b4a10 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 --2- 172.21.15.170:0/1280748838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7b4e10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f043c0c56a0 tx=0x7f043c0da3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48a6d846611f0b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f032e7fc640 1 -- 172.21.15.170:0/1280748838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0e0890 con 0x5595cf7b4a10 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f032e7fc640 1 -- 172.21.15.170:0/1280748838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c0f4040 con 0x5595cf7b4a10 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f032e7fc640 1 -- 172.21.15.170:0/1280748838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0dc420 con 0x5595cf7b4a10 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- 172.21.15.170:0/1280748838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 msgr2=0x5595cf7b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 --2- 172.21.15.170:0/1280748838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7b4e10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f043c0c56a0 tx=0x7f043c0da3f0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- 172.21.15.170:0/1280748838 shutdown_connections 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 --2- 172.21.15.170:0/1280748838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7b4e10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- 172.21.15.170:0/1280748838 >> 172.21.15.170:0/1280748838 conn(0x5595cf7b0d50 msgr2=0x5595cf7aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- 172.21.15.170:0/1280748838 shutdown_connections 2024-03-28T16:49:15.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- 172.21.15.170:0/1280748838 wait complete. 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 Processor -- start 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- start start 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7bb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044d29f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595cf7bbd60 con 0x5595cf7b4a10 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7bb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7bb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55774/0 (socket says 172.21.15.170:55774) 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 learned_addr learned my addr 172.21.15.170:0/260035476 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.803+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595cf7a9170 con 0x5595cf7b4a10 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044adba640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7bb820 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f043c0da430 tx=0x7f043c0baf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f032cff9640 1 -- 172.21.15.170:0/260035476 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0dcae0 con 0x5595cf7b4a10 2024-03-28T16:49:15.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f032cff9640 1 -- 172.21.15.170:0/260035476 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c0da800 con 0x5595cf7b4a10 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5595cf7bbf60 con 0x5595cf7b4a10 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f032cff9640 1 -- 172.21.15.170:0/260035476 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f043c0daaa0 con 0x5595cf7b4a10 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595cf7bbf60 con 0x5595cf7b4a10 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f032cff9640 1 -- 172.21.15.170:0/260035476 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f043c101040 con 0x5595cf7b4a10 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f032cff9640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03140342b0 0x7f0314036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f032cff9640 1 -- 172.21.15.170:0/260035476 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f043c0dfdb0 con 0x5595cf7b4a10 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044a5b9640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03140342b0 0x7f0314036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf7c6ac0 0x5595cf7c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7c96a0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044b5bb640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf7c6ac0 0x5595cf7c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044a5b9640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03140342b0 0x7f0314036770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f043400c010 tx=0x7f0434010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044b5bb640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf7c6ac0 0x5595cf7c8f50 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 realm 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7c9a70 con 0x5595cf7c6ac0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca650 con 0x5595cf7c6ac0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7cb000 0x5595cf7cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7cd930 con 0x5595cf7cb000 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.807+0000 7f044adba640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7cb000 0x5595cf7cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044adba640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7cb000 0x5595cf7cd3f0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044d29f980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044d29f980 20 searching for the correct realm 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5595cf7cf7c0 0x5595cf7d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d2320 con 0x5595cf7cf7c0 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044a5b9640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5595cf7cf7c0 0x5595cf7d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044a5b9640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5595cf7cf7c0 0x5595cf7d1c50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044a5b9640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f043401fc60 0x7f043403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d2320 con 0x7f043401fc60 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044b5bb640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f043401fc60 0x7f043403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044b5bb640 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f043401fc60 0x7f043403a3c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.811+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x7f043401fc60 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x7f043401fc60 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.815+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x7f043401fc60 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.819+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x7f043401fc60 2024-03-28T16:49:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7cb000 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ca2f0 con 0x7f043401fc60 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ce130 con 0x5595cf7c6ac0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ce130 con 0x5595cf7c6ac0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ce130 con 0x7f043401fc60 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 zone default found 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 4 Realm: () 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 zonegroup default 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ce980 con 0x5595cf7cb000 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7ced80 con 0x5595cf7cf7c0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.823+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d42a0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 20 started sync module instance, tier type = 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d3a20 con 0x7f043401fc60 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94101919577472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d3dd0 con 0x7f043401fc60 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d5000 con 0x5595cf7c6ac0 2024-03-28T16:49:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94101919577472] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94101919581888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d4fc0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.827+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d5a60 con 0x5595cf7cf7c0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94101919585600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d62b0 con 0x5595cf7cf7c0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d6ca0 con 0x7f043401fc60 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94101919581888] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94101919590272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d7050 con 0x7f043401fc60 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d79c0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94101919585600] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.831+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94101919590272] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.835+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.835+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94101919582944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d7520 con 0x5595cf7c6ac0 2024-03-28T16:49:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.835+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d7d20 con 0x5595cf7cb000 2024-03-28T16:49:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.835+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94101919582944] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.835+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.835+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94101919593760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d7940 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.835+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d89c0 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94101919593760] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94101919597184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d91c0 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d9b30 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94101919602592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7da260 con 0x5595cf7cb000 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 20 add_watcher() i=0 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 20 add_watcher() i=1 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 20 add_watcher() i=2 2024-03-28T16:49:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 20 add_watcher() i=3 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 20 add_watcher() i=4 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 20 add_watcher() i=5 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94101919597184] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.839+0000 7f044d29f980 20 add_watcher() i=6 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94101919602592] v29'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 20 add_watcher() i=7 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d8d20 con 0x5595cf7c6ac0 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94101919603968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d8d20 con 0x5595cf7c6ac0 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94101919603968] v29'74 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f03017e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 5 note: GC not initialized 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f027bfff640 20 reqs_thread_entry: start 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 20 Filter name: none 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f0279ffb640 20 reqs_thread_entry: start 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7fee10 con 0x5595cf7c6ac0 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.843+0000 7f044d29f980 20 rados_obj.operate() r=0 bl.length=513 2024-03-28T16:49:15.852 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:15.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f03017e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:15.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94101919603968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7fde70 con 0x5595cf7c6ac0 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94101919603968] v29'75 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94101919577472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d9cc0 con 0x7f043401fc60 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94101919577472] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f044d29f980 20 remove_watcher() i=0 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f044d29f980 2 removed watcher, disabling cache 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.847+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94101919581888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d3580 con 0x5595cf7c6ac0 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94101919581888] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044d29f980 20 remove_watcher() i=1 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94101919585600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d3580 con 0x5595cf7cf7c0 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044a5b9640 1 -- 172.21.15.170:0/260035476 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94101919585600] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0434033040 con 0x5595cf7cf7c0 2024-03-28T16:49:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044d29f980 20 remove_watcher() i=2 2024-03-28T16:49:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94101919590272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d5540 con 0x7f043401fc60 2024-03-28T16:49:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94101919590272] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x7f043401fc60 2024-03-28T16:49:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044d29f980 20 remove_watcher() i=3 2024-03-28T16:49:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.851+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94101919582944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d5540 con 0x5595cf7c6ac0 2024-03-28T16:49:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.855+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94101919582944] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f044400a4d0 con 0x5595cf7c6ac0 2024-03-28T16:49:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.855+0000 7f044d29f980 20 remove_watcher() i=4 2024-03-28T16:49:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.855+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94101919593760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d4b60 con 0x5595cf7cb000 2024-03-28T16:49:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.855+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94101919593760] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.855+0000 7f044d29f980 20 remove_watcher() i=5 2024-03-28T16:49:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.855+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94101919597184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d7520 con 0x5595cf7cb000 2024-03-28T16:49:15.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94101919597184] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 20 remove_watcher() i=6 2024-03-28T16:49:15.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94101919602592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5595cf7d7520 con 0x5595cf7cb000 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044adba640 1 -- 172.21.15.170:0/260035476 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94101919602592] v29'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f043c0cc070 con 0x5595cf7cb000 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 20 remove_watcher() i=7 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7cb000 msgr2=0x5595cf7cd3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7cb000 0x5595cf7cd3f0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf7c6ac0 msgr2=0x5595cf7c8f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf7c6ac0 0x5595cf7c8f50 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f043401fc60 msgr2=0x7f043403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f043401fc60 0x7f043403a3c0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5595cf7cf7c0 msgr2=0x5595cf7d1c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5595cf7cf7c0 0x5595cf7d1c50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03140342b0 msgr2=0x7f0314036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03140342b0 0x7f0314036770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f043400c010 tx=0x7f0434010000 comp rx=0 tx=0).stop 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 msgr2=0x5595cf7bb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044b5bb640 1 -- 172.21.15.170:0/260035476 reap_dead start 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7bb820 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f043c0da430 tx=0x7f043c0baf50 comp rx=0 tx=0).stop 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 shutdown_connections 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/260035476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7b4a10 0x5595cf7bb820 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 >> 172.21.15.170:0/260035476 conn(0x5595cf7b0d50 msgr2=0x5595cf7ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 shutdown_connections 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/260035476 wait complete. 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03240342b0 msgr2=0x7f0324036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03240342b0 0x7f0324036770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5595cf792270 tx=0x7f043400f000 comp rx=0 tx=0).stop 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7a7a70 msgr2=0x5595cf7a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7a7a70 0x5595cf7a7e90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f043c000bf0 tx=0x7f043c0b4d30 comp rx=0 tx=0).stop 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 shutdown_connections 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03240342b0 0x7f0324036770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 --2- 172.21.15.170:0/2899736277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf7a7a70 0x5595cf7a7e90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 >> 172.21.15.170:0/2899736277 conn(0x5595cf79c880 msgr2=0x5595cf79d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 shutdown_connections 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.859+0000 7f044d29f980 1 -- 172.21.15.170:0/2899736277 wait complete. 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7930a0 msgr2=0x5595cf795540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5595cf7930a0 0x5595cf795540 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf60d980 msgr2=0x5595cf790e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5595cf60d980 0x5595cf790e70 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5595cf7970f0 msgr2=0x5595cf7994e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5595cf7970f0 0x5595cf7994e0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03300342b0 msgr2=0x7f0330036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f03300342b0 0x7f0330036770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0434000f70 tx=0x7f0434001380 comp rx=0 tx=0).stop 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 msgr2=0x5595cf78a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 --2- 172.21.15.170:0/365999673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5595cf60c4c0 0x5595cf78a290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f043c01bb80 tx=0x7f043c0180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044b5bb640 1 -- 172.21.15.170:0/365999673 reap_dead start 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 shutdown_connections 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 >> 172.21.15.170:0/365999673 conn(0x5595cf692850 msgr2=0x5595cf77dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 shutdown_connections 2024-03-28T16:49:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.863+0000 7f044d29f980 1 -- 172.21.15.170:0/365999673 wait complete. 2024-03-28T16:49:15.871 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:15.871 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:15.871 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:15.871 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:15.946 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:15.946 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:15.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0aac17980 1 Processor -- start 2024-03-28T16:49:15.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0aac17980 1 -- start start 2024-03-28T16:49:15.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0aac17980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ce492cb630 0x55ce492cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0aac17980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ce49211520 con 0x55ce492c7130 2024-03-28T16:49:15.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0aac17980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ce491d80f0 con 0x55ce492cb630 2024-03-28T16:49:15.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0a7f31640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ce492cb630 0x55ce492cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0a7f31640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ce492cb630 0x55ce492cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55786/0 (socket says 172.21.15.170:55786) 2024-03-28T16:49:15.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0a7f31640 1 -- 172.21.15.170:0/208602364 learned_addr learned my addr 172.21.15.170:0/208602364 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0a7f31640 1 -- 172.21.15.170:0/208602364 >> v1:172.21.15.119:6789/0 conn(0x55ce492c7130 legacy=0x55ce492c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.947+0000 7fb0a7f31640 1 -- 172.21.15.170:0/208602364 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce492cef70 con 0x55ce492cb630 2024-03-28T16:49:15.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/208602364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce492cda20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00a5c0 tx=0x7fb09c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2aaf2aeffd0765 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0a7730640 1 -- 172.21.15.170:0/208602364 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb09c0aa470 con 0x55ce492cb630 2024-03-28T16:49:15.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0a7730640 1 -- 172.21.15.170:0/208602364 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c0aa5d0 con 0x55ce492cb630 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0a7730640 1 -- 172.21.15.170:0/208602364 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb09c0aa890 con 0x55ce492cb630 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0aac17980 1 -- 172.21.15.170:0/208602364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 msgr2=0x55ce492cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0aac17980 1 --2- 172.21.15.170:0/208602364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce492cda20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00a5c0 tx=0x7fb09c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0aac17980 1 -- 172.21.15.170:0/208602364 shutdown_connections 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0aac17980 1 --2- 172.21.15.170:0/208602364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce492cda20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0aac17980 1 -- 172.21.15.170:0/208602364 >> 172.21.15.170:0/208602364 conn(0x55ce4923a850 msgr2=0x55ce4923ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0aac17980 1 -- 172.21.15.170:0/208602364 shutdown_connections 2024-03-28T16:49:15.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.951+0000 7fb0aac17980 1 -- 172.21.15.170:0/208602364 wait complete. 2024-03-28T16:49:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.963+0000 7fb0aac17980 1 Processor -- start 2024-03-28T16:49:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.963+0000 7fb0aac17980 1 -- start start 2024-03-28T16:49:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.963+0000 7fb0aac17980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492c7130 0x55ce493b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.963+0000 7fb0aac17980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce491d5620 con 0x55ce492c7130 2024-03-28T16:49:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.963+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492c7130 0x55ce493b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.963+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492c7130 0x55ce493b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55788/0 (socket says 172.21.15.170:55788) 2024-03-28T16:49:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.963+0000 7fb0a8732640 1 -- 172.21.15.170:0/4090553444 learned_addr learned my addr 172.21.15.170:0/4090553444 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0a8732640 1 -- 172.21.15.170:0/4090553444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c005df0 con 0x55ce492c7130 2024-03-28T16:49:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0a8732640 1 --2- 172.21.15.170:0/4090553444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492c7130 0x55ce493b0fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb098001d10 tx=0x7fb09800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cd42db4c8f7331 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7faf9660c640 1 -- 172.21.15.170:0/4090553444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098014070 con 0x55ce492c7130 2024-03-28T16:49:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7faf9660c640 1 -- 172.21.15.170:0/4090553444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800f560 con 0x55ce492c7130 2024-03-28T16:49:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7faf9660c640 1 -- 172.21.15.170:0/4090553444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb09800f840 con 0x55ce492c7130 2024-03-28T16:49:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- 172.21.15.170:0/4090553444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492c7130 msgr2=0x55ce493b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 --2- 172.21.15.170:0/4090553444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492c7130 0x55ce493b0fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb098001d10 tx=0x7fb09800b5c0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- 172.21.15.170:0/4090553444 shutdown_connections 2024-03-28T16:49:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 --2- 172.21.15.170:0/4090553444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492c7130 0x55ce493b0fe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- 172.21.15.170:0/4090553444 >> 172.21.15.170:0/4090553444 conn(0x55ce4923a850 msgr2=0x55ce493acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- 172.21.15.170:0/4090553444 shutdown_connections 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- 172.21.15.170:0/4090553444 wait complete. 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 Processor -- start 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- start start 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce493b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce491d0e00 con 0x55ce492cb630 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce493b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce493b8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55800/0 (socket says 172.21.15.170:55800) 2024-03-28T16:49:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0a8732640 1 -- 172.21.15.170:0/3646350500 learned_addr learned my addr 172.21.15.170:0/3646350500 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0a8732640 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce49211520 con 0x55ce492cb630 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0a8732640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce493b8120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb098002000 tx=0x7fb098000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7faf94e09640 1 -- 172.21.15.170:0/3646350500 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098014050 con 0x55ce492cb630 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce493b8660 con 0x55ce492cb630 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7faf94e09640 1 -- 172.21.15.170:0/3646350500 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800fe70 con 0x55ce492cb630 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.967+0000 7faf94e09640 1 -- 172.21.15.170:0/3646350500 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb09801b8e0 con 0x55ce492cb630 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce493b8af0 con 0x55ce492cb630 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7faf94e09640 1 -- 172.21.15.170:0/3646350500 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb098022020 con 0x55ce492cb630 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7faf94e09640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf900342b0 0x7faf90036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7faf94e09640 1 -- 172.21.15.170:0/3646350500 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb098048d80 con 0x55ce492cb630 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf900342b0 0x7faf90036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493bcd40 0x55ce493bf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493bf670 con 0x55ce493bcd40 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a8f33640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493bcd40 0x55ce493bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf900342b0 0x7faf90036770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0ae000 tx=0x7fb09c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a8f33640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493bcd40 0x55ce493bf130 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a8f33640 1 -- 172.21.15.170:0/3646350500 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493bcd40 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493bf9d0 con 0x55ce493bcd40 2024-03-28T16:49:15.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a8f33640 1 -- 172.21.15.170:0/3646350500 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493bcd40 2024-03-28T16:49:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493c1360 0x55ce493c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493c3f90 con 0x55ce493c1360 2024-03-28T16:49:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a8732640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493c1360 0x55ce493c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.971+0000 7fb0a8732640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493c1360 0x55ce493c3800 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a8732640 1 -- 172.21.15.170:0/3646350500 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fb098016070 con 0x55ce493c1360 2024-03-28T16:49:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493c01d0 con 0x55ce493bcd40 2024-03-28T16:49:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a8f33640 1 -- 172.21.15.170:0/3646350500 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493bcd40 2024-03-28T16:49:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ce493c53b0 0x55ce493c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493c7e60 con 0x55ce493c53b0 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ce493c53b0 0x55ce493c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ce493c53b0 0x55ce493c77a0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a7f31640 1 -- 172.21.15.170:0/3646350500 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493c53b0 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 Processor -- start 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 -- start start 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493d5d30 0x55ce493d6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce49169e30 con 0x55ce493d5d30 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493d5d30 0x55ce493d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493d5d30 0x55ce493d6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55810/0 (socket says 172.21.15.170:55810) 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a8732640 1 -- 172.21.15.170:0/689893385 learned_addr learned my addr 172.21.15.170:0/689893385 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a8732640 1 -- 172.21.15.170:0/689893385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce493d6690 con 0x55ce493d5d30 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0a8732640 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493d5d30 0x55ce493d6150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb098001670 tx=0x7fb09802e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7faf8dffb640 1 -- 172.21.15.170:0/689893385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098024830 con 0x55ce493d5d30 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7faf8dffb640 1 -- 172.21.15.170:0/689893385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb098036070 con 0x55ce493d5d30 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce493d68e0 con 0x55ce493d5d30 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7faf8dffb640 1 -- 172.21.15.170:0/689893385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098040b70 con 0x55ce493d5d30 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.975+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce493d7430 con 0x55ce493d5d30 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7faf8dffb640 1 -- 172.21.15.170:0/689893385 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb098040cd0 con 0x55ce493d5d30 2024-03-28T16:49:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7faf8dffb640 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf800342b0 0x7faf80036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7faf8dffb640 1 -- 172.21.15.170:0/689893385 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb0980320b0 con 0x55ce493d5d30 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf800342b0 0x7faf80036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 Processor -- start 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- start start 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce4915d790 con 0x55ce493e2c60 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf800342b0 0x7faf80036770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55ce493c4320 tx=0x7fb09c005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55812/0 (socket says 172.21.15.170:55812) 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 -- 172.21.15.170:0/4122127449 learned_addr learned my addr 172.21.15.170:0/4122127449 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 -- 172.21.15.170:0/4122127449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce493e3650 con 0x55ce493e2c60 2024-03-28T16:49:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 --2- 172.21.15.170:0/4122127449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e3080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb098008fd0 tx=0x7fb098051b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f76166f85d3c6e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7faf877fe640 1 -- 172.21.15.170:0/4122127449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0980410b0 con 0x55ce493e2c60 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7faf877fe640 1 -- 172.21.15.170:0/4122127449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb098057050 con 0x55ce493e2c60 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7faf877fe640 1 -- 172.21.15.170:0/4122127449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098051550 con 0x55ce493e2c60 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- 172.21.15.170:0/4122127449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 msgr2=0x55ce493e3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 --2- 172.21.15.170:0/4122127449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e3080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb098008fd0 tx=0x7fb098051b70 comp rx=0 tx=0).stop 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- 172.21.15.170:0/4122127449 shutdown_connections 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 --2- 172.21.15.170:0/4122127449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e3080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- 172.21.15.170:0/4122127449 >> 172.21.15.170:0/4122127449 conn(0x55ce493defa0 msgr2=0x55ce493dd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- 172.21.15.170:0/4122127449 shutdown_connections 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- 172.21.15.170:0/4122127449 wait complete. 2024-03-28T16:49:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 Processor -- start 2024-03-28T16:49:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- start start 2024-03-28T16:49:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0aac17980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce493e9fd0 con 0x55ce493e2c60 2024-03-28T16:49:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55818/0 (socket says 172.21.15.170:55818) 2024-03-28T16:49:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.979+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 learned_addr learned my addr 172.21.15.170:0/1973487767 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce493d7430 con 0x55ce493e2c60 2024-03-28T16:49:15.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8732640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e9a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb098051bb0 tx=0x7fb098015f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7faf85ffb640 1 -- 172.21.15.170:0/1973487767 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0980479b0 con 0x55ce493e2c60 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7faf85ffb640 1 -- 172.21.15.170:0/1973487767 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb098047b10 con 0x55ce493e2c60 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce493ea1d0 con 0x55ce493e2c60 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7faf85ffb640 1 -- 172.21.15.170:0/1973487767 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb098047df0 con 0x55ce493e2c60 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce493ea1d0 con 0x55ce493e2c60 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7faf85ffb640 1 -- 172.21.15.170:0/1973487767 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb09803b870 con 0x55ce493e2c60 2024-03-28T16:49:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7faf85ffb640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf780342b0 0x7faf78036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7faf85ffb640 1 -- 172.21.15.170:0/1973487767 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb098072af0 con 0x55ce493e2c60 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf780342b0 0x7faf78036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493f4e80 0x55ce493f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f7a70 con 0x55ce493f4e80 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8f33640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493f4e80 0x55ce493f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf780342b0 0x7faf78036770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb09c01b380 tx=0x7fb09c0c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8f33640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493f4e80 0x55ce493f7320 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:15.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 realm 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f7e40 con 0x55ce493f4e80 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f8a20 con 0x55ce493f4e80 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493f93d0 0x55ce493fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.983+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493fbd00 con 0x55ce493f93d0 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8732640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493f93d0 0x55ce493fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8732640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493f93d0 0x55ce493fb7c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0aac17980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0aac17980 20 searching for the correct realm 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ce493fdb90 0x55ce49400050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49400720 con 0x55ce493fdb90 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ce493fdb90 0x55ce49400050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ce493fdb90 0x55ce49400050 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a7f31640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb09c0e0040 0x7fb09c0d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49400720 con 0x7fb09c0e0040 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8f33640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb09c0e0040 0x7fb09c0d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8f33640 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb09c0e0040 0x7fb09c0d3400 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.987+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x7fb09c0e0040 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x7fb09c0e0040 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.991+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x7fb09c0e0040 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.995+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x7fb09c0e0040 2024-03-28T16:49:16.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493fdb90 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f93d0 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x55ce493f4e80 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493f86c0 con 0x7fb09c0e0040 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:16.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493fc500 con 0x55ce493f4e80 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493fc500 con 0x55ce493f4e80 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493fc500 con 0x7fb09c0e0040 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 zone default found 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 4 Realm: () 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 zonegroup default 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:15.999+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493fcd50 con 0x55ce493f93d0 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce493fd130 con 0x55ce493fdb90 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49402630 con 0x55ce493fdb90 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 20 started sync module instance, tier type = 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49401e70 con 0x7fb09c0e0040 2024-03-28T16:49:16.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94344480561616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49402220 con 0x7fb09c0e0040 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49403390 con 0x55ce493f4e80 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.003+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94344480561616] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94344480565840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce494033b0 con 0x55ce493f4e80 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49403e50 con 0x55ce493fdb90 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94344480569648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce494046a0 con 0x55ce493fdb90 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49405070 con 0x7fb09c0e0040 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94344480565840] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94344480574320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49405420 con 0x7fb09c0e0040 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49405d90 con 0x55ce493f4e80 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.007+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94344480569648] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94344480574320] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94344480566896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce494058f0 con 0x55ce493f4e80 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce494060f0 con 0x55ce493f93d0 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94344480566896] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94344480577776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49405d10 con 0x55ce493f93d0 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.011+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49406de0 con 0x55ce493f93d0 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94344480577776] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94344480580688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49406d10 con 0x55ce493f93d0 2024-03-28T16:49:16.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49407740 con 0x55ce493f93d0 2024-03-28T16:49:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94344480584288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49407be0 con 0x55ce493f93d0 2024-03-28T16:49:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 20 add_watcher() i=0 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 20 add_watcher() i=1 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 20 add_watcher() i=2 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 20 add_watcher() i=3 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 20 add_watcher() i=4 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 20 add_watcher() i=5 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94344480580688] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.015+0000 7fb0aac17980 20 add_watcher() i=6 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94344480584288] v29'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 20 add_watcher() i=7 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49406870 con 0x55ce493f4e80 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94344480585536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49406870 con 0x55ce493f4e80 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94344480585536] v29'76 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7faf5e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 5 note: GC not initialized 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7faf5d7e2640 20 reqs_thread_entry: start 2024-03-28T16:49:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 20 Filter name: none 2024-03-28T16:49:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7faeceffd640 20 reqs_thread_entry: start 2024-03-28T16:49:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce4942c670 con 0x55ce493f4e80 2024-03-28T16:49:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.019+0000 7fb0aac17980 20 rados_obj.operate() r=0 bl.length=513 2024-03-28T16:49:16.027 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:16.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7faf5e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94344480585536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce4942b6d0 con 0x55ce493f4e80 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94344480585536] v29'77 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94344480561616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49407740 con 0x7fb09c0e0040 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94344480561616] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0aac17980 20 remove_watcher() i=0 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0aac17980 2 removed watcher, disabling cache 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94344480565840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce494019d0 con 0x55ce493f4e80 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94344480565840] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0aac17980 20 remove_watcher() i=1 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.023+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94344480569648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce494019d0 con 0x55ce493fdb90 2024-03-28T16:49:16.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.027+0000 7fb0a7f31640 1 -- 172.21.15.170:0/1973487767 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94344480569648] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb09c0b0020 con 0x55ce493fdb90 2024-03-28T16:49:16.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.027+0000 7fb0aac17980 20 remove_watcher() i=2 2024-03-28T16:49:16.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.027+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94344480574320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49403930 con 0x7fb09c0e0040 2024-03-28T16:49:16.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.027+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94344480574320] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x7fb09c0e0040 2024-03-28T16:49:16.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.027+0000 7fb0aac17980 20 remove_watcher() i=3 2024-03-28T16:49:16.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.027+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94344480566896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49404b70 con 0x55ce493f4e80 2024-03-28T16:49:16.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.031+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94344480566896] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb0a000a4d0 con 0x55ce493f4e80 2024-03-28T16:49:16.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.031+0000 7fb0aac17980 20 remove_watcher() i=4 2024-03-28T16:49:16.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.031+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94344480577776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49402ef0 con 0x55ce493f93d0 2024-03-28T16:49:16.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.031+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94344480577776] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.031+0000 7fb0aac17980 20 remove_watcher() i=5 2024-03-28T16:49:16.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.031+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94344480580688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49402ef0 con 0x55ce493f93d0 2024-03-28T16:49:16.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94344480580688] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 20 remove_watcher() i=6 2024-03-28T16:49:16.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94344480584288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ce49402ef0 con 0x55ce493f93d0 2024-03-28T16:49:16.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0a8732640 1 -- 172.21.15.170:0/1973487767 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94344480584288] v29'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb098062020 con 0x55ce493f93d0 2024-03-28T16:49:16.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 20 remove_watcher() i=7 2024-03-28T16:49:16.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493f93d0 msgr2=0x55ce493fb7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493f93d0 0x55ce493fb7c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493f4e80 msgr2=0x55ce493f7320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493f4e80 0x55ce493f7320 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb09c0e0040 msgr2=0x7fb09c0d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb09c0e0040 0x7fb09c0d3400 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ce493fdb90 msgr2=0x55ce49400050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ce493fdb90 0x55ce49400050 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf780342b0 msgr2=0x7faf78036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf780342b0 0x7faf78036770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb09c01b380 tx=0x7fb09c0c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 msgr2=0x55ce493e9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0a8f33640 1 -- 172.21.15.170:0/1973487767 reap_dead start 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e9a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb098051bb0 tx=0x7fb098015f10 comp rx=0 tx=0).stop 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 shutdown_connections 2024-03-28T16:49:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/1973487767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493e2c60 0x55ce493e9a90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 >> 172.21.15.170:0/1973487767 conn(0x55ce493defa0 msgr2=0x55ce493dc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 shutdown_connections 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/1973487767 wait complete. 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf800342b0 msgr2=0x7faf80036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf800342b0 0x7faf80036770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55ce493c4320 tx=0x7fb09c005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493d5d30 msgr2=0x55ce493d6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493d5d30 0x55ce493d6150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb098001670 tx=0x7fb09802e9e0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 shutdown_connections 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf800342b0 0x7faf80036770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 --2- 172.21.15.170:0/689893385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce493d5d30 0x55ce493d6150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 >> 172.21.15.170:0/689893385 conn(0x55ce493cab40 msgr2=0x55ce493cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 shutdown_connections 2024-03-28T16:49:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.035+0000 7fb0aac17980 1 -- 172.21.15.170:0/689893385 wait complete. 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493c1360 msgr2=0x55ce493c3800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ce493c1360 0x55ce493c3800 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493bcd40 msgr2=0x55ce493bf130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ce493bcd40 0x55ce493bf130 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ce493c53b0 msgr2=0x55ce493c77a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ce493c53b0 0x55ce493c77a0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf900342b0 msgr2=0x7faf90036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7faf900342b0 0x7faf90036770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0ae000 tx=0x7fb09c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 msgr2=0x55ce493b8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 --2- 172.21.15.170:0/3646350500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ce492cb630 0x55ce493b8120 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb098002000 tx=0x7fb098000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0a8f33640 1 -- 172.21.15.170:0/3646350500 reap_dead start 2024-03-28T16:49:16.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 shutdown_connections 2024-03-28T16:49:16.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 >> 172.21.15.170:0/3646350500 conn(0x55ce4923a850 msgr2=0x55ce493ab680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 shutdown_connections 2024-03-28T16:49:16.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.039+0000 7fb0aac17980 1 -- 172.21.15.170:0/3646350500 wait complete. 2024-03-28T16:49:16.047 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:16.047 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-03-28T16:49:16.047 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-03-28T16:49:16.047 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-03-28T16:49:16.126 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:16.126 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4feaa9980 1 Processor -- start 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4feaa9980 1 -- start start 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4feaa9980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b411399850 0x55b411429fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4feaa9980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b4113714c0 con 0x55b41142a510 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4feaa9980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b4113380f0 con 0x55b411399850 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4fc5c4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b411399850 0x55b411429fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4fc5c4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b411399850 0x55b411429fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55834/0 (socket says 172.21.15.170:55834) 2024-03-28T16:49:16.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4fbdc3640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55b41142a510 0x55b41142c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60966/0 (socket says 172.21.15.170:60966) 2024-03-28T16:49:16.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/3107830711 learned_addr learned my addr 172.21.15.170:0/3107830711 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/3107830711 >> v1:172.21.15.119:6789/0 conn(0x55b41142a510 legacy=0x55b41142c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.127+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/3107830711 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4114259f0 con 0x55b411399850 2024-03-28T16:49:16.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/3107830711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411429fd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00a5c0 tx=0x7fb4ec01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca82c6e375a7706 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4fb5c2640 1 -- 172.21.15.170:0/3107830711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0aa470 con 0x55b411399850 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4fb5c2640 1 -- 172.21.15.170:0/3107830711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0aa5d0 con 0x55b411399850 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4fb5c2640 1 -- 172.21.15.170:0/3107830711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0aa890 con 0x55b411399850 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4feaa9980 1 -- 172.21.15.170:0/3107830711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 msgr2=0x55b411429fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/3107830711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411429fd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00a5c0 tx=0x7fb4ec01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4feaa9980 1 -- 172.21.15.170:0/3107830711 shutdown_connections 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/3107830711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411429fd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4feaa9980 1 -- 172.21.15.170:0/3107830711 >> 172.21.15.170:0/3107830711 conn(0x55b41141f6b0 msgr2=0x55b411421ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4feaa9980 1 -- 172.21.15.170:0/3107830711 shutdown_connections 2024-03-28T16:49:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.131+0000 7fb4feaa9980 1 -- 172.21.15.170:0/3107830711 wait complete. 2024-03-28T16:49:16.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.143+0000 7fb4feaa9980 1 Processor -- start 2024-03-28T16:49:16.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.143+0000 7fb4feaa9980 1 -- start start 2024-03-28T16:49:16.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b4115110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b411335620 con 0x55b411399850 2024-03-28T16:49:16.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b4115110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b4115110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55840/0 (socket says 172.21.15.170:55840) 2024-03-28T16:49:16.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/1434372229 learned_addr learned my addr 172.21.15.170:0/1434372229 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/1434372229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec005df0 con 0x55b411399850 2024-03-28T16:49:16.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/1434372229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b4115110c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec018f70 tx=0x7fb4ec003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feadb1057fced376 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb3ea60c640 1 -- 172.21.15.170:0/1434372229 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0aa9c0 con 0x55b411399850 2024-03-28T16:49:16.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb3ea60c640 1 -- 172.21.15.170:0/1434372229 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0aab20 con 0x55b411399850 2024-03-28T16:49:16.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb3ea60c640 1 -- 172.21.15.170:0/1434372229 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0aadc0 con 0x55b411399850 2024-03-28T16:49:16.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1434372229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 msgr2=0x55b4115110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/1434372229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b4115110c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec018f70 tx=0x7fb4ec003030 comp rx=0 tx=0).stop 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1434372229 shutdown_connections 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/1434372229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b4115110c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1434372229 >> 172.21.15.170:0/1434372229 conn(0x55b41141f6b0 msgr2=0x55b41150cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1434372229 shutdown_connections 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1434372229 wait complete. 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 Processor -- start 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- start start 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411517e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4feaa9980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b411330e00 con 0x55b411399850 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411517e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411517e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55842/0 (socket says 172.21.15.170:55842) 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/4039176564 learned_addr learned my addr 172.21.15.170:0/4039176564 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.147+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4114259f0 con 0x55b411399850 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411517e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec018f70 tx=0x7fb4ec0180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb3e8e09640 1 -- 172.21.15.170:0/4039176564 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0cb070 con 0x55b411399850 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b4115183b0 con 0x55b411399850 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb3e8e09640 1 -- 172.21.15.170:0/4039176564 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0bd810 con 0x55b411399850 2024-03-28T16:49:16.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b411518780 con 0x55b411399850 2024-03-28T16:49:16.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb3e8e09640 1 -- 172.21.15.170:0/4039176564 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0bdab0 con 0x55b411399850 2024-03-28T16:49:16.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb3e8e09640 1 -- 172.21.15.170:0/4039176564 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb4ec0bdc10 con 0x55b411399850 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb3e8e09640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3e40342b0 0x7fb3e4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb3e8e09640 1 -- 172.21.15.170:0/4039176564 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb4ec0e4830 con 0x55b411399850 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3e40342b0 0x7fb3e4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b41142a510 0x55b41151eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41151f030 con 0x55b41142a510 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fcdc5640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b41142a510 0x55b41151eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3e40342b0 0x7fb3e4036770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40012f0 tx=0x7fb4e4006700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fcdc5640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b41142a510 0x55b41151eaf0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/4039176564 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b41142a510 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41151f390 con 0x55b41142a510 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/4039176564 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b41142a510 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b4115210b0 0x55b4115234a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115239e0 con 0x55b4115210b0 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.151+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b4115210b0 0x55b4115234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b4115210b0 0x55b4115234a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/4039176564 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fb4ec0cd070 con 0x55b4115210b0 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41151fe60 con 0x55b41142a510 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/4039176564 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b41142a510 2024-03-28T16:49:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b411524d50 0x55b411527140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411527710 con 0x55b411524d50 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b411524d50 0x55b411527140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b411524d50 0x55b411527140 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/4039176564 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b411524d50 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4feaa9980 1 Processor -- start 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4feaa9980 1 -- start start 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4feaa9980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411535640 0x55b411535a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4feaa9980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4112c9e30 con 0x55b411535640 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411535640 0x55b411535a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411535640 0x55b411535a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55844/0 (socket says 172.21.15.170:55844) 2024-03-28T16:49:16.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/1668661974 learned_addr learned my addr 172.21.15.170:0/1668661974 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.155+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/1668661974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b411535fa0 con 0x55b411535640 2024-03-28T16:49:16.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411535640 0x55b411535a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0e2ec0 tx=0x7fb4ec0aed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3e1ffb640 1 -- 172.21.15.170:0/1668661974 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0abab0 con 0x55b411535640 2024-03-28T16:49:16.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b4115361f0 con 0x55b411535640 2024-03-28T16:49:16.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3e1ffb640 1 -- 172.21.15.170:0/1668661974 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0c6480 con 0x55b411535640 2024-03-28T16:49:16.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3e1ffb640 1 -- 172.21.15.170:0/1668661974 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0c6720 con 0x55b411535640 2024-03-28T16:49:16.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b411536d40 con 0x55b411535640 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3e1ffb640 1 -- 172.21.15.170:0/1668661974 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb4ec0c6880 con 0x55b411535640 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3e1ffb640 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3d40342b0 0x7fb3d4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3e1ffb640 1 -- 172.21.15.170:0/1668661974 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb4ec0e60b0 con 0x55b411535640 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3d40342b0 0x7fb3d4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4feaa9980 1 Processor -- start 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4feaa9980 1 -- start start 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4feaa9980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411542970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4feaa9980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4112bd790 con 0x55b411542590 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3d40342b0 0x7fb3d4036770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006300 tx=0x7fb4e400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411542970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411542970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55848/0 (socket says 172.21.15.170:55848) 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/498332594 learned_addr learned my addr 172.21.15.170:0/498332594 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/498332594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b411542f40 con 0x55b411542590 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/498332594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411542970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0abc70 tx=0x7fb4ec0ed430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c71c3edd02ca7c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3db7fe640 1 -- 172.21.15.170:0/498332594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0bca50 con 0x55b411542590 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3db7fe640 1 -- 172.21.15.170:0/498332594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0fd040 con 0x55b411542590 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.159+0000 7fb3db7fe640 1 -- 172.21.15.170:0/498332594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0c7dd0 con 0x55b411542590 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/498332594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 msgr2=0x55b411542970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/498332594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411542970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0abc70 tx=0x7fb4ec0ed430 comp rx=0 tx=0).stop 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/498332594 shutdown_connections 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/498332594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411542970 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/498332594 >> 172.21.15.170:0/498332594 conn(0x55b41153e920 msgr2=0x55b41153c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/498332594 shutdown_connections 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/498332594 wait complete. 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 Processor -- start 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- start start 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411549490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4115499d0 con 0x55b411542590 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411549490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4fc5c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411549490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55860/0 (socket says 172.21.15.170:55860) 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 learned_addr learned my addr 172.21.15.170:0/916519165 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b411536d40 con 0x55b411542590 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411549490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0d0530 tx=0x7fb4ec0e47a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb3d9ffb640 1 -- 172.21.15.170:0/916519165 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0c53e0 con 0x55b411542590 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b411536d40 con 0x55b411542590 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb3d9ffb640 1 -- 172.21.15.170:0/916519165 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0c5540 con 0x55b411542590 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb3d9ffb640 1 -- 172.21.15.170:0/916519165 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec0a73c0 con 0x55b411542590 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b411536d40 con 0x55b411542590 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb3d9ffb640 1 -- 172.21.15.170:0/916519165 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb4ec0c7590 con 0x55b411542590 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb3d9ffb640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3cc034260 0x7fb3cc036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb3d9ffb640 1 -- 172.21.15.170:0/916519165 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb4ec0fd660 con 0x55b411542590 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3cc034260 0x7fb3cc036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b411554690 0x55b411556b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115572a0 con 0x55b411554690 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.163+0000 7fb4fcdc5640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b411554690 0x55b411556b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3cc034260 0x7fb3cc036720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55b4115201c0 tx=0x7fb4e4010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fcdc5640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b411554690 0x55b411556b50 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 realm 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557670 con 0x55b411554690 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:16.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411558220 con 0x55b411554690 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b411558bd0 0x55b41155afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41155b610 con 0x55b411558bd0 2024-03-28T16:49:16.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b411558bd0 0x55b41155afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fc5c4640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b411558bd0 0x55b41155afc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 20 searching for the correct realm 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b41155d7b0 0x55b41155fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115602f0 con 0x55b41155d7b0 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b41155d7b0 0x55b41155fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b41155d7b0 0x55b41155fc70 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fbdc3640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb4e4024070 0x7fb4e4031970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115602f0 con 0x7fb4e4024070 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.167+0000 7fb4fcdc5640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb4e4024070 0x7fb4e4031970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb4e4024070 0x7fb4e4031970 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x7fb4e4024070 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.171+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x7fb4e4024070 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.175+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x7fb4e4024070 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x7fb4e4024070 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b41155d7b0 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411558bd0 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x55b411554690 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411557ec0 con 0x7fb4e4024070 2024-03-28T16:49:16.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.179+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41155bbf0 con 0x55b411554690 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41155bbf0 con 0x55b411554690 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41155bbc0 con 0x7fb4e4024070 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 zone default found 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 4 Realm: () 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:16.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 zonegroup default 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41155c490 con 0x55b411558bd0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41155c890 con 0x55b41155d7b0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115622e0 con 0x55b41155d7b0 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 started sync module instance, tier type = 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115619f0 con 0x7fb4e4024070 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94231873328464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411561da0 con 0x7fb4e4024070 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563040 con 0x55b411554690 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.183+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94231873328464] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.187+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.187+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94231873332992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563020 con 0x55b411554690 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.187+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563ab0 con 0x55b41155d7b0 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.187+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94231873332992] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.187+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.187+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94231873336688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563e60 con 0x55b41155d7b0 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.187+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115647d0 con 0x7fb4e4024070 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94231873340480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411564ec0 con 0x7fb4e4024070 2024-03-28T16:49:16.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411565990 con 0x55b411554690 2024-03-28T16:49:16.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94231873336688] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94231873340480] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94231873344624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411565950 con 0x55b411554690 2024-03-28T16:49:16.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411564a90 con 0x55b411558bd0 2024-03-28T16:49:16.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.191+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94231873344624] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94231873334048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115661e0 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115669e0 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94231873334048] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94231873346880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115671e0 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411567b50 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94231873353872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115682b0 con 0x55b411558bd0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 20 add_watcher() i=0 2024-03-28T16:49:16.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 20 add_watcher() i=1 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 20 add_watcher() i=2 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 20 add_watcher() i=3 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 20 add_watcher() i=4 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.195+0000 7fb4feaa9980 20 add_watcher() i=5 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94231873346880] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 20 add_watcher() i=6 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94231873353872] v29'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 20 add_watcher() i=7 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411566dc0 con 0x55b411554690 2024-03-28T16:49:16.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94231873355088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411566dc0 con 0x55b411554690 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94231873355088] v29'78 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb3b27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 5 note: GC not initialized 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb3b17e2640 20 reqs_thread_entry: start 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.199+0000 7fb4feaa9980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 20 Filter name: none 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb322ffd640 20 reqs_thread_entry: start 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41158d220 con 0x55b411554690 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 20 rados_obj.operate() r=0 bl.length=513 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41158dbc0 con 0x7fb4e4024070 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~621 in=621b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b4115934f0 con 0x55b411554690 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~621] v29'8 uv8 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94231873531824 in=809b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41158ed80 con 0x55b411554690 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94231873344624 notify 124554051591 ret 0) v3 ==== 839+0+0 (crc 0 0 0) 0x7fb4f400a930 con 0x55b411554690 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb3db7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051591 cookie 94231873344624 notifier 4382 bl.length()=797 2024-03-28T16:49:16.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb3db7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb3db7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(65 notify.4 [notify cookie 94231873531824 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb3db7fe640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fb3c0003ea0 con 0x55b411554690 2024-03-28T16:49:16.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94231873531824 notify 124554051591 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb4f400aac0 con 0x55b411554690 2024-03-28T16:49:16.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.203+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411592fb0 con 0x7fb4e4024070 2024-03-28T16:49:16.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4feaa9980 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-03-28T16:49:16.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4feaa9980 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-03-28T16:49:16.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4feaa9980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-03-28T16:49:16.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94231873531824 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41158e8e0 con 0x7fb4e4024070 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== watch-notify(notify (1) cookie 94231873328464 notify 124554051587 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x7fb4f400aac0 con 0x7fb4e4024070 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb3db7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051587 cookie 94231873328464 notifier 4382 bl.length()=166 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb3db7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb3db7fe640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(68 notify.0 [notify cookie 94231873531824 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb3db7fe640 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fb3c0003ea0 con 0x7fb4e4024070 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== watch-notify(notify_complete (2) cookie 94231873531824 notify 124554051587 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb4f400adf0 con 0x7fb4e4024070 2024-03-28T16:49:16.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.214 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb3b27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94231873355088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b41158e950 con 0x55b411554690 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94231873355088] v29'79 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x55b411554690 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.207+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94231873328464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411567d10 con 0x7fb4e4024070 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.211+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94231873328464] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f40083f0 con 0x7fb4e4024070 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.211+0000 7fb4feaa9980 20 remove_watcher() i=0 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.211+0000 7fb4feaa9980 2 removed watcher, disabling cache 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.211+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94231873332992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411561550 con 0x55b411554690 2024-03-28T16:49:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.211+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94231873332992] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f404f040 con 0x55b411554690 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.211+0000 7fb4feaa9980 20 remove_watcher() i=1 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.211+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94231873336688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411561550 con 0x55b41155d7b0 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4fbdc3640 1 -- 172.21.15.170:0/916519165 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94231873336688] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4e40044f0 con 0x55b41155d7b0 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4feaa9980 20 remove_watcher() i=2 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94231873340480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563570 con 0x7fb4e4024070 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94231873340480] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f404f040 con 0x7fb4e4024070 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4feaa9980 20 remove_watcher() i=3 2024-03-28T16:49:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94231873344624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563570 con 0x55b411554690 2024-03-28T16:49:16.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94231873344624] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4f404f040 con 0x55b411554690 2024-03-28T16:49:16.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4feaa9980 20 remove_watcher() i=4 2024-03-28T16:49:16.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.215+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94231873334048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563570 con 0x55b411558bd0 2024-03-28T16:49:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.219+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94231873334048] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.219+0000 7fb4feaa9980 20 remove_watcher() i=5 2024-03-28T16:49:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.219+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94231873346880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563570 con 0x55b411558bd0 2024-03-28T16:49:16.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.219+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94231873346880] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.219+0000 7fb4feaa9980 20 remove_watcher() i=6 2024-03-28T16:49:16.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.219+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94231873353872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55b411563570 con 0x55b411558bd0 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4fc5c4640 1 -- 172.21.15.170:0/916519165 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94231873353872] v29'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb4ec112070 con 0x55b411558bd0 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 20 remove_watcher() i=7 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b411558bd0 msgr2=0x55b41155afc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b411558bd0 0x55b41155afc0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b411554690 msgr2=0x55b411556b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b411554690 0x55b411556b50 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb4e4024070 msgr2=0x7fb4e4031970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb4e4024070 0x7fb4e4031970 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b41155d7b0 msgr2=0x55b41155fc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b41155d7b0 0x55b41155fc70 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3cc034260 msgr2=0x7fb3cc036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3cc034260 0x7fb3cc036720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55b4115201c0 tx=0x7fb4e4010000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 msgr2=0x55b411549490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/916519165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411542590 0x55b411549490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0d0530 tx=0x7fb4ec0e47a0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 reap_dead start 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/916519165 reap_dead start 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 shutdown_connections 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 >> 172.21.15.170:0/916519165 conn(0x55b41153e920 msgr2=0x55b41153bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 shutdown_connections 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/916519165 wait complete. 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3d40342b0 msgr2=0x7fb3d4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3d40342b0 0x7fb3d4036770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006300 tx=0x7fb4e400f000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411535640 msgr2=0x55b411535a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411535640 0x55b411535a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0e2ec0 tx=0x7fb4ec0aed30 comp rx=0 tx=0).stop 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 shutdown_connections 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3d40342b0 0x7fb3d4036770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/1668661974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411535640 0x55b411535a60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 >> 172.21.15.170:0/1668661974 conn(0x55b41152a450 msgr2=0x55b41152af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 shutdown_connections 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/1668661974 wait complete. 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b4115210b0 msgr2=0x55b4115234a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b4115210b0 0x55b4115234a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b41142a510 msgr2=0x55b41151eaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b41142a510 0x55b41151eaf0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b411524d50 msgr2=0x55b411527140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b411524d50 0x55b411527140 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3e40342b0 msgr2=0x7fb3e4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb3e40342b0 0x7fb3e4036770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40012f0 tx=0x7fb4e4006700 comp rx=0 tx=0).stop 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 msgr2=0x55b411517e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 --2- 172.21.15.170:0/4039176564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b411399850 0x55b411517e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec018f70 tx=0x7fb4ec0180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4fcdc5640 1 -- 172.21.15.170:0/4039176564 reap_dead start 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 shutdown_connections 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 >> 172.21.15.170:0/4039176564 conn(0x55b41141f6b0 msgr2=0x55b41150b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 shutdown_connections 2024-03-28T16:49:16.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.223+0000 7fb4feaa9980 1 -- 172.21.15.170:0/4039176564 wait complete. 2024-03-28T16:49:16.234 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:16.235 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:49:16.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:49:16.235 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:49:16.309 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:16.309 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:16.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d9785980 1 Processor -- start 2024-03-28T16:49:16.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d9785980 1 -- start start 2024-03-28T16:49:16.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d9785980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e1df5f5070 0x55e1df683bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d9785980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e1df5cd520 con 0x55e1df684130 2024-03-28T16:49:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d9785980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e1df5940f0 con 0x55e1df5f5070 2024-03-28T16:49:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d72a0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e1df5f5070 0x55e1df683bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d72a0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e1df5f5070 0x55e1df683bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55868/0 (socket says 172.21.15.170:55868) 2024-03-28T16:49:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d6a9f640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55e1df684130 0x55e1df686520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60974/0 (socket says 172.21.15.170:60974) 2024-03-28T16:49:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d72a0640 1 -- 172.21.15.170:0/177455577 learned_addr learned my addr 172.21.15.170:0/177455577 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d72a0640 1 -- 172.21.15.170:0/177455577 >> v1:172.21.15.119:6789/0 conn(0x55e1df684130 legacy=0x55e1df686520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d72a0640 1 -- 172.21.15.170:0/177455577 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1df688ce0 con 0x55e1df5f5070 2024-03-28T16:49:16.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.311+0000 7f58d72a0640 1 --2- 172.21.15.170:0/177455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df683bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f58c800a5c0 tx=0x7f58c801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bfe0a5776f6138 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d629e640 1 -- 172.21.15.170:0/177455577 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80aa470 con 0x55e1df5f5070 2024-03-28T16:49:16.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d629e640 1 -- 172.21.15.170:0/177455577 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80aa5d0 con 0x55e1df5f5070 2024-03-28T16:49:16.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d629e640 1 -- 172.21.15.170:0/177455577 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80aa890 con 0x55e1df5f5070 2024-03-28T16:49:16.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d9785980 1 -- 172.21.15.170:0/177455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 msgr2=0x55e1df683bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d9785980 1 --2- 172.21.15.170:0/177455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df683bf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f58c800a5c0 tx=0x7f58c801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:16.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d9785980 1 -- 172.21.15.170:0/177455577 shutdown_connections 2024-03-28T16:49:16.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d9785980 1 --2- 172.21.15.170:0/177455577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df683bf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d9785980 1 -- 172.21.15.170:0/177455577 >> 172.21.15.170:0/177455577 conn(0x55e1df67b6d0 msgr2=0x55e1df67daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d9785980 1 -- 172.21.15.170:0/177455577 shutdown_connections 2024-03-28T16:49:16.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.315+0000 7f58d9785980 1 -- 172.21.15.170:0/177455577 wait complete. 2024-03-28T16:49:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d9785980 1 Processor -- start 2024-03-28T16:49:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d9785980 1 -- start start 2024-03-28T16:49:16.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d9785980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df76ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d9785980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df591620 con 0x55e1df5f5070 2024-03-28T16:49:16.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df76ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df76ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55876/0 (socket says 172.21.15.170:55876) 2024-03-28T16:49:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d72a0640 1 -- 172.21.15.170:0/698080473 learned_addr learned my addr 172.21.15.170:0/698080473 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d72a0640 1 -- 172.21.15.170:0/698080473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8005df0 con 0x55e1df5f5070 2024-03-28T16:49:16.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f58d72a0640 1 --2- 172.21.15.170:0/698080473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df76ae80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58c8018f70 tx=0x7f58c8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2631938d68d6b9e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f57c560a640 1 -- 172.21.15.170:0/698080473 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80aa9c0 con 0x55e1df5f5070 2024-03-28T16:49:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f57c560a640 1 -- 172.21.15.170:0/698080473 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80aab20 con 0x55e1df5f5070 2024-03-28T16:49:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.327+0000 7f57c560a640 1 -- 172.21.15.170:0/698080473 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80aadc0 con 0x55e1df5f5070 2024-03-28T16:49:16.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- 172.21.15.170:0/698080473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 msgr2=0x55e1df76ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 --2- 172.21.15.170:0/698080473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df76ae80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58c8018f70 tx=0x7f58c8003030 comp rx=0 tx=0).stop 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- 172.21.15.170:0/698080473 shutdown_connections 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 --2- 172.21.15.170:0/698080473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df76ae80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- 172.21.15.170:0/698080473 >> 172.21.15.170:0/698080473 conn(0x55e1df67b6d0 msgr2=0x55e1df766a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- 172.21.15.170:0/698080473 shutdown_connections 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- 172.21.15.170:0/698080473 wait complete. 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 Processor -- start 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- start start 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df771bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df58ce00 con 0x55e1df5f5070 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df771bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df771bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55888/0 (socket says 172.21.15.170:55888) 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d72a0640 1 -- 172.21.15.170:0/2069360070 learned_addr learned my addr 172.21.15.170:0/2069360070 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d72a0640 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1df688ce0 con 0x55e1df5f5070 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d72a0640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df771bd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58c8018f70 tx=0x7f58c80180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f57c3e07640 1 -- 172.21.15.170:0/2069360070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80cb070 con 0x55e1df5f5070 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1df772110 con 0x55e1df5f5070 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f57c3e07640 1 -- 172.21.15.170:0/2069360070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80bd810 con 0x55e1df5f5070 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1df7724e0 con 0x55e1df5f5070 2024-03-28T16:49:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f57c3e07640 1 -- 172.21.15.170:0/2069360070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80bdab0 con 0x55e1df5f5070 2024-03-28T16:49:16.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.331+0000 7f57c3e07640 1 -- 172.21.15.170:0/2069360070 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f58c80bdc10 con 0x55e1df5f5070 2024-03-28T16:49:16.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f57c3e07640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57bc0342b0 0x7f57bc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f57c3e07640 1 -- 172.21.15.170:0/2069360070 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f58c80e4880 con 0x55e1df5f5070 2024-03-28T16:49:16.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57bc0342b0 0x7f57bc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df684130 0x55e1df778800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df778d40 con 0x55e1df684130 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d7aa1640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df684130 0x55e1df778800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57bc0342b0 0x7f57bc036770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f58cc000f70 tx=0x7f58cc001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d7aa1640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df684130 0x55e1df778800 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d7aa1640 1 -- 172.21.15.170:0/2069360070 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df684130 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7790a0 con 0x55e1df684130 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d7aa1640 1 -- 172.21.15.170:0/2069360070 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df684130 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df77aa30 0x55e1df77ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df77d5d0 con 0x55e1df77aa30 2024-03-28T16:49:16.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d72a0640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df77aa30 0x55e1df77ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d72a0640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df77aa30 0x55e1df77ce40 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d72a0640 1 -- 172.21.15.170:0/2069360070 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f58c80cd070 con 0x55e1df77aa30 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7798a0 con 0x55e1df684130 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d7aa1640 1 -- 172.21.15.170:0/2069360070 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df684130 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1df77e9f0 0x55e1df780de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7814a0 con 0x55e1df77e9f0 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1df77e9f0 0x55e1df780de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.335+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1df77e9f0 0x55e1df780de0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d6a9f640 1 -- 172.21.15.170:0/2069360070 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f58cc00f6a0 con 0x55e1df77e9f0 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d9785980 1 Processor -- start 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d9785980 1 -- start start 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d9785980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df78f3d0 0x55e1df78f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d9785980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df525e30 con 0x55e1df78f3d0 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df78f3d0 0x55e1df78f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df78f3d0 0x55e1df78f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55902/0 (socket says 172.21.15.170:55902) 2024-03-28T16:49:16.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d72a0640 1 -- 172.21.15.170:0/302041982 learned_addr learned my addr 172.21.15.170:0/302041982 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d72a0640 1 -- 172.21.15.170:0/302041982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1df78fd30 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d72a0640 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df78f3d0 0x55e1df78f7f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f58c80e4010 tx=0x7f58c80aec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f57c1602640 1 -- 172.21.15.170:0/302041982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80cfbd0 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1df78ff80 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f57c1602640 1 -- 172.21.15.170:0/302041982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80cfd30 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1df790ad0 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f57c1602640 1 -- 172.21.15.170:0/302041982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80c6480 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f57c1602640 1 -- 172.21.15.170:0/302041982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f58c80c65e0 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f57c1602640 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57b00342b0 0x7f57b0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f57c1602640 1 -- 172.21.15.170:0/302041982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f58c80f8aa0 con 0x55e1df78f3d0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.339+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57b00342b0 0x7f57b0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 Processor -- start 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- start start 2024-03-28T16:49:16.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57b00342b0 0x7f57b0036770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55e1df77d960 tx=0x7f58cc00f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df79c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df519790 con 0x55e1df79c240 2024-03-28T16:49:16.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df79c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df79c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55912/0 (socket says 172.21.15.170:55912) 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 -- 172.21.15.170:0/63916626 learned_addr learned my addr 172.21.15.170:0/63916626 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 -- 172.21.15.170:0/63916626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1df79cc10 con 0x55e1df79c240 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 --2- 172.21.15.170:0/63916626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df79c640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58c80b3f90 tx=0x7f58c8006ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b392699fd81c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f57baffd640 1 -- 172.21.15.170:0/63916626 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c8100070 con 0x55e1df79c240 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f57baffd640 1 -- 172.21.15.170:0/63916626 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80c5900 con 0x55e1df79c240 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f57baffd640 1 -- 172.21.15.170:0/63916626 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80c6890 con 0x55e1df79c240 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- 172.21.15.170:0/63916626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 msgr2=0x55e1df79c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 --2- 172.21.15.170:0/63916626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df79c640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f58c80b3f90 tx=0x7f58c8006ae0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- 172.21.15.170:0/63916626 shutdown_connections 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 --2- 172.21.15.170:0/63916626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df79c640 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- 172.21.15.170:0/63916626 >> 172.21.15.170:0/63916626 conn(0x55e1df798580 msgr2=0x55e1df7965e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- 172.21.15.170:0/63916626 shutdown_connections 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- 172.21.15.170:0/63916626 wait complete. 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 Processor -- start 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- start start 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df6814d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d9785980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1df681a10 con 0x55e1df79c240 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df6814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df6814d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55918/0 (socket says 172.21.15.170:55918) 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 learned_addr learned my addr 172.21.15.170:0/4256101155 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c80f2030 con 0x55e1df79c240 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.343+0000 7f58d72a0640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df6814d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58c80c7bb0 tx=0x7f58c80c5f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f57b97fa640 1 -- 172.21.15.170:0/4256101155 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80c5510 con 0x55e1df79c240 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1df67fbf0 con 0x55e1df79c240 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1df67fbf0 con 0x55e1df79c240 2024-03-28T16:49:16.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f57b97fa640 1 -- 172.21.15.170:0/4256101155 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8006390 con 0x55e1df79c240 2024-03-28T16:49:16.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f57b97fa640 1 -- 172.21.15.170:0/4256101155 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58c80c7c20 con 0x55e1df79c240 2024-03-28T16:49:16.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f57b97fa640 1 -- 172.21.15.170:0/4256101155 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f58c80f8ce0 con 0x55e1df79c240 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f57b97fa640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57a00342b0 0x7f57a0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f57b97fa640 1 -- 172.21.15.170:0/4256101155 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f58c80cc220 con 0x55e1df79c240 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57a00342b0 0x7f57a0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df7b04d0 0x55e1df7b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b30c0 con 0x55e1df7b04d0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d7aa1640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df7b04d0 0x55e1df7b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57a00342b0 0x7f57a0036770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c010 tx=0x7f58cc010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d7aa1640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df7b04d0 0x55e1df7b2970 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 realm 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3490 con 0x55e1df7b04d0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b4040 con 0x55e1df7b04d0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df7b49f0 0x55e1df7b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b7360 con 0x55e1df7b49f0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.347+0000 7f58d72a0640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df7b49f0 0x55e1df7b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d72a0640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df7b49f0 0x55e1df7b6de0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d9785980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d9785980 20 searching for the correct realm 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1df7b91f0 0x55e1df7bb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bbd50 con 0x55e1df7b91f0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1df7b91f0 0x55e1df7bb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1df7b91f0 0x55e1df7bb680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d6a9f640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f58cc01fc60 0x7f58cc03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bbd50 con 0x7f58cc01fc60 2024-03-28T16:49:16.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f58cc01fc60 0x7f58cc03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f58cc01fc60 0x7f58cc03a3c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.351+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x7f58cc01fc60 2024-03-28T16:49:16.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x7f58cc01fc60 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.355+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x7f58cc01fc60 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.359+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x7f58cc01fc60 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b91f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b49f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x55e1df7b04d0 2024-03-28T16:49:16.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b3ce0 con 0x7f58cc01fc60 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b7b60 con 0x55e1df7b04d0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b7b60 con 0x55e1df7b04d0 2024-03-28T16:49:16.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b7b60 con 0x7f58cc01fc60 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 zone default found 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 4 Realm: () 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 zonegroup default 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b83b0 con 0x55e1df7b49f0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7b87c0 con 0x55e1df7b91f0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bdc70 con 0x55e1df7b91f0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 started sync module instance, tier type = 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:16.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.363+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bd450 con 0x7f58cc01fc60 2024-03-28T16:49:16.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.367+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.367+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94428605435824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bd800 con 0x7f58cc01fc60 2024-03-28T16:49:16.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.367+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7be9d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.367+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94428605435824] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.367+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.367+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94428605440144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7be990 con 0x55e1df7b04d0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.367+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bf420 con 0x55e1df7b91f0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94428605443808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bfc20 con 0x55e1df7b91f0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c0650 con 0x7f58cc01fc60 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94428605440144] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94428605448528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c0a30 con 0x7f58cc01fc60 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c13a0 con 0x55e1df7b04d0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94428605443808] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94428605448528] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94428605441200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c0f00 con 0x55e1df7b04d0 2024-03-28T16:49:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.371+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c1700 con 0x55e1df7b49f0 2024-03-28T16:49:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.375+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94428605441200] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.375+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.375+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94428605452032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c1320 con 0x55e1df7b49f0 2024-03-28T16:49:16.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.375+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c23f0 con 0x55e1df7b49f0 2024-03-28T16:49:16.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.375+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94428605452032] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94428605455296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c2620 con 0x55e1df7b49f0 2024-03-28T16:49:16.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c30f0 con 0x55e1df7b49f0 2024-03-28T16:49:16.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94428605455296] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94428605459408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c3590 con 0x55e1df7b49f0 2024-03-28T16:49:16.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=0 2024-03-28T16:49:16.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=1 2024-03-28T16:49:16.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=2 2024-03-28T16:49:16.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=3 2024-03-28T16:49:16.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=4 2024-03-28T16:49:16.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=5 2024-03-28T16:49:16.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=6 2024-03-28T16:49:16.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94428605459408] v29'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 20 add_watcher() i=7 2024-03-28T16:49:16.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:16.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.379+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c21b0 con 0x55e1df7b04d0 2024-03-28T16:49:16.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94428605460656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c3e10 con 0x55e1df7b04d0 2024-03-28T16:49:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94428605460656] v29'80 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f578dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 5 note: GC not initialized 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f578cfe1640 20 reqs_thread_entry: start 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 20 Filter name: none 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f57067fc640 20 reqs_thread_entry: start 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7e80c0 con 0x55e1df7b04d0 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 20 rados_obj.operate() r=0 bl.length=621 2024-03-28T16:49:16.384 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f578dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94428605460656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7e7250 con 0x55e1df7b04d0 2024-03-28T16:49:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94428605460656] v29'81 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.383+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94428605435824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7c30b0 con 0x7f58cc01fc60 2024-03-28T16:49:16.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.387+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94428605435824] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.387+0000 7f58d9785980 20 remove_watcher() i=0 2024-03-28T16:49:16.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.387+0000 7f58d9785980 2 removed watcher, disabling cache 2024-03-28T16:49:16.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.387+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94428605440144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bcfb0 con 0x55e1df7b04d0 2024-03-28T16:49:16.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.387+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94428605440144] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.387+0000 7f58d9785980 20 remove_watcher() i=1 2024-03-28T16:49:16.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.387+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94428605443808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7bcfb0 con 0x55e1df7b91f0 2024-03-28T16:49:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d6a9f640 1 -- 172.21.15.170:0/4256101155 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94428605443808] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58cc033040 con 0x55e1df7b91f0 2024-03-28T16:49:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d9785980 20 remove_watcher() i=2 2024-03-28T16:49:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94428605448528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7be090 con 0x7f58cc01fc60 2024-03-28T16:49:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94428605448528] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x7f58cc01fc60 2024-03-28T16:49:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d9785980 20 remove_watcher() i=3 2024-03-28T16:49:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94428605441200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7be090 con 0x55e1df7b04d0 2024-03-28T16:49:16.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94428605441200] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58d000a4d0 con 0x55e1df7b04d0 2024-03-28T16:49:16.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d9785980 20 remove_watcher() i=4 2024-03-28T16:49:16.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.391+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94428605452032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7be530 con 0x55e1df7b49f0 2024-03-28T16:49:16.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.395+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94428605452032] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.395+0000 7f58d9785980 20 remove_watcher() i=5 2024-03-28T16:49:16.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.395+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94428605455296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7be530 con 0x55e1df7b49f0 2024-03-28T16:49:16.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.395+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94428605455296] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.395+0000 7f58d9785980 20 remove_watcher() i=6 2024-03-28T16:49:16.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.395+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94428605459408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e1df7be530 con 0x55e1df7b49f0 2024-03-28T16:49:16.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d72a0640 1 -- 172.21.15.170:0/4256101155 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94428605459408] v29'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f58c8114070 con 0x55e1df7b49f0 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 20 remove_watcher() i=7 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df7b49f0 msgr2=0x55e1df7b6de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df7b49f0 0x55e1df7b6de0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df7b04d0 msgr2=0x55e1df7b2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df7b04d0 0x55e1df7b2970 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f58cc01fc60 msgr2=0x7f58cc03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f58cc01fc60 0x7f58cc03a3c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1df7b91f0 msgr2=0x55e1df7bb680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e1df7b91f0 0x55e1df7bb680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57a00342b0 msgr2=0x7f57a0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57a00342b0 0x7f57a0036770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c010 tx=0x7f58cc010000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 msgr2=0x55e1df6814d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d7aa1640 1 -- 172.21.15.170:0/4256101155 reap_dead start 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df6814d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58c80c7bb0 tx=0x7f58c80c5f40 comp rx=0 tx=0).stop 2024-03-28T16:49:16.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 shutdown_connections 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/4256101155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df79c240 0x55e1df6814d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 >> 172.21.15.170:0/4256101155 conn(0x55e1df798580 msgr2=0x55e1df795be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 shutdown_connections 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/4256101155 wait complete. 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57b00342b0 msgr2=0x7f57b0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57b00342b0 0x7f57b0036770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55e1df77d960 tx=0x7f58cc00f000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df78f3d0 msgr2=0x55e1df78f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df78f3d0 0x55e1df78f7f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f58c80e4010 tx=0x7f58c80aec70 comp rx=0 tx=0).stop 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 shutdown_connections 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57b00342b0 0x7f57b0036770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/302041982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df78f3d0 0x55e1df78f7f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 >> 172.21.15.170:0/302041982 conn(0x55e1df7841e0 msgr2=0x55e1df784cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 shutdown_connections 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/302041982 wait complete. 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df77aa30 msgr2=0x55e1df77ce40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e1df77aa30 0x55e1df77ce40 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df684130 msgr2=0x55e1df778800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e1df684130 0x55e1df778800 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1df77e9f0 msgr2=0x55e1df780de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e1df77e9f0 0x55e1df780de0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57bc0342b0 msgr2=0x7f57bc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f57bc0342b0 0x7f57bc036770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f58cc000f70 tx=0x7f58cc001380 comp rx=0 tx=0).stop 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 msgr2=0x55e1df771bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 --2- 172.21.15.170:0/2069360070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1df5f5070 0x55e1df771bd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58c8018f70 tx=0x7f58c80180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d7aa1640 1 -- 172.21.15.170:0/2069360070 reap_dead start 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 shutdown_connections 2024-03-28T16:49:16.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 >> 172.21.15.170:0/2069360070 conn(0x55e1df67b6d0 msgr2=0x55e1df765520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 shutdown_connections 2024-03-28T16:49:16.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.399+0000 7f58d9785980 1 -- 172.21.15.170:0/2069360070 wait complete. 2024-03-28T16:49:16.410 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:16.411 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-03-28T16:49:16.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-03-28T16:49:16.411 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-03-28T16:49:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:16.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbd0baa980 1 Processor -- start 2024-03-28T16:49:16.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbd0baa980 1 -- start start 2024-03-28T16:49:16.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbd0baa980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5606aa9ccf90 0x5606aa9cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbd0baa980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5606aa9194c0 con 0x5606aa9cd8b0 2024-03-28T16:49:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbd0baa980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5606aa8e00f0 con 0x5606aa9ccf90 2024-03-28T16:49:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbce6c5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5606aa9ccf90 0x5606aa9cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbce6c5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5606aa9ccf90 0x5606aa9cd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55926/0 (socket says 172.21.15.170:55926) 2024-03-28T16:49:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbce6c5640 1 -- 172.21.15.170:0/1317307473 learned_addr learned my addr 172.21.15.170:0/1317307473 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbcdec4640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5606aa9cd8b0 0x5606aa9ce220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:60982/0 (socket says 172.21.15.170:60982) 2024-03-28T16:49:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbce6c5640 1 -- 172.21.15.170:0/1317307473 >> v1:172.21.15.119:6789/0 conn(0x5606aa9cd8b0 legacy=0x5606aa9ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.483+0000 7fdbce6c5640 1 -- 172.21.15.170:0/1317307473 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606aa9cea50 con 0x5606aa9ccf90 2024-03-28T16:49:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/1317307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aa9cd370 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800a5c0 tx=0x7fdbb801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7267e7b4bd9c56eb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbcd6c3640 1 -- 172.21.15.170:0/1317307473 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80aa470 con 0x5606aa9ccf90 2024-03-28T16:49:16.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbcd6c3640 1 -- 172.21.15.170:0/1317307473 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80aa5d0 con 0x5606aa9ccf90 2024-03-28T16:49:16.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbcd6c3640 1 -- 172.21.15.170:0/1317307473 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80aa890 con 0x5606aa9ccf90 2024-03-28T16:49:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbd0baa980 1 -- 172.21.15.170:0/1317307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 msgr2=0x5606aa9cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/1317307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aa9cd370 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800a5c0 tx=0x7fdbb801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbd0baa980 1 -- 172.21.15.170:0/1317307473 shutdown_connections 2024-03-28T16:49:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/1317307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aa9cd370 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbd0baa980 1 -- 172.21.15.170:0/1317307473 >> 172.21.15.170:0/1317307473 conn(0x5606aa9c8860 msgr2=0x5606aa9cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbd0baa980 1 -- 172.21.15.170:0/1317307473 shutdown_connections 2024-03-28T16:49:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.487+0000 7fdbd0baa980 1 -- 172.21.15.170:0/1317307473 wait complete. 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.499+0000 7fdbd0baa980 1 Processor -- start 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- start start 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaab73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606aa8dd620 con 0x5606aa9ccf90 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaab73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaab73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55930/0 (socket says 172.21.15.170:55930) 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 -- 172.21.15.170:0/117852597 learned_addr learned my addr 172.21.15.170:0/117852597 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 -- 172.21.15.170:0/117852597 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8005df0 con 0x5606aa9ccf90 2024-03-28T16:49:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/117852597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaab73b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80ac040 tx=0x7fdbb8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240d6212c50e7920 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdabc608640 1 -- 172.21.15.170:0/117852597 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80aa9b0 con 0x5606aa9ccf90 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdabc608640 1 -- 172.21.15.170:0/117852597 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80aab30 con 0x5606aa9ccf90 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdabc608640 1 -- 172.21.15.170:0/117852597 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80aae10 con 0x5606aa9ccf90 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- 172.21.15.170:0/117852597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 msgr2=0x5606aaab73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/117852597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaab73b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80ac040 tx=0x7fdbb8003030 comp rx=0 tx=0).stop 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- 172.21.15.170:0/117852597 shutdown_connections 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/117852597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaab73b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- 172.21.15.170:0/117852597 >> 172.21.15.170:0/117852597 conn(0x5606aa9c8860 msgr2=0x5606aaab2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- 172.21.15.170:0/117852597 shutdown_connections 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- 172.21.15.170:0/117852597 wait complete. 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 Processor -- start 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- start start 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaabe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbd0baa980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606aa8d8e00 con 0x5606aa9ccf90 2024-03-28T16:49:16.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaabe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaabe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55940/0 (socket says 172.21.15.170:55940) 2024-03-28T16:49:16.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 -- 172.21.15.170:0/4282242330 learned_addr learned my addr 172.21.15.170:0/4282242330 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.503+0000 7fdbce6c5640 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606aa9cea50 con 0x5606aa9ccf90 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaabe100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8005940 tx=0x7fdbb80180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdabae05640 1 -- 172.21.15.170:0/4282242330 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80c6690 con 0x5606aa9ccf90 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5606aaabe640 con 0x5606aa9ccf90 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdabae05640 1 -- 172.21.15.170:0/4282242330 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80c67f0 con 0x5606aa9ccf90 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606aaabea70 con 0x5606aa9ccf90 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdabae05640 1 -- 172.21.15.170:0/4282242330 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80c6a90 con 0x5606aa9ccf90 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdabae05640 1 -- 172.21.15.170:0/4282242330 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fdbb80c6bf0 con 0x5606aa9ccf90 2024-03-28T16:49:16.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdabae05640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdab40342b0 0x7fdab4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdabae05640 1 -- 172.21.15.170:0/4282242330 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fdbb80e4280 con 0x5606aa9ccf90 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdab40342b0 0x7fdab4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aa9cd8b0 0x5606aaac4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aaac5270 con 0x5606aa9cd8b0 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbceec6640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aa9cd8b0 0x5606aaac4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdab40342b0 0x7fdab4036770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000f70 tx=0x7fdbc0001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbceec6640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aa9cd8b0 0x5606aaac4d30 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbceec6640 1 -- 172.21.15.170:0/4282242330 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aa9cd8b0 2024-03-28T16:49:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aaac55d0 con 0x5606aa9cd8b0 2024-03-28T16:49:16.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbceec6640 1 -- 172.21.15.170:0/4282242330 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aa9cd8b0 2024-03-28T16:49:16.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aaac72f0 0x5606aaac96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aaac9c60 con 0x5606aaac72f0 2024-03-28T16:49:16.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.507+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aaac72f0 0x5606aaac96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aaac72f0 0x5606aaac96e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbce6c5640 1 -- 172.21.15.170:0/4282242330 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fdbb80b0020 con 0x5606aaac72f0 2024-03-28T16:49:16.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aaac6130 con 0x5606aa9cd8b0 2024-03-28T16:49:16.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbceec6640 1 -- 172.21.15.170:0/4282242330 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aa9cd8b0 2024-03-28T16:49:16.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5606aaacb060 0x5606aaacd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aaacdaf0 con 0x5606aaacb060 2024-03-28T16:49:16.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5606aaacb060 0x5606aaacd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5606aaacb060 0x5606aaacd450 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbcdec4640 1 -- 172.21.15.170:0/4282242330 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fdbc000f6a0 con 0x5606aaacb060 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbd0baa980 1 Processor -- start 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbd0baa980 1 -- start start 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbd0baa980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aaadba20 0x5606aaadbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbd0baa980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606aa871e30 con 0x5606aaadba20 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aaadba20 0x5606aaadbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aaadba20 0x5606aaadbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55956/0 (socket says 172.21.15.170:55956) 2024-03-28T16:49:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbce6c5640 1 -- 172.21.15.170:0/927749937 learned_addr learned my addr 172.21.15.170:0/927749937 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbce6c5640 1 -- 172.21.15.170:0/927749937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606aaadc380 con 0x5606aaadba20 2024-03-28T16:49:16.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.511+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aaadba20 0x5606aaadbe40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80c5e70 tx=0x7fdbb80c55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab3fff640 1 -- 172.21.15.170:0/927749937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80c5870 con 0x5606aaadba20 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5606aaadc5d0 con 0x5606aaadba20 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab3fff640 1 -- 172.21.15.170:0/927749937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80e2400 con 0x5606aaadba20 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606aaadd120 con 0x5606aaadba20 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab3fff640 1 -- 172.21.15.170:0/927749937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80bda70 con 0x5606aaadba20 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab3fff640 1 -- 172.21.15.170:0/927749937 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fdbb80e0040 con 0x5606aaadba20 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab3fff640 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdaa80342b0 0x7fdaa8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab3fff640 1 -- 172.21.15.170:0/927749937 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fdbb80edaa0 con 0x5606aaadba20 2024-03-28T16:49:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdaa80342b0 0x7fdaa8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 Processor -- start 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- start start 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdaa80342b0 0x7fdaa8036770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5606aaac6490 tx=0x7fdbc000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aa943990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606aa865790 con 0x5606aa943570 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aa943990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aa943990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55958/0 (socket says 172.21.15.170:55958) 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbce6c5640 1 -- 172.21.15.170:0/3188653025 learned_addr learned my addr 172.21.15.170:0/3188653025 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbce6c5640 1 -- 172.21.15.170:0/3188653025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606aa941e10 con 0x5606aa943570 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/3188653025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aa943990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80b3a90 tx=0x7fdbb80a87b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769a06fd9a1b7fad server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab1ffb640 1 -- 172.21.15.170:0/3188653025 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80bdbd0 con 0x5606aa943570 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab1ffb640 1 -- 172.21.15.170:0/3188653025 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80bdd30 con 0x5606aa943570 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdab1ffb640 1 -- 172.21.15.170:0/3188653025 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80e4de0 con 0x5606aa943570 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- 172.21.15.170:0/3188653025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 msgr2=0x5606aa943990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/3188653025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aa943990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80b3a90 tx=0x7fdbb80a87b0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- 172.21.15.170:0/3188653025 shutdown_connections 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/3188653025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aa943990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- 172.21.15.170:0/3188653025 >> 172.21.15.170:0/3188653025 conn(0x5606aaae4d20 msgr2=0x5606aaae2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.515+0000 7fdbd0baa980 1 -- 172.21.15.170:0/3188653025 shutdown_connections 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 -- 172.21.15.170:0/3188653025 wait complete. 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 Processor -- start 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 -- start start 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aaaf1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606aaaf1f40 con 0x5606aa943570 2024-03-28T16:49:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aaaf1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbce6c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aaaf1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55962/0 (socket says 172.21.15.170:55962) 2024-03-28T16:49:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 learned_addr learned my addr 172.21.15.170:0/549816970 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606aaadd120 con 0x5606aa943570 2024-03-28T16:49:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aaaf1a00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80eb040 tx=0x7fdbb80eafd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fda9ffff640 1 -- 172.21.15.170:0/549816970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80bdbd0 con 0x5606aa943570 2024-03-28T16:49:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5606aaadd120 con 0x5606aa943570 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fda9ffff640 1 -- 172.21.15.170:0/549816970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80b5e50 con 0x5606aa943570 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606aaaf2320 con 0x5606aa943570 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fda9ffff640 1 -- 172.21.15.170:0/549816970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbb80e4720 con 0x5606aa943570 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fda9ffff640 1 -- 172.21.15.170:0/549816970 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fdbb80e4940 con 0x5606aa943570 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fda9ffff640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fda98034210 0x7fda980366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fda9ffff640 1 -- 172.21.15.170:0/549816970 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fdbb80c8070 con 0x5606aa943570 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fda98034210 0x7fda980366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aaafcb70 0x5606aaaff010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aaaff760 con 0x5606aaafcb70 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.519+0000 7fdbceec6640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aaafcb70 0x5606aaaff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fda98034210 0x7fda980366d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c010 tx=0x7fdbc0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbceec6640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aaafcb70 0x5606aaaff010 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 realm 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aaaffb30 con 0x5606aaafcb70 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab006e0 con 0x5606aaafcb70 2024-03-28T16:49:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aab01090 0x5606aab03480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab03ad0 con 0x5606aab01090 2024-03-28T16:49:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aab01090 0x5606aab03480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbce6c5640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aab01090 0x5606aab03480 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:16.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 20 searching for the correct realm 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606aab05cb0 0x5606aab08170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab087f0 con 0x5606aab05cb0 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606aab05cb0 0x5606aab08170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606aab05cb0 0x5606aab08170 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbcdec4640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdbc001fc60 0x7fdbc003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab087f0 con 0x7fdbc001fc60 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.523+0000 7fdbceec6640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdbc001fc60 0x7fdbc003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdbc001fc60 0x7fdbc003a3c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x7fdbc001fc60 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.527+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x7fdbc001fc60 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.531+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x7fdbc001fc60 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x7fdbc001fc60 2024-03-28T16:49:16.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab01090 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.535+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x7fdbc001fc60 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:16.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab040b0 con 0x5606aaafcb70 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab040b0 con 0x5606aaafcb70 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab040b0 con 0x7fdbc001fc60 2024-03-28T16:49:16.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 zone default found 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 4 Realm: () 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 zonegroup default 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab04900 con 0x5606aab01090 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0a610 con 0x5606aab05cb0 2024-03-28T16:49:16.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0a970 con 0x5606aab05cb0 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 started sync module instance, tier type = 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab09ef0 con 0x7fdbc001fc60 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94586633493072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab09e70 con 0x7fdbc001fc60 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.539+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0b610 con 0x5606aaafcb70 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94586633493072] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94586633497808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab09e70 con 0x5606aaafcb70 2024-03-28T16:49:16.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0bbe0 con 0x5606aab05cb0 2024-03-28T16:49:16.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94586633500416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0c430 con 0x5606aab05cb0 2024-03-28T16:49:16.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0cde0 con 0x7fdbc001fc60 2024-03-28T16:49:16.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.543+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94586633497808] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94586633498864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0d190 con 0x7fdbc001fc60 2024-03-28T16:49:16.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0db00 con 0x5606aaafcb70 2024-03-28T16:49:16.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94586633500416] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94586633498864] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94586633505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0d660 con 0x5606aaafcb70 2024-03-28T16:49:16.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.547+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0de60 con 0x5606aab01090 2024-03-28T16:49:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94586633505024] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94586633508448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0da80 con 0x5606aab01090 2024-03-28T16:49:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0eb00 con 0x5606aab01090 2024-03-28T16:49:16.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94586633508448] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94586633511840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0f300 con 0x5606aab01090 2024-03-28T16:49:16.552 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.551+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0fb80 con 0x5606aab01090 2024-03-28T16:49:16.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94586633515776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab10220 con 0x5606aab01090 2024-03-28T16:49:16.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=0 2024-03-28T16:49:16.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=1 2024-03-28T16:49:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=2 2024-03-28T16:49:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=3 2024-03-28T16:49:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=4 2024-03-28T16:49:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=5 2024-03-28T16:49:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94586633511840] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=6 2024-03-28T16:49:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94586633515776] v29'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 add_watcher() i=7 2024-03-28T16:49:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab13120 con 0x5606aaafcb70 2024-03-28T16:49:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94586633518160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab10590 con 0x5606aaafcb70 2024-03-28T16:49:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94586633518160] v29'82 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fda84fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 5 note: GC not initialized 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fd9ff7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.555+0000 7fdbd0baa980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 20 Filter name: none 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fd9fd7fa640 20 reqs_thread_entry: start 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab34fd0 con 0x5606aaafcb70 2024-03-28T16:49:16.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 20 rados_obj.operate() r=0 bl.length=621 2024-03-28T16:49:16.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:16.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:16.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~539 in=539b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab3b410 con 0x5606aaafcb70 2024-03-28T16:49:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~539] v29'9 uv9 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94586633694928 in=727b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab36d90 con 0x5606aaafcb70 2024-03-28T16:49:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94586633505024 notify 124554051592 ret 0) v3 ==== 757+0+0 (crc 0 0 0) 0x7fdbc804a040 con 0x5606aaafcb70 2024-03-28T16:49:16.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdab1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051592 cookie 94586633505024 notifier 4396 bl.length()=715 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdab1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(64 notify.4 [notify cookie 94586633694928 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdab1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdab1ffb640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fda94003ea0 con 0x5606aaafcb70 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94586633694928 notify 124554051592 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fdbc80438e0 con 0x5606aaafcb70 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94586633694928 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab368f0 con 0x7fdbc001fc60 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== watch-notify(notify (1) cookie 94586633493072 notify 124554051588 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x7fdbc80438e0 con 0x7fdbc001fc60 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdab1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051588 cookie 94586633493072 notifier 4396 bl.length()=159 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdab1ffb640 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fda94004200 con 0x7fdbc001fc60 2024-03-28T16:49:16.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(66 notify.0 [notify cookie 94586633694928 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== watch-notify(notify_complete (2) cookie 94586633694928 notify 124554051588 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fdbc8043c10 con 0x7fdbc001fc60 2024-03-28T16:49:16.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(67 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.559+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab3aed0 con 0x5606aaafcb70 2024-03-28T16:49:16.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(68 foo:foo1 [delete] v29'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.564 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:16.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fda84fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94586633518160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab368c0 con 0x5606aaafcb70 2024-03-28T16:49:16.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94586633518160] v29'83 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94586633493072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0fc50 con 0x7fdbc001fc60 2024-03-28T16:49:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94586633493072] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbd0baa980 20 remove_watcher() i=0 2024-03-28T16:49:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbd0baa980 2 removed watcher, disabling cache 2024-03-28T16:49:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.563+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94586633497808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aaafcb70 2024-03-28T16:49:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94586633497808] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbd0baa980 20 remove_watcher() i=1 2024-03-28T16:49:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94586633500416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab00380 con 0x5606aab05cb0 2024-03-28T16:49:16.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbcdec4640 1 -- 172.21.15.170:0/549816970 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94586633500416] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc0033040 con 0x5606aab05cb0 2024-03-28T16:49:16.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbd0baa980 20 remove_watcher() i=2 2024-03-28T16:49:16.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94586633498864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0acd0 con 0x7fdbc001fc60 2024-03-28T16:49:16.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94586633498864] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x7fdbc001fc60 2024-03-28T16:49:16.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbd0baa980 20 remove_watcher() i=3 2024-03-28T16:49:16.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.567+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94586633505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0b170 con 0x5606aaafcb70 2024-03-28T16:49:16.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.571+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94586633505024] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbc800a4d0 con 0x5606aaafcb70 2024-03-28T16:49:16.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.571+0000 7fdbd0baa980 20 remove_watcher() i=4 2024-03-28T16:49:16.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.571+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94586633508448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0b170 con 0x5606aab01090 2024-03-28T16:49:16.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.571+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94586633508448] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.571+0000 7fdbd0baa980 20 remove_watcher() i=5 2024-03-28T16:49:16.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.571+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94586633511840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0b170 con 0x5606aab01090 2024-03-28T16:49:16.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94586633511840] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 20 remove_watcher() i=6 2024-03-28T16:49:16.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94586633515776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5606aab0b170 con 0x5606aab01090 2024-03-28T16:49:16.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbce6c5640 1 -- 172.21.15.170:0/549816970 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94586633515776] v29'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdbb8106040 con 0x5606aab01090 2024-03-28T16:49:16.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 20 remove_watcher() i=7 2024-03-28T16:49:16.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aab01090 msgr2=0x5606aab03480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aab01090 0x5606aab03480 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aaafcb70 msgr2=0x5606aaaff010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aaafcb70 0x5606aaaff010 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdbc001fc60 msgr2=0x7fdbc003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdbc001fc60 0x7fdbc003a3c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606aab05cb0 msgr2=0x5606aab08170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606aab05cb0 0x5606aab08170 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fda98034210 msgr2=0x7fda980366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fda98034210 0x7fda980366d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c010 tx=0x7fdbc0010000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 msgr2=0x5606aaaf1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbceec6640 1 -- 172.21.15.170:0/549816970 reap_dead start 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aaaf1a00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80eb040 tx=0x7fdbb80eafd0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 shutdown_connections 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/549816970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa943570 0x5606aaaf1a00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 >> 172.21.15.170:0/549816970 conn(0x5606aaae4d20 msgr2=0x5606aaae2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 shutdown_connections 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/549816970 wait complete. 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdaa80342b0 msgr2=0x7fdaa8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdaa80342b0 0x7fdaa8036770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x5606aaac6490 tx=0x7fdbc000f000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aaadba20 msgr2=0x5606aaadbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aaadba20 0x5606aaadbe40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80c5e70 tx=0x7fdbb80c55c0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 shutdown_connections 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdaa80342b0 0x7fdaa8036770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/927749937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aaadba20 0x5606aaadbe40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 >> 172.21.15.170:0/927749937 conn(0x5606aaad0830 msgr2=0x5606aaad1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 shutdown_connections 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/927749937 wait complete. 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aaac72f0 msgr2=0x5606aaac96e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606aaac72f0 0x5606aaac96e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aa9cd8b0 msgr2=0x5606aaac4d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606aa9cd8b0 0x5606aaac4d30 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5606aaacb060 msgr2=0x5606aaacd450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.575+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5606aaacb060 0x5606aaacd450 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdab40342b0 msgr2=0x7fdab4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fdab40342b0 0x7fdab4036770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0000f70 tx=0x7fdbc0001380 comp rx=0 tx=0).stop 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 msgr2=0x5606aaabe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 --2- 172.21.15.170:0/4282242330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606aa9ccf90 0x5606aaabe100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8005940 tx=0x7fdbb80180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbceec6640 1 -- 172.21.15.170:0/4282242330 reap_dead start 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 shutdown_connections 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 >> 172.21.15.170:0/4282242330 conn(0x5606aa9c8860 msgr2=0x5606aaab1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 shutdown_connections 2024-03-28T16:49:16.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.579+0000 7fdbd0baa980 1 -- 172.21.15.170:0/4282242330 wait complete. 2024-03-28T16:49:16.586 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:16.587 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-03-28T16:49:16.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-03-28T16:49:16.587 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-03-28T16:49:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:16.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223fadc980 1 Processor -- start 2024-03-28T16:49:16.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223fadc980 1 -- start start 2024-03-28T16:49:16.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223fadc980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ac93634350 0x55ac93636740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223fadc980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ac9357d520 con 0x55ac9362fe40 2024-03-28T16:49:16.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223fadc980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ac935440f0 con 0x55ac93634350 2024-03-28T16:49:16.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223cdf6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ac93634350 0x55ac93636740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223cdf6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ac93634350 0x55ac93636740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55966/0 (socket says 172.21.15.170:55966) 2024-03-28T16:49:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223cdf6640 1 -- 172.21.15.170:0/2713815794 learned_addr learned my addr 172.21.15.170:0/2713815794 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223cdf6640 1 -- 172.21.15.170:0/2713815794 >> v1:172.21.15.119:6789/0 conn(0x55ac9362fe40 legacy=0x55ac93630220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.667+0000 7f223cdf6640 1 -- 172.21.15.170:0/2713815794 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac93637c90 con 0x55ac93634350 2024-03-28T16:49:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223cdf6640 1 --2- 172.21.15.170:0/2713815794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93634350 0x55ac93636740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f223400a5c0 tx=0x7f223401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3620721e3fee8f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f222ffff640 1 -- 172.21.15.170:0/2713815794 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22340aa470 con 0x55ac93634350 2024-03-28T16:49:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f222ffff640 1 -- 172.21.15.170:0/2713815794 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22340aa5d0 con 0x55ac93634350 2024-03-28T16:49:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f222ffff640 1 -- 172.21.15.170:0/2713815794 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22340aa890 con 0x55ac93634350 2024-03-28T16:49:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223fadc980 1 -- 172.21.15.170:0/2713815794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93634350 msgr2=0x55ac93636740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223fadc980 1 --2- 172.21.15.170:0/2713815794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93634350 0x55ac93636740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f223400a5c0 tx=0x7f223401b340 comp rx=0 tx=0).stop 2024-03-28T16:49:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223fadc980 1 -- 172.21.15.170:0/2713815794 shutdown_connections 2024-03-28T16:49:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223fadc980 1 --2- 172.21.15.170:0/2713815794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93634350 0x55ac93636740 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223fadc980 1 -- 172.21.15.170:0/2713815794 >> 172.21.15.170:0/2713815794 conn(0x55ac9362b710 msgr2=0x55ac9362db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223fadc980 1 -- 172.21.15.170:0/2713815794 shutdown_connections 2024-03-28T16:49:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.671+0000 7f223fadc980 1 -- 172.21.15.170:0/2713815794 wait complete. 2024-03-28T16:49:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223fadc980 1 Processor -- start 2024-03-28T16:49:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223fadc980 1 -- start start 2024-03-28T16:49:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223fadc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac935a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223fadc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac93541620 con 0x55ac9362fe40 2024-03-28T16:49:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac935a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac935a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55970/0 (socket says 172.21.15.170:55970) 2024-03-28T16:49:16.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223d5f7640 1 -- 172.21.15.170:0/2344367914 learned_addr learned my addr 172.21.15.170:0/2344367914 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223d5f7640 1 -- 172.21.15.170:0/2344367914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234005df0 con 0x55ac9362fe40 2024-03-28T16:49:16.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f223d5f7640 1 --2- 172.21.15.170:0/2344367914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac935a5010 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2230001d10 tx=0x7f223000b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb7b56f44f65511 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f212b606640 1 -- 172.21.15.170:0/2344367914 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2230014070 con 0x55ac9362fe40 2024-03-28T16:49:16.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f212b606640 1 -- 172.21.15.170:0/2344367914 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000f560 con 0x55ac9362fe40 2024-03-28T16:49:16.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.683+0000 7f212b606640 1 -- 172.21.15.170:0/2344367914 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223000f840 con 0x55ac9362fe40 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- 172.21.15.170:0/2344367914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 msgr2=0x55ac935a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 --2- 172.21.15.170:0/2344367914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac935a5010 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2230001d10 tx=0x7f223000b5c0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- 172.21.15.170:0/2344367914 shutdown_connections 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 --2- 172.21.15.170:0/2344367914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac935a5010 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- 172.21.15.170:0/2344367914 >> 172.21.15.170:0/2344367914 conn(0x55ac9362b710 msgr2=0x55ac93715970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- 172.21.15.170:0/2344367914 shutdown_connections 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- 172.21.15.170:0/2344367914 wait complete. 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 Processor -- start 2024-03-28T16:49:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- start start 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac93724020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac9353ce00 con 0x55ac9362fe40 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac93724020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac93724020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55980/0 (socket says 172.21.15.170:55980) 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223d5f7640 1 -- 172.21.15.170:0/3629267564 learned_addr learned my addr 172.21.15.170:0/3629267564 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223d5f7640 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac9357d520 con 0x55ac9362fe40 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223d5f7640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac93724020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55ac93719c20 tx=0x7f2230000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f2129e03640 1 -- 172.21.15.170:0/3629267564 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2230014050 con 0x55ac9362fe40 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac93724560 con 0x55ac9362fe40 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f2129e03640 1 -- 172.21.15.170:0/3629267564 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2230001520 con 0x55ac9362fe40 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f2129e03640 1 -- 172.21.15.170:0/3629267564 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223001bad0 con 0x55ac9362fe40 2024-03-28T16:49:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac937249f0 con 0x55ac9362fe40 2024-03-28T16:49:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.687+0000 7f2129e03640 1 -- 172.21.15.170:0/3629267564 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2230022020 con 0x55ac9362fe40 2024-03-28T16:49:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f2129e03640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21240342b0 0x7f2124036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f2129e03640 1 -- 172.21.15.170:0/3629267564 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f2230048be0 con 0x55ac9362fe40 2024-03-28T16:49:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223cdf6640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21240342b0 0x7f2124036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac93728bf0 0x55ac9372afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9372b520 con 0x55ac93728bf0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223ddf8640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac93728bf0 0x55ac9372afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223cdf6640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21240342b0 0x7f2124036770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22340ae000 tx=0x7f22340a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223ddf8640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac93728bf0 0x55ac9372afe0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223ddf8640 1 -- 172.21.15.170:0/3629267564 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac93728bf0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9372b880 con 0x55ac93728bf0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223ddf8640 1 -- 172.21.15.170:0/3629267564 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac93728bf0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac9372d5a0 0x55ac9372f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9372fed0 con 0x55ac9372d5a0 2024-03-28T16:49:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223d5f7640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac9372d5a0 0x55ac9372f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223d5f7640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac9372d5a0 0x55ac9372f990 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223d5f7640 1 -- 172.21.15.170:0/3629267564 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f2230016070 con 0x55ac9372d5a0 2024-03-28T16:49:16.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9372c410 con 0x55ac93728bf0 2024-03-28T16:49:16.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223ddf8640 1 -- 172.21.15.170:0/3629267564 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac93728bf0 2024-03-28T16:49:16.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ac937312f0 0x55ac937336e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93733d80 con 0x55ac937312f0 2024-03-28T16:49:16.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.691+0000 7f223cdf6640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ac937312f0 0x55ac937336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223cdf6640 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ac937312f0 0x55ac937336e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223cdf6640 1 -- 172.21.15.170:0/3629267564 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f22340b0020 con 0x55ac937312f0 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223fadc980 1 Processor -- start 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223fadc980 1 -- start start 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223fadc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93741d00 0x55ac93742140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223fadc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac934d5e30 con 0x55ac93741d00 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93741d00 0x55ac93742140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93741d00 0x55ac93742140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55982/0 (socket says 172.21.15.170:55982) 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223d5f7640 1 -- 172.21.15.170:0/1062084094 learned_addr learned my addr 172.21.15.170:0/1062084094 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223d5f7640 1 -- 172.21.15.170:0/1062084094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac93742680 con 0x55ac93741d00 2024-03-28T16:49:16.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223d5f7640 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93741d00 0x55ac93742140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f22300016b0 tx=0x7f2230024e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f2122ffd640 1 -- 172.21.15.170:0/1062084094 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2230026920 con 0x55ac93741d00 2024-03-28T16:49:16.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f2122ffd640 1 -- 172.21.15.170:0/1062084094 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2230026a80 con 0x55ac93741d00 2024-03-28T16:49:16.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac93743040 con 0x55ac93741d00 2024-03-28T16:49:16.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f2122ffd640 1 -- 172.21.15.170:0/1062084094 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2230026d20 con 0x55ac93741d00 2024-03-28T16:49:16.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac93743040 con 0x55ac93741d00 2024-03-28T16:49:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f2122ffd640 1 -- 172.21.15.170:0/1062084094 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2230040990 con 0x55ac93741d00 2024-03-28T16:49:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f2122ffd640 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21140342b0 0x7f2114036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f2122ffd640 1 -- 172.21.15.170:0/1062084094 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f223002a180 con 0x55ac93741d00 2024-03-28T16:49:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.695+0000 7f223cdf6640 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21140342b0 0x7f2114036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 Processor -- start 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- start start 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223cdf6640 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21140342b0 0x7f2114036770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ac93730260 tx=0x7f2234005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac9374ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac934c9790 con 0x55ac9374eae0 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac9374ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac9374ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55984/0 (socket says 172.21.15.170:55984) 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 -- 172.21.15.170:0/1002049054 learned_addr learned my addr 172.21.15.170:0/1002049054 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 -- 172.21.15.170:0/1002049054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac9374f4d0 con 0x55ac9374eae0 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 --2- 172.21.15.170:0/1002049054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac9374ef00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f223002e3e0 tx=0x7f223002e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79195efefd3edb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f2120ff9640 1 -- 172.21.15.170:0/1002049054 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22300271b0 con 0x55ac9374eae0 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f2120ff9640 1 -- 172.21.15.170:0/1002049054 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223003bcf0 con 0x55ac9374eae0 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f2120ff9640 1 -- 172.21.15.170:0/1002049054 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223003b3d0 con 0x55ac9374eae0 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- 172.21.15.170:0/1002049054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 msgr2=0x55ac9374ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 --2- 172.21.15.170:0/1002049054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac9374ef00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f223002e3e0 tx=0x7f223002e4d0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- 172.21.15.170:0/1002049054 shutdown_connections 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 --2- 172.21.15.170:0/1002049054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac9374ef00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- 172.21.15.170:0/1002049054 >> 172.21.15.170:0/1002049054 conn(0x55ac9374ae20 msgr2=0x55ac93748e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- 172.21.15.170:0/1002049054 shutdown_connections 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- 172.21.15.170:0/1002049054 wait complete. 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 Processor -- start 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- start start 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac937557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223fadc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac93755d20 con 0x55ac9374eae0 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac937557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac937557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55988/0 (socket says 172.21.15.170:55988) 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 learned_addr learned my addr 172.21.15.170:0/4228005355 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.699+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac93742680 con 0x55ac9374eae0 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223d5f7640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac937557e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f223002eb90 tx=0x7f2230049f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f2112ffd640 1 -- 172.21.15.170:0/4228005355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2230065070 con 0x55ac9374eae0 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f2112ffd640 1 -- 172.21.15.170:0/4228005355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223003b8b0 con 0x55ac9374eae0 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac93742680 con 0x55ac9374eae0 2024-03-28T16:49:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f2112ffd640 1 -- 172.21.15.170:0/4228005355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2230025a80 con 0x55ac9374eae0 2024-03-28T16:49:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac93742680 con 0x55ac9374eae0 2024-03-28T16:49:16.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f2112ffd640 1 -- 172.21.15.170:0/4228005355 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2230025be0 con 0x55ac9374eae0 2024-03-28T16:49:16.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f2112ffd640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21080342b0 0x7f2108036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f2112ffd640 1 -- 172.21.15.170:0/4228005355 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f223006cae0 con 0x55ac9374eae0 2024-03-28T16:49:16.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223cdf6640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21080342b0 0x7f2108036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac937608c0 0x55ac93762d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937634d0 con 0x55ac937608c0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223ddf8640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac937608c0 0x55ac93762d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223cdf6640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21080342b0 0x7f2108036770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f223401b380 tx=0x7f22340c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223ddf8640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac937608c0 0x55ac93762d80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 realm 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937638f0 con 0x55ac937608c0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937644a0 con 0x55ac937608c0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac93764e50 0x55ac93767240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.703+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937678e0 con 0x55ac93764e50 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223d5f7640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac93764e50 0x55ac93767240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223d5f7640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac93764e50 0x55ac93767240 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223fadc980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223fadc980 20 searching for the correct realm 2024-03-28T16:49:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ac93769a60 0x55ac9376bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376c5a0 con 0x55ac93769a60 2024-03-28T16:49:16.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223cdf6640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ac93769a60 0x55ac9376bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223cdf6640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ac93769a60 0x55ac9376bf20 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223cdf6640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f22340e0040 0x7f22340e0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376c5a0 con 0x7f22340e0040 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223ddf8640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f22340e0040 0x7f22340e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223ddf8640 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f22340e0040 0x7f22340e0930 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.707+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x7f22340e0040 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x7f22340e0040 2024-03-28T16:49:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.711+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x7f22340e0040 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.715+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x7f22340e0040 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93769a60 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac93764e50 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x55ac937608c0 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93764140 con 0x7f22340e0040 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:16.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93767e80 con 0x55ac937608c0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93767e80 con 0x55ac937608c0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93767e50 con 0x7f22340e0040 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 zone default found 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 4 Realm: () 2024-03-28T16:49:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 zonegroup default 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937686d0 con 0x55ac93764e50 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93768b10 con 0x55ac93769a60 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.719+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376e4e0 con 0x55ac93769a60 2024-03-28T16:49:16.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 20 started sync module instance, tier type = 2024-03-28T16:49:16.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:16.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376dca0 con 0x7f22340e0040 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94199696775168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376e080 con 0x7f22340e0040 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376f240 con 0x55ac937608c0 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94199696775168] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94199696779520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376f200 con 0x55ac937608c0 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.723+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376fcb0 con 0x55ac93769a60 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94199696779520] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94199696783280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93770060 con 0x55ac93769a60 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937709d0 con 0x7f22340e0040 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94199696787008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93771220 con 0x7f22340e0040 2024-03-28T16:49:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93771bd0 con 0x55ac937608c0 2024-03-28T16:49:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94199696787008] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.727+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94199696783280] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.731+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.731+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94199696780576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93770d00 con 0x55ac937608c0 2024-03-28T16:49:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.731+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93772420 con 0x55ac93764e50 2024-03-28T16:49:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.731+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94199696780576] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.731+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.731+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94199696793472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93770d00 con 0x55ac93764e50 2024-03-28T16:49:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.731+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93772c20 con 0x55ac93764e50 2024-03-28T16:49:16.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94199696793472] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94199696791280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93772c20 con 0x55ac93764e50 2024-03-28T16:49:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937735f0 con 0x55ac93764e50 2024-03-28T16:49:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94199696791280] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94199696797936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937737d0 con 0x55ac93764e50 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 20 add_watcher() i=0 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 20 add_watcher() i=1 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 20 add_watcher() i=2 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 20 add_watcher() i=3 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 20 add_watcher() i=4 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 20 add_watcher() i=5 2024-03-28T16:49:16.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.735+0000 7f223fadc980 20 add_watcher() i=6 2024-03-28T16:49:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94199696797936] v29'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 20 add_watcher() i=7 2024-03-28T16:49:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93776b90 con 0x55ac937608c0 2024-03-28T16:49:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94199696799424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93773ff0 con 0x55ac937608c0 2024-03-28T16:49:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94199696799424] v29'84 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f20f3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 5 note: GC not initialized 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f20f2fe5640 20 reqs_thread_entry: start 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 20 Filter name: none 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f20f0fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937989d0 con 0x55ac937608c0 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~539 out=539b,getxattrs out=75b] v0'0 uv9 ondisk = 0) v8 ==== 231+0+662 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 20 rados_obj.operate() r=0 bl.length=539 2024-03-28T16:49:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:16.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:16.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.739+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9379eb80 con 0x55ac937608c0 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v29'10 uv10 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223fadc980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223fadc980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223fadc980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94199696975360 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9379a430 con 0x55ac937608c0 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94199696780576 notify 124554051593 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x7f223804a040 con 0x55ac937608c0 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f2120ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051593 cookie 94199696780576 notifier 4403 bl.length()=689 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f2120ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f2120ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(64 notify.4 [notify cookie 94199696975360 out=8b] v0'0 uv75 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f2120ff9640 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f2100003ea0 con 0x55ac937608c0 2024-03-28T16:49:16.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== watch-notify(notify_complete (2) cookie 94199696975360 notify 124554051593 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f2238043b50 con 0x55ac937608c0 2024-03-28T16:49:16.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.744 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:16.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f20f3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94199696799424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93799f90 con 0x55ac937608c0 2024-03-28T16:49:16.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94199696799424] v29'85 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.743+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94199696775168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937736c0 con 0x7f22340e0040 2024-03-28T16:49:16.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94199696775168] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223fadc980 20 remove_watcher() i=0 2024-03-28T16:49:16.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223fadc980 2 removed watcher, disabling cache 2024-03-28T16:49:16.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94199696779520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937736c0 con 0x55ac937608c0 2024-03-28T16:49:16.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94199696779520] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223fadc980 20 remove_watcher() i=1 2024-03-28T16:49:16.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94199696783280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376e900 con 0x55ac93769a60 2024-03-28T16:49:16.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223cdf6640 1 -- 172.21.15.170:0/4228005355 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94199696783280] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f22340b0020 con 0x55ac93769a60 2024-03-28T16:49:16.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223fadc980 20 remove_watcher() i=2 2024-03-28T16:49:16.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.747+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94199696787008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937736c0 con 0x7f22340e0040 2024-03-28T16:49:16.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.751+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94199696787008] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x7f22340e0040 2024-03-28T16:49:16.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.751+0000 7f223fadc980 20 remove_watcher() i=3 2024-03-28T16:49:16.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.751+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94199696780576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376f7b0 con 0x55ac937608c0 2024-03-28T16:49:16.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.751+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94199696780576] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f223800a4d0 con 0x55ac937608c0 2024-03-28T16:49:16.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.751+0000 7f223fadc980 20 remove_watcher() i=4 2024-03-28T16:49:16.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.751+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94199696793472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac93770640 con 0x55ac93764e50 2024-03-28T16:49:16.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.755+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94199696793472] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.755+0000 7f223fadc980 20 remove_watcher() i=5 2024-03-28T16:49:16.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.755+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94199696791280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac937736c0 con 0x55ac93764e50 2024-03-28T16:49:16.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.755+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94199696791280] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.755+0000 7f223fadc980 20 remove_watcher() i=6 2024-03-28T16:49:16.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.755+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94199696797936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55ac9376f7b0 con 0x55ac93764e50 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223d5f7640 1 -- 172.21.15.170:0/4228005355 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94199696797936] v29'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2230016070 con 0x55ac93764e50 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 20 remove_watcher() i=7 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac93764e50 msgr2=0x55ac93767240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac93764e50 0x55ac93767240 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac937608c0 msgr2=0x55ac93762d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac937608c0 0x55ac93762d80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f22340e0040 msgr2=0x7f22340e0930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f22340e0040 0x7f22340e0930 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ac93769a60 msgr2=0x55ac9376bf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ac93769a60 0x55ac9376bf20 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21080342b0 msgr2=0x7f2108036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21080342b0 0x7f2108036770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f223401b380 tx=0x7f22340c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 msgr2=0x55ac937557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223ddf8640 1 -- 172.21.15.170:0/4228005355 reap_dead start 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac937557e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f223002eb90 tx=0x7f2230049f30 comp rx=0 tx=0).stop 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 shutdown_connections 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/4228005355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9374eae0 0x55ac937557e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 >> 172.21.15.170:0/4228005355 conn(0x55ac9374ae20 msgr2=0x55ac93748480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 shutdown_connections 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/4228005355 wait complete. 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21140342b0 msgr2=0x7f2114036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21140342b0 0x7f2114036770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ac93730260 tx=0x7f2234005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93741d00 msgr2=0x55ac93742140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93741d00 0x55ac93742140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f22300016b0 tx=0x7f2230024e40 comp rx=0 tx=0).stop 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 shutdown_connections 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21140342b0 0x7f2114036770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/1062084094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac93741d00 0x55ac93742140 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 >> 172.21.15.170:0/1062084094 conn(0x55ac93736ac0 msgr2=0x55ac937375d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 shutdown_connections 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/1062084094 wait complete. 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac9372d5a0 msgr2=0x55ac9372f990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ac9372d5a0 0x55ac9372f990 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac93728bf0 msgr2=0x55ac9372afe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ac93728bf0 0x55ac9372afe0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ac937312f0 msgr2=0x55ac937336e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ac937312f0 0x55ac937336e0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21240342b0 msgr2=0x7f2124036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21240342b0 0x7f2124036770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f22340ae000 tx=0x7f22340a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 msgr2=0x55ac93724020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 --2- 172.21.15.170:0/3629267564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac9362fe40 0x55ac93724020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55ac93719c20 tx=0x7f2230000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223ddf8640 1 -- 172.21.15.170:0/3629267564 reap_dead start 2024-03-28T16:49:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 shutdown_connections 2024-03-28T16:49:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 >> 172.21.15.170:0/3629267564 conn(0x55ac9362b710 msgr2=0x55ac93714480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 shutdown_connections 2024-03-28T16:49:16.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.759+0000 7f223fadc980 1 -- 172.21.15.170:0/3629267564 wait complete. 2024-03-28T16:49:16.770 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:16.771 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-03-28T16:49:16.771 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-03-28T16:49:16.771 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-03-28T16:49:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac957e980 1 Processor -- start 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac957e980 1 -- start start 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac957e980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5608583402b0 0x5608583426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac957e980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5608582884c0 con 0x5608582b0890 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac957e980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56085824f0f0 con 0x5608583402b0 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac6898640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5608583402b0 0x5608583426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac6898640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5608583402b0 0x5608583426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56004/0 (socket says 172.21.15.170:56004) 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac7099640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5608582b0890 0x56085833eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32774/0 (socket says 172.21.15.170:32774) 2024-03-28T16:49:16.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac6898640 1 -- 172.21.15.170:0/3046994134 learned_addr learned my addr 172.21.15.170:0/3046994134 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac6898640 1 -- 172.21.15.170:0/3046994134 >> v1:172.21.15.119:6789/0 conn(0x5608582b0890 legacy=0x56085833eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.847+0000 7feac6898640 1 -- 172.21.15.170:0/3046994134 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560858343c80 con 0x5608583402b0 2024-03-28T16:49:16.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac6898640 1 --2- 172.21.15.170:0/3046994134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608583402b0 0x5608583426a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feab000a5c0 tx=0x7feab001b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3979d378746e8896 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac6097640 1 -- 172.21.15.170:0/3046994134 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab00aa470 con 0x5608583402b0 2024-03-28T16:49:16.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac6097640 1 -- 172.21.15.170:0/3046994134 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab00aa5d0 con 0x5608583402b0 2024-03-28T16:49:16.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac6097640 1 -- 172.21.15.170:0/3046994134 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab00aa890 con 0x5608583402b0 2024-03-28T16:49:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac957e980 1 -- 172.21.15.170:0/3046994134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608583402b0 msgr2=0x5608583426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac957e980 1 --2- 172.21.15.170:0/3046994134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608583402b0 0x5608583426a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feab000a5c0 tx=0x7feab001b340 comp rx=0 tx=0).stop 2024-03-28T16:49:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac957e980 1 -- 172.21.15.170:0/3046994134 shutdown_connections 2024-03-28T16:49:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac957e980 1 --2- 172.21.15.170:0/3046994134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608583402b0 0x5608583426a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac957e980 1 -- 172.21.15.170:0/3046994134 >> 172.21.15.170:0/3046994134 conn(0x5608583366b0 msgr2=0x560858338ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac957e980 1 -- 172.21.15.170:0/3046994134 shutdown_connections 2024-03-28T16:49:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.851+0000 7feac957e980 1 -- 172.21.15.170:0/3046994134 wait complete. 2024-03-28T16:49:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac957e980 1 Processor -- start 2024-03-28T16:49:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac957e980 1 -- start start 2024-03-28T16:49:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac957e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x560858425e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac957e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56085824c620 con 0x5608582b0890 2024-03-28T16:49:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x560858425e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x560858425e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56008/0 (socket says 172.21.15.170:56008) 2024-03-28T16:49:16.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac7099640 1 -- 172.21.15.170:0/76853509 learned_addr learned my addr 172.21.15.170:0/76853509 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac7099640 1 -- 172.21.15.170:0/76853509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0005df0 con 0x5608582b0890 2024-03-28T16:49:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7feac7099640 1 --2- 172.21.15.170:0/76853509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x560858425e00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feab8006fc0 tx=0x7feab80019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fcd96ad25c07f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7fe9b560a640 1 -- 172.21.15.170:0/76853509 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8013420 con 0x5608582b0890 2024-03-28T16:49:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7fe9b560a640 1 -- 172.21.15.170:0/76853509 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8013580 con 0x5608582b0890 2024-03-28T16:49:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.863+0000 7fe9b560a640 1 -- 172.21.15.170:0/76853509 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8013880 con 0x5608582b0890 2024-03-28T16:49:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- 172.21.15.170:0/76853509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 msgr2=0x560858425e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 --2- 172.21.15.170:0/76853509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x560858425e00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feab8006fc0 tx=0x7feab80019b0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- 172.21.15.170:0/76853509 shutdown_connections 2024-03-28T16:49:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 --2- 172.21.15.170:0/76853509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x560858425e00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- 172.21.15.170:0/76853509 >> 172.21.15.170:0/76853509 conn(0x5608583366b0 msgr2=0x560858421990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- 172.21.15.170:0/76853509 shutdown_connections 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- 172.21.15.170:0/76853509 wait complete. 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 Processor -- start 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- start start 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x56085842cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560858247e00 con 0x5608582b0890 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x56085842cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x56085842cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56012/0 (socket says 172.21.15.170:56012) 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac7099640 1 -- 172.21.15.170:0/3457076757 learned_addr learned my addr 172.21.15.170:0/3457076757 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac7099640 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608582884c0 con 0x5608582b0890 2024-03-28T16:49:16.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac7099640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x56085842cd70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feab80012f0 tx=0x7feab8001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7fe9b3e07640 1 -- 172.21.15.170:0/3457076757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8006320 con 0x5608582b0890 2024-03-28T16:49:16.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7fe9b3e07640 1 -- 172.21.15.170:0/3457076757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8006840 con 0x5608582b0890 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56085842d2b0 con 0x5608582b0890 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7fe9b3e07640 1 -- 172.21.15.170:0/3457076757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8004c90 con 0x5608582b0890 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56085842d6e0 con 0x5608582b0890 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7fe9b3e07640 1 -- 172.21.15.170:0/3457076757 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7feab801e3f0 con 0x5608582b0890 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7fe9b3e07640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9ac0342b0 0x7fe9ac036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.867+0000 7fe9b3e07640 1 -- 172.21.15.170:0/3457076757 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7feab80495f0 con 0x5608582b0890 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac6898640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9ac0342b0 0x7fe9ac036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608583402b0 0x560858433950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858433e90 con 0x5608583402b0 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac789a640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608583402b0 0x560858433950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac6898640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9ac0342b0 0x7fe9ac036770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feab00ae000 tx=0x7feab00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac789a640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608583402b0 0x560858433950 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac789a640 1 -- 172.21.15.170:0/3457076757 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x5608583402b0 2024-03-28T16:49:16.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584341f0 con 0x5608583402b0 2024-03-28T16:49:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac789a640 1 -- 172.21.15.170:0/3457076757 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7feac00083f0 con 0x5608583402b0 2024-03-28T16:49:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858435f10 0x560858438300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858438840 con 0x560858435f10 2024-03-28T16:49:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac7099640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858435f10 0x560858438300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac7099640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858435f10 0x560858438300 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac7099640 1 -- 172.21.15.170:0/3457076757 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7feab8024070 con 0x560858435f10 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858434d80 con 0x5608583402b0 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac789a640 1 -- 172.21.15.170:0/3457076757 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7feac00083f0 con 0x5608583402b0 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560858439c00 0x56085843bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085843c640 con 0x560858439c00 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac6898640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560858439c00 0x56085843bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac6898640 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560858439c00 0x56085843bff0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.871+0000 7feac6898640 1 -- 172.21.15.170:0/3457076757 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7feab00b0020 con 0x560858439c00 2024-03-28T16:49:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 Processor -- start 2024-03-28T16:49:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 -- start start 2024-03-28T16:49:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56085833c6f0 0x56085833ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608581e0e30 con 0x56085833c6f0 2024-03-28T16:49:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56085833c6f0 0x56085833ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56085833c6f0 0x56085833ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56026/0 (socket says 172.21.15.170:56026) 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac7099640 1 -- 172.21.15.170:0/579589454 learned_addr learned my addr 172.21.15.170:0/579589454 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac7099640 1 -- 172.21.15.170:0/579589454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085833b0d0 con 0x56085833c6f0 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac7099640 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56085833c6f0 0x56085833ab90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feab8047bc0 tx=0x7feab8010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7fe9b1602640 1 -- 172.21.15.170:0/579589454 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8010900 con 0x56085833c6f0 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56085833b320 con 0x56085833c6f0 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7fe9b1602640 1 -- 172.21.15.170:0/579589454 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8010a80 con 0x56085833c6f0 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7fe9b1602640 1 -- 172.21.15.170:0/579589454 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8026c00 con 0x56085833c6f0 2024-03-28T16:49:16.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56085833b7b0 con 0x56085833c6f0 2024-03-28T16:49:16.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7fe9b1602640 1 -- 172.21.15.170:0/579589454 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7feab8026d60 con 0x56085833c6f0 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7fe9b1602640 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9a00342b0 0x7fe9a0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7fe9b1602640 1 -- 172.21.15.170:0/579589454 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7feab8015630 con 0x56085833c6f0 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac6898640 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9a00342b0 0x7fe9a0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 Processor -- start 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 -- start start 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac6898640 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9a00342b0 0x7fe9a0036770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5608584350e0 tx=0x7feab0005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858459a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.875+0000 7feac957e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608581d4790 con 0x560858459650 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858459a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858459a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56032/0 (socket says 172.21.15.170:56032) 2024-03-28T16:49:16.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 -- 172.21.15.170:0/1281793173 learned_addr learned my addr 172.21.15.170:0/1281793173 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 -- 172.21.15.170:0/1281793173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085845a020 con 0x560858459650 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 --2- 172.21.15.170:0/1281793173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858459a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feab8034010 tx=0x7feab803b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf00015196e77c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7fe9aaffd640 1 -- 172.21.15.170:0/1281793173 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab801ea70 con 0x560858459650 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7fe9aaffd640 1 -- 172.21.15.170:0/1281793173 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab801ebd0 con 0x560858459650 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7fe9aaffd640 1 -- 172.21.15.170:0/1281793173 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8027530 con 0x560858459650 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 -- 172.21.15.170:0/1281793173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 msgr2=0x560858459a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 --2- 172.21.15.170:0/1281793173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858459a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feab8034010 tx=0x7feab803b6c0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 -- 172.21.15.170:0/1281793173 shutdown_connections 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 --2- 172.21.15.170:0/1281793173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858459a50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 -- 172.21.15.170:0/1281793173 >> 172.21.15.170:0/1281793173 conn(0x5608584559e0 msgr2=0x560858453a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 -- 172.21.15.170:0/1281793173 shutdown_connections 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 -- 172.21.15.170:0/1281793173 wait complete. 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 Processor -- start 2024-03-28T16:49:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 -- start start 2024-03-28T16:49:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858460500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac957e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560858460a40 con 0x560858459650 2024-03-28T16:49:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858460500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858460500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56046/0 (socket says 172.21.15.170:56046) 2024-03-28T16:49:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 learned_addr learned my addr 172.21.15.170:0/2015256716 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56085833b320 con 0x560858459650 2024-03-28T16:49:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7feac7099640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858460500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feab801ed80 tx=0x7feab8015300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.879+0000 7fe9a97fa640 1 -- 172.21.15.170:0/2015256716 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab8027630 con 0x560858459650 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7fe9a97fa640 1 -- 172.21.15.170:0/2015256716 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8033c60 con 0x560858459650 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56085833b320 con 0x560858459650 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7fe9a97fa640 1 -- 172.21.15.170:0/2015256716 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feab803b830 con 0x560858459650 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56085833b320 con 0x560858459650 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7fe9a97fa640 1 -- 172.21.15.170:0/2015256716 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7feab803b990 con 0x560858459650 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7fe9a97fa640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9900342b0 0x7fe990036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7fe9a97fa640 1 -- 172.21.15.170:0/2015256716 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7feab8072af0 con 0x560858459650 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac6898640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9900342b0 0x7fe990036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56085846b750 0x56085846dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846e340 con 0x56085846b750 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac789a640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56085846b750 0x56085846dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac6898640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9900342b0 0x7fe990036770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feab0000c10 tx=0x7feab00c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac789a640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56085846b750 0x56085846dbf0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 realm 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846e760 con 0x56085846b750 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846efb0 con 0x56085846b750 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858470020 0x560858472410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858472ab0 con 0x560858470020 2024-03-28T16:49:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.883+0000 7feac7099640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858470020 0x560858472410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac7099640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858470020 0x560858472410 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac957e980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac957e980 20 searching for the correct realm 2024-03-28T16:49:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560858474c70 0x560858477130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584777b0 con 0x560858474c70 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac6898640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560858474c70 0x560858477130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac6898640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560858474c70 0x560858477130 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac6898640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7feab00e0040 0x7feab00d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584777b0 con 0x7feab00e0040 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7feab00e0040 0x7feab00d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7feab00e0040 0x7feab00d3400 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.887+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x7feab00e0040 2024-03-28T16:49:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x7feab00e0040 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.891+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x7feab00e0040 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.895+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x7feab00e0040 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858474c70 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x560858470020 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x56085846b750 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085846f310 con 0x7feab00e0040 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858473070 con 0x56085846b750 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858473070 con 0x56085846b750 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858473020 con 0x7feab00e0040 2024-03-28T16:49:16.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 zone default found 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 4 Realm: () 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 zonegroup default 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584738c0 con 0x560858470020 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858473d20 con 0x560858474c70 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584796d0 con 0x560858474c70 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 started sync module instance, tier type = 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.899+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858478eb0 con 0x7feab00e0040 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.903+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.903+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94593840810512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858479260 con 0x7feab00e0040 2024-03-28T16:49:16.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.903+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847a430 con 0x56085846b750 2024-03-28T16:49:16.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.903+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94593840810512] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.903+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.903+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94593840812288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847a050 con 0x56085846b750 2024-03-28T16:49:16.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.903+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847aad0 con 0x560858474c70 2024-03-28T16:49:16.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94593840817616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847b350 con 0x560858474c70 2024-03-28T16:49:16.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847bd00 con 0x7feab00e0040 2024-03-28T16:49:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94593840812288] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94593840814832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847c0e0 con 0x7feab00e0040 2024-03-28T16:49:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ca50 con 0x56085846b750 2024-03-28T16:49:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94593840817616] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.907+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94593840814832] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94593840822304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847c5b0 con 0x56085846b750 2024-03-28T16:49:16.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847cdb0 con 0x560858470020 2024-03-28T16:49:16.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94593840822304] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94593840825776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847c9d0 con 0x560858470020 2024-03-28T16:49:16.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847daa0 con 0x560858470020 2024-03-28T16:49:16.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.911+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94593840825776] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94593840829248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847e2a0 con 0x560858470020 2024-03-28T16:49:16.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ec10 con 0x560858470020 2024-03-28T16:49:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94593840829248] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94593840834416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847f030 con 0x560858470020 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=0 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=1 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=2 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=3 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=4 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=5 2024-03-28T16:49:16.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=6 2024-03-28T16:49:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94593840834416] v29'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 20 add_watcher() i=7 2024-03-28T16:49:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:16.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.915+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584822f0 con 0x56085846b750 2024-03-28T16:49:16.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94593840835840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847f5c0 con 0x56085846b750 2024-03-28T16:49:16.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94593840835840] v29'86 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7fe97dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 5 note: GC not initialized 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7fe97cfe1640 20 reqs_thread_entry: start 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 20 Filter name: none 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7fe8f67fc640 20 reqs_thread_entry: start 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584a4260 con 0x56085846b750 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv10 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 20 rados_obj.operate() r=0 bl.length=513 2024-03-28T16:49:16.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:16.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584aa260 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v29'11 uv11 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94593841012000 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584a5c10 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94593840822304 notify 124554051594 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7feac000aab0 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7fe9aaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051594 cookie 94593840822304 notifier 4410 bl.length()=533 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7fe9aaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7fe9aaffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(64 notify.4 [notify cookie 94593841012000 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7fe9aaffd640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fe98c003ea0 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.919+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94593841012000 notify 124554051594 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7feac000ac40 con 0x56085846b750 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac957e980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-03-28T16:49:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94593841012000 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584a5770 con 0x7feab00e0040 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== watch-notify(notify (1) cookie 94593840810512 notify 124554051589 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x7feac000ac40 con 0x7feab00e0040 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7fe9aaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051589 cookie 94593840810512 notifier 4410 bl.length()=159 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(66 notify.0 [notify cookie 94593841012000 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7fe9aaffd640 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fe98c003ea0 con 0x7feab00e0040 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== watch-notify(notify_complete (2) cookie 94593841012000 notify 124554051589 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7feac0043440 con 0x7feab00e0040 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(67 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584a9d20 con 0x7feab00e0040 2024-03-28T16:49:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(68 foo:foo2 [delete] v29'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.925 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7fe97dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.923+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94593840835840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608584a31c0 con 0x56085846b750 2024-03-28T16:49:16.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94593840835840] v29'87 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94593840810512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ed00 con 0x7feab00e0040 2024-03-28T16:49:16.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94593840810512] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac957e980 20 remove_watcher() i=0 2024-03-28T16:49:16.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac957e980 2 removed watcher, disabling cache 2024-03-28T16:49:16.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94593840812288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ed00 con 0x56085846b750 2024-03-28T16:49:16.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94593840812288] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac957e980 20 remove_watcher() i=1 2024-03-28T16:49:16.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.927+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94593840817616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560858479100 con 0x560858474c70 2024-03-28T16:49:16.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.931+0000 7feac6898640 1 -- 172.21.15.170:0/2015256716 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94593840817616] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab00b0020 con 0x560858474c70 2024-03-28T16:49:16.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.931+0000 7feac957e980 20 remove_watcher() i=2 2024-03-28T16:49:16.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.931+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94593840814832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ed00 con 0x7feab00e0040 2024-03-28T16:49:16.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.931+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94593840814832] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x7feab00e0040 2024-03-28T16:49:16.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.931+0000 7feac957e980 20 remove_watcher() i=3 2024-03-28T16:49:16.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.931+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94593840822304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847a5d0 con 0x56085846b750 2024-03-28T16:49:16.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94593840822304] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feac00083f0 con 0x56085846b750 2024-03-28T16:49:16.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac957e980 20 remove_watcher() i=4 2024-03-28T16:49:16.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94593840825776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ed00 con 0x560858470020 2024-03-28T16:49:16.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94593840825776] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac957e980 20 remove_watcher() i=5 2024-03-28T16:49:16.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94593840829248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ed00 con 0x560858470020 2024-03-28T16:49:16.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94593840829248] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac957e980 20 remove_watcher() i=6 2024-03-28T16:49:16.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.935+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94593840834416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56085847ed00 con 0x560858470020 2024-03-28T16:49:16.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac7099640 1 -- 172.21.15.170:0/2015256716 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94593840834416] v29'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7feab8024070 con 0x560858470020 2024-03-28T16:49:16.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 20 remove_watcher() i=7 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858470020 msgr2=0x560858472410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858470020 0x560858472410 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56085846b750 msgr2=0x56085846dbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56085846b750 0x56085846dbf0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7feab00e0040 msgr2=0x7feab00d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7feab00e0040 0x7feab00d3400 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560858474c70 msgr2=0x560858477130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560858474c70 0x560858477130 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9900342b0 msgr2=0x7fe990036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9900342b0 0x7fe990036770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feab0000c10 tx=0x7feab00c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac789a640 1 -- 172.21.15.170:0/2015256716 reap_dead start 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 msgr2=0x560858460500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858460500 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feab801ed80 tx=0x7feab8015300 comp rx=0 tx=0).stop 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 shutdown_connections 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/2015256716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560858459650 0x560858460500 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 >> 172.21.15.170:0/2015256716 conn(0x5608584559e0 msgr2=0x560858453040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 shutdown_connections 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/2015256716 wait complete. 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9a00342b0 msgr2=0x7fe9a0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9a00342b0 0x7fe9a0036770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5608584350e0 tx=0x7feab0005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56085833c6f0 msgr2=0x56085833ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56085833c6f0 0x56085833ab90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7feab8047bc0 tx=0x7feab8010650 comp rx=0 tx=0).stop 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 shutdown_connections 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9a00342b0 0x7fe9a0036770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/579589454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56085833c6f0 0x56085833ab90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 >> 172.21.15.170:0/579589454 conn(0x56085843f360 msgr2=0x56085843fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 shutdown_connections 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/579589454 wait complete. 2024-03-28T16:49:16.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858435f10 msgr2=0x560858438300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560858435f10 0x560858438300 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608583402b0 msgr2=0x560858433950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608583402b0 0x560858433950 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560858439c00 msgr2=0x56085843bff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560858439c00 0x56085843bff0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9ac0342b0 msgr2=0x7fe9ac036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe9ac0342b0 0x7fe9ac036770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feab00ae000 tx=0x7feab00a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 msgr2=0x56085842cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 --2- 172.21.15.170:0/3457076757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608582b0890 0x56085842cd70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feab80012f0 tx=0x7feab8001260 comp rx=0 tx=0).stop 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac789a640 1 -- 172.21.15.170:0/3457076757 reap_dead start 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 shutdown_connections 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 >> 172.21.15.170:0/3457076757 conn(0x5608583366b0 msgr2=0x5608584204a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 shutdown_connections 2024-03-28T16:49:16.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:16.939+0000 7feac957e980 1 -- 172.21.15.170:0/3457076757 wait complete. 2024-03-28T16:49:16.949 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:16.950 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-03-28T16:49:16.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-03-28T16:49:16.950 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-03-28T16:49:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f361aec2980 1 Processor -- start 2024-03-28T16:49:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f361aec2980 1 -- start start 2024-03-28T16:49:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f361aec2980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560000797070 0x560000825bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f361aec2980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56000076f520 con 0x560000826130 2024-03-28T16:49:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f361aec2980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5600007360f0 con 0x560000797070 2024-03-28T16:49:17.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f36189dd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560000797070 0x560000825bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f36189dd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560000797070 0x560000825bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56054/0 (socket says 172.21.15.170:56054) 2024-03-28T16:49:17.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f36189dd640 1 -- 172.21.15.170:0/1990180967 learned_addr learned my addr 172.21.15.170:0/1990180967 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f36189dd640 1 -- 172.21.15.170:0/1990180967 >> v1:172.21.15.119:6789/0 conn(0x560000826130 legacy=0x560000828520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.023+0000 7f36189dd640 1 -- 172.21.15.170:0/1990180967 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000082ace0 con 0x560000797070 2024-03-28T16:49:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f36189dd640 1 --2- 172.21.15.170:0/1990180967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000825bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f360800a4e0 tx=0x7f360801b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81c44170f0733df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f36179db640 1 -- 172.21.15.170:0/1990180967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080aa540 con 0x560000797070 2024-03-28T16:49:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f36179db640 1 -- 172.21.15.170:0/1990180967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36080a6070 con 0x560000797070 2024-03-28T16:49:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f36179db640 1 -- 172.21.15.170:0/1990180967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080aa850 con 0x560000797070 2024-03-28T16:49:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f361aec2980 1 -- 172.21.15.170:0/1990180967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 msgr2=0x560000825bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f361aec2980 1 --2- 172.21.15.170:0/1990180967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000825bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f360800a4e0 tx=0x7f360801b090 comp rx=0 tx=0).stop 2024-03-28T16:49:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f361aec2980 1 -- 172.21.15.170:0/1990180967 shutdown_connections 2024-03-28T16:49:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f361aec2980 1 --2- 172.21.15.170:0/1990180967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000825bf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f361aec2980 1 -- 172.21.15.170:0/1990180967 >> 172.21.15.170:0/1990180967 conn(0x56000081d6d0 msgr2=0x56000081faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f361aec2980 1 -- 172.21.15.170:0/1990180967 shutdown_connections 2024-03-28T16:49:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.027+0000 7f361aec2980 1 -- 172.21.15.170:0/1990180967 wait complete. 2024-03-28T16:49:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f361aec2980 1 Processor -- start 2024-03-28T16:49:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f361aec2980 1 -- start start 2024-03-28T16:49:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f361aec2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x56000090ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f361aec2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560000733620 con 0x560000797070 2024-03-28T16:49:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x56000090ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x56000090ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56070/0 (socket says 172.21.15.170:56070) 2024-03-28T16:49:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f36189dd640 1 -- 172.21.15.170:0/1089247815 learned_addr learned my addr 172.21.15.170:0/1089247815 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f36189dd640 1 -- 172.21.15.170:0/1089247815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360800a1c0 con 0x560000797070 2024-03-28T16:49:17.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f36189dd640 1 --2- 172.21.15.170:0/1089247815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x56000090ce10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3608017ca0 tx=0x7f36080033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1931831d9e9fbea server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f3506e0d640 1 -- 172.21.15.170:0/1089247815 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080aa780 con 0x560000797070 2024-03-28T16:49:17.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f3506e0d640 1 -- 172.21.15.170:0/1089247815 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36080aa8e0 con 0x560000797070 2024-03-28T16:49:17.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.039+0000 7f3506e0d640 1 -- 172.21.15.170:0/1089247815 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080aab80 con 0x560000797070 2024-03-28T16:49:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- 172.21.15.170:0/1089247815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 msgr2=0x56000090ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 --2- 172.21.15.170:0/1089247815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x56000090ce10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3608017ca0 tx=0x7f36080033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- 172.21.15.170:0/1089247815 shutdown_connections 2024-03-28T16:49:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 --2- 172.21.15.170:0/1089247815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x56000090ce10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- 172.21.15.170:0/1089247815 >> 172.21.15.170:0/1089247815 conn(0x56000081d6d0 msgr2=0x5600009089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- 172.21.15.170:0/1089247815 shutdown_connections 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- 172.21.15.170:0/1089247815 wait complete. 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 Processor -- start 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- start start 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000913b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000072ee00 con 0x560000797070 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000913b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000913b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56074/0 (socket says 172.21.15.170:56074) 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f36189dd640 1 -- 172.21.15.170:0/3537459038 learned_addr learned my addr 172.21.15.170:0/3537459038 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f36189dd640 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000082ace0 con 0x560000797070 2024-03-28T16:49:17.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f36189dd640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000913b60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f360801aec0 tx=0x7f3608017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f350560a640 1 -- 172.21.15.170:0/3537459038 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080c4070 con 0x560000797070 2024-03-28T16:49:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5600009140a0 con 0x560000797070 2024-03-28T16:49:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f350560a640 1 -- 172.21.15.170:0/3537459038 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608005150 con 0x560000797070 2024-03-28T16:49:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f350560a640 1 -- 172.21.15.170:0/3537459038 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080053f0 con 0x560000797070 2024-03-28T16:49:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600009144d0 con 0x560000797070 2024-03-28T16:49:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f350560a640 1 -- 172.21.15.170:0/3537459038 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f36080a8070 con 0x560000797070 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.043+0000 7f350560a640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35000342b0 0x7f3500036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f350560a640 1 -- 172.21.15.170:0/3537459038 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f36080e3190 con 0x560000797070 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36181dc640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35000342b0 0x7f3500036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000826130 0x56000091a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000091acd0 con 0x560000826130 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36191de640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000826130 0x56000091a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36181dc640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35000342b0 0x7f3500036770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3600000f70 tx=0x7f3600001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36191de640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000826130 0x56000091a790 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36191de640 1 -- 172.21.15.170:0/3537459038 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000826130 2024-03-28T16:49:17.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000091b030 con 0x560000826130 2024-03-28T16:49:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36191de640 1 -- 172.21.15.170:0/3537459038 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f36100083f0 con 0x560000826130 2024-03-28T16:49:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56000091cd50 0x56000091f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000091f680 con 0x56000091cd50 2024-03-28T16:49:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36189dd640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56000091cd50 0x56000091f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36189dd640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56000091cd50 0x56000091f140 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36189dd640 1 -- 172.21.15.170:0/3537459038 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f36080a97e0 con 0x56000091cd50 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000091bb90 con 0x560000826130 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36191de640 1 -- 172.21.15.170:0/3537459038 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f36100083f0 con 0x560000826130 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560000920a20 0x560000922e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009234b0 con 0x560000920a20 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36181dc640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560000920a20 0x560000922e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.047+0000 7f36181dc640 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560000920a20 0x560000922e10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36181dc640 1 -- 172.21.15.170:0/3537459038 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f36000106b0 con 0x560000920a20 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 Processor -- start 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 -- start start 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600009313e0 0x560000931800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600006c7e30 con 0x5600009313e0 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600009313e0 0x560000931800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600009313e0 0x560000931800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56076/0 (socket says 172.21.15.170:56076) 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36189dd640 1 -- 172.21.15.170:0/2346036598 learned_addr learned my addr 172.21.15.170:0/2346036598 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36189dd640 1 -- 172.21.15.170:0/2346036598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560000931d40 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36189dd640 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600009313e0 0x560000931800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f36080ac010 tx=0x7f36080bce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f34fe7fc640 1 -- 172.21.15.170:0/2346036598 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080b38d0 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560000931f90 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f34fe7fc640 1 -- 172.21.15.170:0/2346036598 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36080d8070 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560000932ae0 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f34fe7fc640 1 -- 172.21.15.170:0/2346036598 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080e0120 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f34fe7fc640 1 -- 172.21.15.170:0/2346036598 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f36080e0280 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f34fe7fc640 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f0034260 0x7f34f0036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f34fe7fc640 1 -- 172.21.15.170:0/2346036598 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f36080e4be0 con 0x5600009313e0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36181dc640 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f0034260 0x7f34f0036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 Processor -- start 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f361aec2980 1 -- start start 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.051+0000 7f36181dc640 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f0034260 0x7f34f0036720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x560000918220 tx=0x7f3600006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000823580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600006bb790 con 0x5600008231a0 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000823580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000823580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56088/0 (socket says 172.21.15.170:56088) 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 -- 172.21.15.170:0/4121024 learned_addr learned my addr 172.21.15.170:0/4121024 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 -- 172.21.15.170:0/4121024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600008219d0 con 0x5600008231a0 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 --2- 172.21.15.170:0/4121024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000823580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3608017ca0 tx=0x7f36080bd520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5033000d47fdeb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f34effff640 1 -- 172.21.15.170:0/4121024 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080f1070 con 0x5600008231a0 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f34effff640 1 -- 172.21.15.170:0/4121024 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36080f2040 con 0x5600008231a0 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f34effff640 1 -- 172.21.15.170:0/4121024 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080d2e70 con 0x5600008231a0 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- 172.21.15.170:0/4121024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 msgr2=0x560000823580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 --2- 172.21.15.170:0/4121024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000823580 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3608017ca0 tx=0x7f36080bd520 comp rx=0 tx=0).stop 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- 172.21.15.170:0/4121024 shutdown_connections 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 --2- 172.21.15.170:0/4121024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000823580 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- 172.21.15.170:0/4121024 >> 172.21.15.170:0/4121024 conn(0x56000093a640 msgr2=0x5600009386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- 172.21.15.170:0/4121024 shutdown_connections 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- 172.21.15.170:0/4121024 wait complete. 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 Processor -- start 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- start start 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000947310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f361aec2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560000947850 con 0x5600008231a0 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000947310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000947310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56102/0 (socket says 172.21.15.170:56102) 2024-03-28T16:49:17.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 learned_addr learned my addr 172.21.15.170:0/335185306 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560000932ae0 con 0x5600008231a0 2024-03-28T16:49:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.055+0000 7f36189dd640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000947310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f36080e05f0 tx=0x7f36080dbf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f34ee7fc640 1 -- 172.21.15.170:0/335185306 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080b5840 con 0x5600008231a0 2024-03-28T16:49:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560000947a50 con 0x5600008231a0 2024-03-28T16:49:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f34ee7fc640 1 -- 172.21.15.170:0/335185306 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608105070 con 0x5600008231a0 2024-03-28T16:49:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f34ee7fc640 1 -- 172.21.15.170:0/335185306 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36080b49b0 con 0x5600008231a0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560000947e80 con 0x5600008231a0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f34ee7fc640 1 -- 172.21.15.170:0/335185306 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3608107030 con 0x5600008231a0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f34ee7fc640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34e40342b0 0x7f34e4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f34ee7fc640 1 -- 172.21.15.170:0/335185306 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f360810e810 con 0x5600008231a0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36181dc640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34e40342b0 0x7f34e4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000952710 0x560000954bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955320 con 0x560000952710 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36191de640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000952710 0x560000954bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36181dc640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34e40342b0 0x7f34e4036770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56000091c810 tx=0x7f3600011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36191de640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000952710 0x560000954bd0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 realm 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009556f0 con 0x560000952710 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009562a0 con 0x560000952710 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560000956c50 0x560000959040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009595c0 con 0x560000956c50 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.059+0000 7f36189dd640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560000956c50 0x560000959040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36189dd640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560000956c50 0x560000959040 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f361aec2980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f361aec2980 20 searching for the correct realm 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56000095b450 0x56000095d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095df90 con 0x56000095b450 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36181dc640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56000095b450 0x56000095d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36181dc640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56000095b450 0x56000095d8c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36181dc640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f360000f3d0 0x7f360000f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095df90 con 0x7f360000f3d0 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f360000f3d0 0x7f360000f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f360000f3d0 0x7f360000f810 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.063+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x7f360000f3d0 2024-03-28T16:49:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x7f360000f3d0 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.067+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x7f360000f3d0 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.071+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x7f360000f3d0 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x56000095b450 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000956c50 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x560000952710 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000955f40 con 0x7f360000f3d0 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000959dc0 con 0x560000952710 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000959dc0 con 0x560000952710 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000959dc0 con 0x7f360000f3d0 2024-03-28T16:49:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 zone default found 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 4 Realm: () 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 zonegroup default 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095a610 con 0x560000956c50 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095aa10 con 0x56000095b450 2024-03-28T16:49:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.075+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095fe40 con 0x56000095b450 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 20 started sync module instance, tier type = 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095f690 con 0x7f360000f3d0 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94558009815536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095fa40 con 0x7f360000f3d0 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000960ba0 con 0x560000952710 2024-03-28T16:49:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94558009815536] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94558009819744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000960bc0 con 0x560000952710 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.079+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000961670 con 0x56000095b450 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94558009819744] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94558009820800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000961a50 con 0x56000095b450 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009623c0 con 0x7f360000f3d0 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94558009827376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000962a90 con 0x7f360000f3d0 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000963540 con 0x560000952710 2024-03-28T16:49:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.083+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94558009820800] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94558009827376] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94558009831488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000963500 con 0x560000952710 2024-03-28T16:49:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000963920 con 0x560000956c50 2024-03-28T16:49:17.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94558009831488] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94558009823600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000963cd0 con 0x560000956c50 2024-03-28T16:49:17.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.087+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000964640 con 0x560000956c50 2024-03-28T16:49:17.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94558009835936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000964a60 con 0x560000956c50 2024-03-28T16:49:17.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000965510 con 0x560000956c50 2024-03-28T16:49:17.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94558009823600] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94558009835936] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94558009839632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009645c0 con 0x560000956c50 2024-03-28T16:49:17.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 20 add_watcher() i=0 2024-03-28T16:49:17.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 20 add_watcher() i=1 2024-03-28T16:49:17.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 20 add_watcher() i=2 2024-03-28T16:49:17.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 20 add_watcher() i=3 2024-03-28T16:49:17.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 20 add_watcher() i=4 2024-03-28T16:49:17.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 20 add_watcher() i=5 2024-03-28T16:49:17.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.091+0000 7f361aec2980 20 add_watcher() i=6 2024-03-28T16:49:17.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94558009839632] v29'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 20 add_watcher() i=7 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009645c0 con 0x560000952710 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94558009828592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009645c0 con 0x560000952710 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94558009828592] v29'88 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f34cefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 5 note: GC not initialized 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f34cdfe3640 20 reqs_thread_entry: start 2024-03-28T16:49:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 20 Filter name: none 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f343f7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000989f50 con 0x560000952710 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000989f50 con 0x7f360000f3d0 2024-03-28T16:49:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.095+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f34cefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94558009828592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000989f10 con 0x560000952710 2024-03-28T16:49:17.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94558009828592] v29'89 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94558009815536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000989f10 con 0x7f360000f3d0 2024-03-28T16:49:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94558009815536] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f361aec2980 20 remove_watcher() i=0 2024-03-28T16:49:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f361aec2980 2 removed watcher, disabling cache 2024-03-28T16:49:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.099+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94558009819744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000989f10 con 0x560000952710 2024-03-28T16:49:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94558009819744] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f361aec2980 20 remove_watcher() i=1 2024-03-28T16:49:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94558009820800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000989f10 con 0x56000095b450 2024-03-28T16:49:17.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f36181dc640 1 -- 172.21.15.170:0/335185306 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94558009820800] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3600032040 con 0x56000095b450 2024-03-28T16:49:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f361aec2980 20 remove_watcher() i=2 2024-03-28T16:49:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94558009827376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56000095f1f0 con 0x7f360000f3d0 2024-03-28T16:49:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94558009827376] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x7f360000f3d0 2024-03-28T16:49:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f361aec2980 20 remove_watcher() i=3 2024-03-28T16:49:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.103+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94558009831488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000962030 con 0x560000952710 2024-03-28T16:49:17.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.107+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94558009831488] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36100083f0 con 0x560000952710 2024-03-28T16:49:17.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.107+0000 7f361aec2980 20 remove_watcher() i=4 2024-03-28T16:49:17.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.107+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94558009823600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000963040 con 0x560000956c50 2024-03-28T16:49:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.107+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94558009823600] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.107+0000 7f361aec2980 20 remove_watcher() i=5 2024-03-28T16:49:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.107+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94558009835936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560000962030 con 0x560000956c50 2024-03-28T16:49:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94558009835936] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 20 remove_watcher() i=6 2024-03-28T16:49:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94558009839632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5600009641a0 con 0x560000956c50 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f36189dd640 1 -- 172.21.15.170:0/335185306 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94558009839632] v29'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f36080f4070 con 0x560000956c50 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 20 remove_watcher() i=7 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560000956c50 msgr2=0x560000959040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560000956c50 0x560000959040 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000952710 msgr2=0x560000954bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000952710 0x560000954bd0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f360000f3d0 msgr2=0x7f360000f810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f360000f3d0 0x7f360000f810 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56000095b450 msgr2=0x56000095d8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56000095b450 0x56000095d8c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34e40342b0 msgr2=0x7f34e4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34e40342b0 0x7f34e4036770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56000091c810 tx=0x7f3600011000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f36191de640 1 -- 172.21.15.170:0/335185306 reap_dead start 2024-03-28T16:49:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 msgr2=0x560000947310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000947310 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f36080e05f0 tx=0x7f36080dbf90 comp rx=0 tx=0).stop 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 shutdown_connections 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/335185306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600008231a0 0x560000947310 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 >> 172.21.15.170:0/335185306 conn(0x56000093a640 msgr2=0x560000937ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 shutdown_connections 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/335185306 wait complete. 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f0034260 msgr2=0x7f34f0036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f0034260 0x7f34f0036720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x560000918220 tx=0x7f3600006000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600009313e0 msgr2=0x560000931800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600009313e0 0x560000931800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f36080ac010 tx=0x7f36080bce70 comp rx=0 tx=0).stop 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 shutdown_connections 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f0034260 0x7f34f0036720 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 --2- 172.21.15.170:0/2346036598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5600009313e0 0x560000931800 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 >> 172.21.15.170:0/2346036598 conn(0x5600009261f0 msgr2=0x560000926d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 shutdown_connections 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.111+0000 7f361aec2980 1 -- 172.21.15.170:0/2346036598 wait complete. 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56000091cd50 msgr2=0x56000091f140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56000091cd50 0x56000091f140 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000826130 msgr2=0x56000091a790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560000826130 0x56000091a790 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560000920a20 msgr2=0x560000922e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560000920a20 0x560000922e10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35000342b0 msgr2=0x7f3500036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35000342b0 0x7f3500036770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3600000f70 tx=0x7f3600001420 comp rx=0 tx=0).stop 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 msgr2=0x560000913b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 --2- 172.21.15.170:0/3537459038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560000797070 0x560000913b60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f360801aec0 tx=0x7f3608017e30 comp rx=0 tx=0).stop 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f36191de640 1 -- 172.21.15.170:0/3537459038 reap_dead start 2024-03-28T16:49:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 shutdown_connections 2024-03-28T16:49:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 >> 172.21.15.170:0/3537459038 conn(0x56000081d6d0 msgr2=0x5600009074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 shutdown_connections 2024-03-28T16:49:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.115+0000 7f361aec2980 1 -- 172.21.15.170:0/3537459038 wait complete. 2024-03-28T16:49:17.123 INFO:teuthology.orchestra.run.smithi170.stdout:[] 2024-03-28T16:49:17.123 DEBUG:tasks.util.rgw: json result: [] 2024-03-28T16:49:17.123 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-03-28T16:49:17.123 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-03-28T16:49:17.123 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-03-28T16:49:17.198 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:17.198 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:17.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7295b0f980 1 Processor -- start 2024-03-28T16:49:17.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7295b0f980 1 -- start start 2024-03-28T16:49:17.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7295b0f980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5608120e5690 0x560812178f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7295b0f980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5608120bc520 con 0x5608120e6980 2024-03-28T16:49:17.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7295b0f980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5608120830f0 con 0x5608120e5690 2024-03-28T16:49:17.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7292e29640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5608120e5690 0x560812178f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7292e29640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5608120e5690 0x560812178f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56110/0 (socket says 172.21.15.170:56110) 2024-03-28T16:49:17.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7292e29640 1 -- 172.21.15.170:0/247839723 learned_addr learned my addr 172.21.15.170:0/247839723 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7292e29640 1 -- 172.21.15.170:0/247839723 >> v1:172.21.15.119:6789/0 conn(0x5608120e6980 legacy=0x5608120e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.199+0000 7f7292e29640 1 -- 172.21.15.170:0/247839723 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608120e5aa0 con 0x5608120e5690 2024-03-28T16:49:17.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7292e29640 1 --2- 172.21.15.170:0/247839723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812178f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f728800a5c0 tx=0x7f728801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794bcebcdab63e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7292628640 1 -- 172.21.15.170:0/247839723 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72880ac460 con 0x5608120e5690 2024-03-28T16:49:17.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7292628640 1 -- 172.21.15.170:0/247839723 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72880ac5c0 con 0x5608120e5690 2024-03-28T16:49:17.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7292628640 1 -- 172.21.15.170:0/247839723 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72880ac880 con 0x5608120e5690 2024-03-28T16:49:17.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7295b0f980 1 -- 172.21.15.170:0/247839723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 msgr2=0x560812178f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7295b0f980 1 --2- 172.21.15.170:0/247839723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812178f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f728800a5c0 tx=0x7f728801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:17.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7295b0f980 1 -- 172.21.15.170:0/247839723 shutdown_connections 2024-03-28T16:49:17.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7295b0f980 1 --2- 172.21.15.170:0/247839723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812178f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7295b0f980 1 -- 172.21.15.170:0/247839723 >> 172.21.15.170:0/247839723 conn(0x56081216b850 msgr2=0x56081216dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7295b0f980 1 -- 172.21.15.170:0/247839723 shutdown_connections 2024-03-28T16:49:17.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.203+0000 7f7295b0f980 1 -- 172.21.15.170:0/247839723 wait complete. 2024-03-28T16:49:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 Processor -- start 2024-03-28T16:49:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- start start 2024-03-28T16:49:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x56081225c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560812080620 con 0x5608120e5690 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x56081225c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x56081225c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56120/0 (socket says 172.21.15.170:56120) 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 -- 172.21.15.170:0/2087603593 learned_addr learned my addr 172.21.15.170:0/2087603593 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 -- 172.21.15.170:0/2087603593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288005df0 con 0x5608120e5690 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 --2- 172.21.15.170:0/2087603593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x56081225c2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f728c001d10 tx=0x7f728c00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d009feed405a9f0a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f718160a640 1 -- 172.21.15.170:0/2087603593 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c014070 con 0x5608120e5690 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f718160a640 1 -- 172.21.15.170:0/2087603593 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c00f560 con 0x5608120e5690 2024-03-28T16:49:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f718160a640 1 -- 172.21.15.170:0/2087603593 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c00f840 con 0x5608120e5690 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- 172.21.15.170:0/2087603593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 msgr2=0x56081225c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2087603593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x56081225c2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f728c001d10 tx=0x7f728c00b5c0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- 172.21.15.170:0/2087603593 shutdown_connections 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2087603593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x56081225c2f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- 172.21.15.170:0/2087603593 >> 172.21.15.170:0/2087603593 conn(0x56081216b850 msgr2=0x560812257e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- 172.21.15.170:0/2087603593 shutdown_connections 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- 172.21.15.170:0/2087603593 wait complete. 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 Processor -- start 2024-03-28T16:49:17.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- start start 2024-03-28T16:49:17.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812263140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f7295b0f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081207be00 con 0x5608120e5690 2024-03-28T16:49:17.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812263140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812263140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56128/0 (socket says 172.21.15.170:56128) 2024-03-28T16:49:17.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 -- 172.21.15.170:0/4280071004 learned_addr learned my addr 172.21.15.170:0/4280071004 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.219+0000 7f729362a640 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608120bc520 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f729362a640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812263140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56081225d180 tx=0x7f728c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f717fe07640 1 -- 172.21.15.170:0/4280071004 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c014050 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560812263680 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f717fe07640 1 -- 172.21.15.170:0/4280071004 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c001520 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f717fe07640 1 -- 172.21.15.170:0/4280071004 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c01bab0 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560812263b10 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f717fe07640 1 -- 172.21.15.170:0/4280071004 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f728c022020 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f717fe07640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f71780342b0 0x7f7178036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f717fe07640 1 -- 172.21.15.170:0/4280071004 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f728c048f70 con 0x5608120e5690 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7292e29640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f71780342b0 0x7f7178036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608120e6980 0x560812269dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56081226a300 con 0x5608120e6980 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7293e2b640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608120e6980 0x560812269dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7292e29640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f71780342b0 0x7f7178036770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72880ae000 tx=0x7f72880a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7293e2b640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608120e6980 0x560812269dc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7293e2b640 1 -- 172.21.15.170:0/4280071004 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x5608120e6980 2024-03-28T16:49:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56081226a660 con 0x5608120e6980 2024-03-28T16:49:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7293e2b640 1 -- 172.21.15.170:0/4280071004 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f728400a4d0 con 0x5608120e6980 2024-03-28T16:49:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56081226c3d0 0x56081226e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56081226ed00 con 0x56081226c3d0 2024-03-28T16:49:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.223+0000 7f729362a640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56081226c3d0 0x56081226e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f729362a640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56081226c3d0 0x56081226e7c0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f729362a640 1 -- 172.21.15.170:0/4280071004 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f728c016070 con 0x56081226c3d0 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56081226b240 con 0x5608120e6980 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7293e2b640 1 -- 172.21.15.170:0/4280071004 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f728400a4d0 con 0x5608120e6980 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560812270120 0x560812272510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560812272b60 con 0x560812270120 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7292e29640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560812270120 0x560812272510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7292e29640 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560812270120 0x560812272510 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7292e29640 1 -- 172.21.15.170:0/4280071004 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f72880a9070 con 0x560812270120 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7295b0f980 1 Processor -- start 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7295b0f980 1 -- start start 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7295b0f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560812280a90 0x560812280eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f7295b0f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560812014e30 con 0x560812280a90 2024-03-28T16:49:17.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560812280a90 0x560812280eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560812280a90 0x560812280eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56130/0 (socket says 172.21.15.170:56130) 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f729362a640 1 -- 172.21.15.170:0/1419399753 learned_addr learned my addr 172.21.15.170:0/1419399753 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f729362a640 1 -- 172.21.15.170:0/1419399753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608122813f0 con 0x560812280a90 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.227+0000 7f729362a640 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560812280a90 0x560812280eb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f728c0474b0 tx=0x7f728c02ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f717d602640 1 -- 172.21.15.170:0/1419399753 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c025ca0 con 0x560812280a90 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560812281640 con 0x560812280a90 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560812282190 con 0x560812280a90 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f717d602640 1 -- 172.21.15.170:0/1419399753 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c035040 con 0x560812280a90 2024-03-28T16:49:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f717d602640 1 -- 172.21.15.170:0/1419399753 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c026a30 con 0x560812280a90 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f717d602640 1 -- 172.21.15.170:0/1419399753 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f728c04a070 con 0x560812280a90 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f717d602640 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f716c034300 0x7f716c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f717d602640 1 -- 172.21.15.170:0/1419399753 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f728c03cae0 con 0x560812280a90 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7292e29640 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f716c034300 0x7f716c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7295b0f980 1 Processor -- start 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7295b0f980 1 -- start start 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7292e29640 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f716c034300 0x7f716c0367c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x56081226b5a0 tx=0x7f7288005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7295b0f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x56081228dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7295b0f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560812008790 con 0x56081228d9c0 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x56081228dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x56081228dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56144/0 (socket says 172.21.15.170:56144) 2024-03-28T16:49:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f729362a640 1 -- 172.21.15.170:0/2413322543 learned_addr learned my addr 172.21.15.170:0/2413322543 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f729362a640 1 -- 172.21.15.170:0/2413322543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56081228e370 con 0x56081228d9c0 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f729362a640 1 --2- 172.21.15.170:0/2413322543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x56081228dda0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f728c0254d0 tx=0x7f728c026f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56bc355561ea29f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7176ffd640 1 -- 172.21.15.170:0/2413322543 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c0483b0 con 0x56081228d9c0 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7176ffd640 1 -- 172.21.15.170:0/2413322543 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c041480 con 0x56081228d9c0 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.231+0000 7f7176ffd640 1 -- 172.21.15.170:0/2413322543 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c0257a0 con 0x56081228d9c0 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2413322543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 msgr2=0x56081228dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2413322543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x56081228dda0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f728c0254d0 tx=0x7f728c026f60 comp rx=0 tx=0).stop 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2413322543 shutdown_connections 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2413322543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x56081228dda0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2413322543 >> 172.21.15.170:0/2413322543 conn(0x560812289d50 msgr2=0x560812287db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2413322543 shutdown_connections 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2413322543 wait complete. 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 Processor -- start 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- start start 2024-03-28T16:49:17.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x5608122948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560812294de0 con 0x56081228d9c0 2024-03-28T16:49:17.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x5608122948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f729362a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x5608122948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56148/0 (socket says 172.21.15.170:56148) 2024-03-28T16:49:17.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 learned_addr learned my addr 172.21.15.170:0/2530977122 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560812282190 con 0x56081228d9c0 2024-03-28T16:49:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f729362a640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x5608122948a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f728c02eb50 tx=0x7f728c003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f71757fa640 1 -- 172.21.15.170:0/2530977122 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c028330 con 0x56081228d9c0 2024-03-28T16:49:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f71757fa640 1 -- 172.21.15.170:0/2530977122 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c028490 con 0x56081228d9c0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560812282190 con 0x56081228d9c0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f71757fa640 1 -- 172.21.15.170:0/2530977122 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f728c026150 con 0x56081228d9c0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560812282190 con 0x56081228d9c0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f71757fa640 1 -- 172.21.15.170:0/2530977122 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f728c05f030 con 0x56081228d9c0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f71757fa640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f715c034260 0x7f715c036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f71757fa640 1 -- 172.21.15.170:0/2530977122 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f728c06eb00 con 0x56081228d9c0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7292e29640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f715c034260 0x7f715c036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56081229fa40 0x5608122a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.235+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a2630 con 0x56081229fa40 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7293e2b640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56081229fa40 0x5608122a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7292e29640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f715c034260 0x7f715c036720 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72880af040 tx=0x7f72880a8000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7293e2b640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56081229fa40 0x5608122a1ee0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 realm 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a2a00 con 0x56081229fa40 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a35b0 con 0x56081229fa40 2024-03-28T16:49:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5608122a3f60 0x5608122a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a68d0 con 0x5608122a3f60 2024-03-28T16:49:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f729362a640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5608122a3f60 0x5608122a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f729362a640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5608122a3f60 0x5608122a6350 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:17.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 20 searching for the correct realm 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5608122a87d0 0x5608122aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ab2b0 con 0x5608122a87d0 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7292e29640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5608122a87d0 0x5608122aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7292e29640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5608122a87d0 0x5608122aabe0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7292e29640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f72880db040 0x7f72880cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.239+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ab2b0 con 0x7f72880db040 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f72880db040 0x7f72880cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f72880db040 0x7f72880cd420 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x7f72880db040 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.243+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x7f72880db040 2024-03-28T16:49:17.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.247+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x7f72880db040 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x7f72880db040 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a87d0 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x5608122a3f60 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x56081229fa40 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.251+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a3250 con 0x7f72880db040 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:17.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a7140 con 0x56081229fa40 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a7110 con 0x56081229fa40 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a7110 con 0x7f72880db040 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 zone default found 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 4 Realm: () 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 zonegroup default 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a7990 con 0x5608122a3f60 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122a7d90 con 0x5608122a87d0 2024-03-28T16:49:17.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ad150 con 0x5608122a87d0 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 started sync module instance, tier type = 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ac9b0 con 0x7f72880db040 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94592664519952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122acd60 con 0x7f72880db040 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.255+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ade90 con 0x56081229fa40 2024-03-28T16:49:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94592664519952] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94592664524112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122aded0 con 0x56081229fa40 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ae980 con 0x5608122a87d0 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94592664524112] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94592664528000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122aed60 con 0x5608122a87d0 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.259+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122af6d0 con 0x7f72880db040 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94592664531776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122aff20 con 0x7f72880db040 2024-03-28T16:49:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b08d0 con 0x56081229fa40 2024-03-28T16:49:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94592664528000] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94592664531776] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94592664525168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122afa00 con 0x56081229fa40 2024-03-28T16:49:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.263+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b1150 con 0x5608122a3f60 2024-03-28T16:49:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94592664525168] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94592664538288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122afa00 con 0x5608122a3f60 2024-03-28T16:49:17.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b1990 con 0x5608122a3f60 2024-03-28T16:49:17.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94592664538288] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94592664536048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b1c30 con 0x5608122a3f60 2024-03-28T16:49:17.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.267+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b2700 con 0x5608122a3f60 2024-03-28T16:49:17.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94592664536048] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94592664543712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b26c0 con 0x5608122a3f60 2024-03-28T16:49:17.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=0 2024-03-28T16:49:17.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=1 2024-03-28T16:49:17.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=2 2024-03-28T16:49:17.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=3 2024-03-28T16:49:17.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=4 2024-03-28T16:49:17.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=5 2024-03-28T16:49:17.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=6 2024-03-28T16:49:17.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94592664543712] v29'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 20 add_watcher() i=7 2024-03-28T16:49:17.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:17.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b5b40 con 0x56081229fa40 2024-03-28T16:49:17.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.271+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94592664540336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122b2fe0 con 0x56081229fa40 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94592664540336] v29'90 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7149fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 5 note: GC not initialized 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7148fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 20 Filter name: none 2024-03-28T16:49:17.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f70c27fc640 20 reqs_thread_entry: start 2024-03-28T16:49:17.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122d7630 con 0x56081229fa40 2024-03-28T16:49:17.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:17.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122d7630 con 0x7f72880db040 2024-03-28T16:49:17.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7149fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94592664540336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122d75f0 con 0x56081229fa40 2024-03-28T16:49:17.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94592664540336] v29'91 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.275+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94592664519952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122d75f0 con 0x7f72880db040 2024-03-28T16:49:17.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.279+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94592664519952] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.279+0000 7f7295b0f980 20 remove_watcher() i=0 2024-03-28T16:49:17.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.279+0000 7f7295b0f980 2 removed watcher, disabling cache 2024-03-28T16:49:17.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.279+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94592664524112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122d75f0 con 0x56081229fa40 2024-03-28T16:49:17.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.279+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94592664524112] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.279+0000 7f7295b0f980 20 remove_watcher() i=1 2024-03-28T16:49:17.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.279+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94592664528000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122d75f0 con 0x5608122a87d0 2024-03-28T16:49:17.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7292e29640 1 -- 172.21.15.170:0/2530977122 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94592664528000] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f72880a9070 con 0x5608122a87d0 2024-03-28T16:49:17.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7295b0f980 20 remove_watcher() i=2 2024-03-28T16:49:17.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94592664531776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ac510 con 0x7f72880db040 2024-03-28T16:49:17.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94592664531776] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x7f72880db040 2024-03-28T16:49:17.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7295b0f980 20 remove_watcher() i=3 2024-03-28T16:49:17.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94592664525168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ad550 con 0x56081229fa40 2024-03-28T16:49:17.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94592664525168] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728400a4d0 con 0x56081229fa40 2024-03-28T16:49:17.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7295b0f980 20 remove_watcher() i=4 2024-03-28T16:49:17.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.283+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94592664538288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122af340 con 0x5608122a3f60 2024-03-28T16:49:17.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.287+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94592664538288] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.287+0000 7f7295b0f980 20 remove_watcher() i=5 2024-03-28T16:49:17.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.287+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94592664536048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122ac510 con 0x5608122a3f60 2024-03-28T16:49:17.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.287+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94592664536048] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.287+0000 7f7295b0f980 20 remove_watcher() i=6 2024-03-28T16:49:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.287+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94592664543712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5608122af340 con 0x5608122a3f60 2024-03-28T16:49:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f729362a640 1 -- 172.21.15.170:0/2530977122 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94592664543712] v29'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f728c016070 con 0x5608122a3f60 2024-03-28T16:49:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 20 remove_watcher() i=7 2024-03-28T16:49:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5608122a3f60 msgr2=0x5608122a6350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5608122a3f60 0x5608122a6350 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56081229fa40 msgr2=0x5608122a1ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56081229fa40 0x5608122a1ee0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f72880db040 msgr2=0x7f72880cd420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f72880db040 0x7f72880cd420 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5608122a87d0 msgr2=0x5608122aabe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5608122a87d0 0x5608122aabe0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f715c034260 msgr2=0x7f715c036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f715c034260 0x7f715c036720 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72880af040 tx=0x7f72880a8000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 msgr2=0x5608122948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/2530977122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56081228d9c0 0x5608122948a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f728c02eb50 tx=0x7f728c003cf0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 reap_dead start 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7293e2b640 1 -- 172.21.15.170:0/2530977122 reap_dead start 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 shutdown_connections 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 >> 172.21.15.170:0/2530977122 conn(0x560812289d50 msgr2=0x5608122873b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 shutdown_connections 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/2530977122 wait complete. 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f716c034300 msgr2=0x7f716c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f716c034300 0x7f716c0367c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x56081226b5a0 tx=0x7f7288005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560812280a90 msgr2=0x560812280eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560812280a90 0x560812280eb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f728c0474b0 tx=0x7f728c02ef70 comp rx=0 tx=0).stop 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 shutdown_connections 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f716c034300 0x7f716c0367c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/1419399753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560812280a90 0x560812280eb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 >> 172.21.15.170:0/1419399753 conn(0x5608122758a0 msgr2=0x5608122763b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 shutdown_connections 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/1419399753 wait complete. 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56081226c3d0 msgr2=0x56081226e7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56081226c3d0 0x56081226e7c0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608120e6980 msgr2=0x560812269dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5608120e6980 0x560812269dc0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560812270120 msgr2=0x560812272510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560812270120 0x560812272510 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f71780342b0 msgr2=0x7f7178036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f71780342b0 0x7f7178036770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72880ae000 tx=0x7f72880a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 msgr2=0x560812263140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 --2- 172.21.15.170:0/4280071004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5608120e5690 0x560812263140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56081225d180 tx=0x7f728c000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7293e2b640 1 -- 172.21.15.170:0/4280071004 reap_dead start 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 shutdown_connections 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 >> 172.21.15.170:0/4280071004 conn(0x56081216b850 msgr2=0x560812256990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 shutdown_connections 2024-03-28T16:49:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.291+0000 7f7295b0f980 1 -- 172.21.15.170:0/4280071004 wait complete. 2024-03-28T16:49:17.301 INFO:teuthology.orchestra.run.smithi170.stdout:[] 2024-03-28T16:49:17.301 DEBUG:tasks.util.rgw: json result: [] 2024-03-28T16:49:17.321 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.321 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-03-28T16:49:17.321 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-03-28T16:49:17.321 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-03-28T16:49:17.371 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:17.371 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:17.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1f1de0980 1 Processor -- start 2024-03-28T16:49:17.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1f1de0980 1 -- start start 2024-03-28T16:49:17.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1f1de0980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56338e7768b0 0x56338e807030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1f1de0980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56338e74e520 con 0x56338e807570 2024-03-28T16:49:17.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1f1de0980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56338e7150f0 con 0x56338e7768b0 2024-03-28T16:49:17.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1ef8fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56338e7768b0 0x56338e807030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1ef8fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56338e7768b0 0x56338e807030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56150/0 (socket says 172.21.15.170:56150) 2024-03-28T16:49:17.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1ef0fa640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x56338e807570 0x56338e809960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32812/0 (socket says 172.21.15.170:32812) 2024-03-28T16:49:17.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1228452566 learned_addr learned my addr 172.21.15.170:0/1228452566 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1228452566 >> v1:172.21.15.119:6789/0 conn(0x56338e807570 legacy=0x56338e809960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.371+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1228452566 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338e802a50 con 0x56338e7768b0 2024-03-28T16:49:17.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/1228452566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e807030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800a5c0 tx=0x7fe1d801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5806a768db68a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1ee8f9640 1 -- 172.21.15.170:0/1228452566 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80aa470 con 0x56338e7768b0 2024-03-28T16:49:17.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1ee8f9640 1 -- 172.21.15.170:0/1228452566 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80aa5d0 con 0x56338e7768b0 2024-03-28T16:49:17.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1ee8f9640 1 -- 172.21.15.170:0/1228452566 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80aa890 con 0x56338e7768b0 2024-03-28T16:49:17.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1228452566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 msgr2=0x56338e807030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1228452566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e807030 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800a5c0 tx=0x7fe1d801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:17.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1228452566 shutdown_connections 2024-03-28T16:49:17.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1228452566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e807030 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1228452566 >> 172.21.15.170:0/1228452566 conn(0x56338e7fc710 msgr2=0x56338e7feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1228452566 shutdown_connections 2024-03-28T16:49:17.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.375+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1228452566 wait complete. 2024-03-28T16:49:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.387+0000 7fe1f1de0980 1 Processor -- start 2024-03-28T16:49:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.387+0000 7fe1f1de0980 1 -- start start 2024-03-28T16:49:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.387+0000 7fe1f1de0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.387+0000 7fe1f1de0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338e712620 con 0x56338e7768b0 2024-03-28T16:49:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56152/0 (socket says 172.21.15.170:56152) 2024-03-28T16:49:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/382552739 learned_addr learned my addr 172.21.15.170:0/382552739 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/382552739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8005df0 con 0x56338e7768b0 2024-03-28T16:49:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/382552739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8ee0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80ac040 tx=0x7fe1d8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b2a628deae79d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe0dce19640 1 -- 172.21.15.170:0/382552739 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80aa9b0 con 0x56338e7768b0 2024-03-28T16:49:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe0dce19640 1 -- 172.21.15.170:0/382552739 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80aab30 con 0x56338e7768b0 2024-03-28T16:49:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe0dce19640 1 -- 172.21.15.170:0/382552739 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80aae10 con 0x56338e7768b0 2024-03-28T16:49:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- 172.21.15.170:0/382552739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 msgr2=0x56338e8ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/382552739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8ee0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80ac040 tx=0x7fe1d8003030 comp rx=0 tx=0).stop 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- 172.21.15.170:0/382552739 shutdown_connections 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/382552739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8ee0c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- 172.21.15.170:0/382552739 >> 172.21.15.170:0/382552739 conn(0x56338e7fc710 msgr2=0x56338e8e9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- 172.21.15.170:0/382552739 shutdown_connections 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- 172.21.15.170:0/382552739 wait complete. 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 Processor -- start 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- start start 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338e70de00 con 0x56338e7768b0 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56162/0 (socket says 172.21.15.170:56162) 2024-03-28T16:49:17.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/161180516 learned_addr learned my addr 172.21.15.170:0/161180516 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338e802a50 con 0x56338e7768b0 2024-03-28T16:49:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8f4e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8005940 tx=0x7fe1d80180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe0db616640 1 -- 172.21.15.170:0/161180516 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80c6750 con 0x56338e7768b0 2024-03-28T16:49:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56338e8f5350 con 0x56338e7768b0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe0db616640 1 -- 172.21.15.170:0/161180516 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80c68b0 con 0x56338e7768b0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe0db616640 1 -- 172.21.15.170:0/161180516 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80c6b70 con 0x56338e7768b0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.391+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56338e8f5780 con 0x56338e7768b0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe0db616640 1 -- 172.21.15.170:0/161180516 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe1d80c6cd0 con 0x56338e7768b0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe0db616640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0d40342b0 0x7fe0d4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe0db616640 1 -- 172.21.15.170:0/161180516 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe1d80e4230 con 0x56338e7768b0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0d40342b0 0x7fe0d4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e807570 0x56338e8fba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e8fbf80 con 0x56338e807570 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f00fc640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e807570 0x56338e8fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0d40342b0 0x7fe0d4036770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0000f70 tx=0x7fe1e0001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f00fc640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e807570 0x56338e8fba40 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f00fc640 1 -- 172.21.15.170:0/161180516 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e807570 2024-03-28T16:49:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e8fc2e0 con 0x56338e807570 2024-03-28T16:49:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f00fc640 1 -- 172.21.15.170:0/161180516 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e807570 2024-03-28T16:49:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e8fe000 0x56338e9003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e900930 con 0x56338e8fe000 2024-03-28T16:49:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e8fe000 0x56338e9003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.395+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e8fe000 0x56338e9003f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/161180516 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fe1d80b0020 con 0x56338e8fe000 2024-03-28T16:49:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e8fcdb0 con 0x56338e807570 2024-03-28T16:49:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f00fc640 1 -- 172.21.15.170:0/161180516 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e807570 2024-03-28T16:49:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56338e901ca0 0x56338e904090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9046a0 con 0x56338e901ca0 2024-03-28T16:49:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56338e901ca0 0x56338e904090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56338e901ca0 0x56338e904090 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/161180516 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fe1e000f6a0 con 0x56338e901ca0 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 Processor -- start 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 -- start start 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e9125d0 0x56338e9129f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338e6a6e30 con 0x56338e9125d0 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e9125d0 0x56338e9129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e9125d0 0x56338e9129f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56172/0 (socket says 172.21.15.170:56172) 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/311958936 learned_addr learned my addr 172.21.15.170:0/311958936 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/311958936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338e912f30 con 0x56338e9125d0 2024-03-28T16:49:17.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e9125d0 0x56338e9129f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80e2a70 tx=0x7fe1d80b3df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe0d8e11640 1 -- 172.21.15.170:0/311958936 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80acb90 con 0x56338e9125d0 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe0d8e11640 1 -- 172.21.15.170:0/311958936 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80accf0 con 0x56338e9125d0 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56338e913120 con 0x56338e9125d0 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe0d8e11640 1 -- 172.21.15.170:0/311958936 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80b33f0 con 0x56338e9125d0 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.399+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56338e913120 con 0x56338e9125d0 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe0d8e11640 1 -- 172.21.15.170:0/311958936 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe1d80b3550 con 0x56338e9125d0 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe0d8e11640 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0c80342b0 0x7fe0c8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe0d8e11640 1 -- 172.21.15.170:0/311958936 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe1d80b50b0 con 0x56338e9125d0 2024-03-28T16:49:17.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0c80342b0 0x7fe0c8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 Processor -- start 2024-03-28T16:49:17.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 -- start start 2024-03-28T16:49:17.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0c80342b0 0x7fe0c8036770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56338e8fd110 tx=0x7fe1e000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e91f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338e69a790 con 0x56338e91f230 2024-03-28T16:49:17.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e91f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e91f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56186/0 (socket says 172.21.15.170:56186) 2024-03-28T16:49:17.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/3371276901 learned_addr learned my addr 172.21.15.170:0/3371276901 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/3371276901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56338e91fc00 con 0x56338e91f230 2024-03-28T16:49:17.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/3371276901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e91f630 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80a5010 tx=0x7fe1d80be6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94229ea08963a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe0d27fc640 1 -- 172.21.15.170:0/3371276901 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80fc070 con 0x56338e91f230 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe0d27fc640 1 -- 172.21.15.170:0/3371276901 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80beaa0 con 0x56338e91f230 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe0d27fc640 1 -- 172.21.15.170:0/3371276901 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80e2520 con 0x56338e91f230 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 -- 172.21.15.170:0/3371276901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 msgr2=0x56338e91f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/3371276901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e91f630 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80a5010 tx=0x7fe1d80be6f0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 -- 172.21.15.170:0/3371276901 shutdown_connections 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/3371276901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e91f630 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 -- 172.21.15.170:0/3371276901 >> 172.21.15.170:0/3371276901 conn(0x56338e91b570 msgr2=0x56338e9195d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 -- 172.21.15.170:0/3371276901 shutdown_connections 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.403+0000 7fe1f1de0980 1 -- 172.21.15.170:0/3371276901 wait complete. 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 Processor -- start 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 -- start start 2024-03-28T16:49:17.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e926100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56338e926640 con 0x56338e91f230 2024-03-28T16:49:17.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e926100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1ef8fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e926100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56192/0 (socket says 172.21.15.170:56192) 2024-03-28T16:49:17.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 learned_addr learned my addr 172.21.15.170:0/1502252607 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d80e8030 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e926100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80e4fd0 tx=0x7fe1d80c2ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe0d0ff9640 1 -- 172.21.15.170:0/1502252607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80a4310 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe0d0ff9640 1 -- 172.21.15.170:0/1502252607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80a4490 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56338e926840 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56338e926840 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe0d0ff9640 1 -- 172.21.15.170:0/1502252607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80a4910 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe0d0ff9640 1 -- 172.21.15.170:0/1502252607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe1d80a4a70 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe0d0ff9640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b80342b0 0x7fe0b8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe0d0ff9640 1 -- 172.21.15.170:0/1502252607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe1d80da2f0 con 0x56338e91f230 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b80342b0 0x7fe0b8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e9312e0 0x56338e933770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e933ec0 con 0x56338e9312e0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1f00fc640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e9312e0 0x56338e933770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.407+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b80342b0 0x7fe0b8036770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c010 tx=0x7fe1e0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f00fc640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e9312e0 0x56338e933770 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 realm 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934290 con 0x56338e9312e0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934e70 con 0x56338e9312e0 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e935820 0x56338e937c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e938150 con 0x56338e935820 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e935820 0x56338e937c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef8fb640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e935820 0x56338e937c10 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 20 searching for the correct realm 2024-03-28T16:49:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56338e939fe0 0x56338e93c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93cb40 con 0x56338e939fe0 2024-03-28T16:49:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56338e939fe0 0x56338e93c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56338e939fe0 0x56338e93c470 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef0fa640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1e002e910 0x7fe1e003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93cb40 con 0x7fe1e002e910 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.411+0000 7fe1f00fc640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1e002e910 0x7fe1e003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1e002e910 0x7fe1e003a3c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x7fe1e002e910 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.415+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x7fe1e002e910 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x7fe1e002e910 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.419+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x7fe1e002e910 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e939fe0 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e935820 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x56338e9312e0 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e934b10 con 0x7fe1e002e910 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:17.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e938950 con 0x56338e9312e0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e938950 con 0x56338e9312e0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.423+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e938950 con 0x7fe1e002e910 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 zone default found 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 4 Realm: () 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 zonegroup default 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9391a0 con 0x56338e935820 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e939580 con 0x56338e939fe0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93ea90 con 0x56338e939fe0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 started sync module instance, tier type = 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93e240 con 0x7fe1e002e910 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94779435376032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93e5f0 con 0x7fe1e002e910 2024-03-28T16:49:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93f7f0 con 0x56338e9312e0 2024-03-28T16:49:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.427+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94779435376032] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94779435380400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93f770 con 0x56338e9312e0 2024-03-28T16:49:17.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e940210 con 0x56338e939fe0 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94779435384048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e940a60 con 0x56338e939fe0 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e941450 con 0x7fe1e002e910 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94779435380400] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94779435388720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e941800 con 0x7fe1e002e910 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.431+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e942170 con 0x56338e9312e0 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94779435384048] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94779435388720] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94779435381456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e941cd0 con 0x56338e9312e0 2024-03-28T16:49:17.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9429c0 con 0x56338e935820 2024-03-28T16:49:17.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94779435394336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e941cd0 con 0x56338e935820 2024-03-28T16:49:17.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9437d0 con 0x56338e935820 2024-03-28T16:49:17.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.435+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94779435381456] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94779435394336] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94779435397936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e943b30 con 0x56338e935820 2024-03-28T16:49:17.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e944330 con 0x56338e935820 2024-03-28T16:49:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94779435397936] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94779435399984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9443f0 con 0x56338e935820 2024-03-28T16:49:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 20 add_watcher() i=0 2024-03-28T16:49:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 20 add_watcher() i=1 2024-03-28T16:49:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 20 add_watcher() i=2 2024-03-28T16:49:17.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 20 add_watcher() i=3 2024-03-28T16:49:17.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 20 add_watcher() i=4 2024-03-28T16:49:17.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 20 add_watcher() i=5 2024-03-28T16:49:17.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.439+0000 7fe1f1de0980 20 add_watcher() i=6 2024-03-28T16:49:17.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94779435399984] v29'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 20 add_watcher() i=7 2024-03-28T16:49:17.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:17.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e947710 con 0x56338e9312e0 2024-03-28T16:49:17.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94779435401408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e944980 con 0x56338e9312e0 2024-03-28T16:49:17.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94779435401408] v29'92 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe0a57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:17.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 5 note: GC not initialized 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe01ffff640 20 reqs_thread_entry: start 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 20 Filter name: none 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe01dffb640 20 reqs_thread_entry: start 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e969210 con 0x56338e9312e0 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e969210 con 0x7fe1e002e910 2024-03-28T16:49:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.443+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe0a57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94779435401408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e92b010 con 0x56338e9312e0 2024-03-28T16:49:17.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94779435401408] v29'93 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94779435376032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9440c0 con 0x7fe1e002e910 2024-03-28T16:49:17.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94779435376032] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe1f1de0980 20 remove_watcher() i=0 2024-03-28T16:49:17.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe1f1de0980 2 removed watcher, disabling cache 2024-03-28T16:49:17.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.447+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94779435380400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9440c0 con 0x56338e9312e0 2024-03-28T16:49:17.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94779435380400] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f1de0980 20 remove_watcher() i=1 2024-03-28T16:49:17.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94779435384048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93eeb0 con 0x56338e939fe0 2024-03-28T16:49:17.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1ef0fa640 1 -- 172.21.15.170:0/1502252607 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94779435384048] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e0033040 con 0x56338e939fe0 2024-03-28T16:49:17.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f1de0980 20 remove_watcher() i=2 2024-03-28T16:49:17.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94779435388720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9440c0 con 0x7fe1e002e910 2024-03-28T16:49:17.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94779435388720] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x7fe1e002e910 2024-03-28T16:49:17.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f1de0980 20 remove_watcher() i=3 2024-03-28T16:49:17.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.451+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94779435381456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93fcf0 con 0x56338e9312e0 2024-03-28T16:49:17.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.455+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94779435381456] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1e800a4d0 con 0x56338e9312e0 2024-03-28T16:49:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.455+0000 7fe1f1de0980 20 remove_watcher() i=4 2024-03-28T16:49:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.455+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94779435394336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9440c0 con 0x56338e935820 2024-03-28T16:49:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.455+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94779435394336] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.455+0000 7fe1f1de0980 20 remove_watcher() i=5 2024-03-28T16:49:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.455+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94779435397936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e93fcf0 con 0x56338e935820 2024-03-28T16:49:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94779435397936] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 20 remove_watcher() i=6 2024-03-28T16:49:17.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94779435399984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56338e9440c0 con 0x56338e935820 2024-03-28T16:49:17.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1ef8fb640 1 -- 172.21.15.170:0/1502252607 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94779435399984] v29'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1d8108040 con 0x56338e935820 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 20 remove_watcher() i=7 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e935820 msgr2=0x56338e937c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e935820 0x56338e937c10 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e9312e0 msgr2=0x56338e933770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e9312e0 0x56338e933770 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1e002e910 msgr2=0x7fe1e003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1e002e910 0x7fe1e003a3c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56338e939fe0 msgr2=0x56338e93c470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56338e939fe0 0x56338e93c470 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b80342b0 msgr2=0x7fe0b8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b80342b0 0x7fe0b8036770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c010 tx=0x7fe1e0010000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f00fc640 1 -- 172.21.15.170:0/1502252607 reap_dead start 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 msgr2=0x56338e926100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e926100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80e4fd0 tx=0x7fe1d80c2ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 shutdown_connections 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/1502252607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e91f230 0x56338e926100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 >> 172.21.15.170:0/1502252607 conn(0x56338e91b570 msgr2=0x56338e918bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 shutdown_connections 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/1502252607 wait complete. 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0c80342b0 msgr2=0x7fe0c8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0c80342b0 0x7fe0c8036770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56338e8fd110 tx=0x7fe1e000f000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e9125d0 msgr2=0x56338e9129f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e9125d0 0x56338e9129f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80e2a70 tx=0x7fe1d80b3df0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 shutdown_connections 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0c80342b0 0x7fe0c8036770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/311958936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e9125d0 0x56338e9129f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 >> 172.21.15.170:0/311958936 conn(0x56338e9073e0 msgr2=0x56338e907ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 shutdown_connections 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/311958936 wait complete. 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e8fe000 msgr2=0x56338e9003f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56338e8fe000 0x56338e9003f0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e807570 msgr2=0x56338e8fba40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56338e807570 0x56338e8fba40 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56338e901ca0 msgr2=0x56338e904090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56338e901ca0 0x56338e904090 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0d40342b0 msgr2=0x7fe0d4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0d40342b0 0x7fe0d4036770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0000f70 tx=0x7fe1e0001380 comp rx=0 tx=0).stop 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 msgr2=0x56338e8f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f1de0980 1 --2- 172.21.15.170:0/161180516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56338e7768b0 0x56338e8f4e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8005940 tx=0x7fe1d80180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.459+0000 7fe1f00fc640 1 -- 172.21.15.170:0/161180516 reap_dead start 2024-03-28T16:49:17.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.463+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 shutdown_connections 2024-03-28T16:49:17.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.463+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 >> 172.21.15.170:0/161180516 conn(0x56338e7fc710 msgr2=0x56338e8e8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.463+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 shutdown_connections 2024-03-28T16:49:17.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.463+0000 7fe1f1de0980 1 -- 172.21.15.170:0/161180516 wait complete. 2024-03-28T16:49:17.470 INFO:teuthology.orchestra.run.smithi170.stdout:["myfoo"] 2024-03-28T16:49:17.470 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-03-28T16:49:17.474 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.474 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-03-28T16:49:17.475 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-03-28T16:49:17.475 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-03-28T16:49:17.548 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:17.548 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:17.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.547+0000 7f6dcf81b980 1 Processor -- start 2024-03-28T16:49:17.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.547+0000 7f6dcf81b980 1 -- start start 2024-03-28T16:49:17.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcf81b980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56547a533130 0x56547a5316e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcf81b980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56547a509510 con 0x56547a531c20 2024-03-28T16:49:17.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcf81b980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56547a4d00f0 con 0x56547a533130 2024-03-28T16:49:17.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcd336640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56547a533130 0x56547a5316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcd336640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56547a533130 0x56547a5316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56208/0 (socket says 172.21.15.170:56208) 2024-03-28T16:49:17.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dccb35640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x56547a531c20 0x56547a5c4f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32822/0 (socket says 172.21.15.170:32822) 2024-03-28T16:49:17.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcd336640 1 -- 172.21.15.170:0/421765113 learned_addr learned my addr 172.21.15.170:0/421765113 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcd336640 1 -- 172.21.15.170:0/421765113 >> v1:172.21.15.119:6789/0 conn(0x56547a531c20 legacy=0x56547a5c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcd336640 1 -- 172.21.15.170:0/421765113 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56547a532030 con 0x56547a533130 2024-03-28T16:49:17.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.551+0000 7f6dcd336640 1 --2- 172.21.15.170:0/421765113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a533130 0x56547a5316e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6db800a4e0 tx=0x7f6db801b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93dae9e942fc7d34 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dc7fff640 1 -- 172.21.15.170:0/421765113 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80ab410 con 0x56547a533130 2024-03-28T16:49:17.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dc7fff640 1 -- 172.21.15.170:0/421765113 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db80ab570 con 0x56547a533130 2024-03-28T16:49:17.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dc7fff640 1 -- 172.21.15.170:0/421765113 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80ab890 con 0x56547a533130 2024-03-28T16:49:17.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dcf81b980 1 -- 172.21.15.170:0/421765113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a533130 msgr2=0x56547a5316e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/421765113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a533130 0x56547a5316e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6db800a4e0 tx=0x7f6db801b030 comp rx=0 tx=0).stop 2024-03-28T16:49:17.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dcf81b980 1 -- 172.21.15.170:0/421765113 shutdown_connections 2024-03-28T16:49:17.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/421765113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a533130 0x56547a5316e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dcf81b980 1 -- 172.21.15.170:0/421765113 >> 172.21.15.170:0/421765113 conn(0x56547a5b8800 msgr2=0x56547a5bac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dcf81b980 1 -- 172.21.15.170:0/421765113 shutdown_connections 2024-03-28T16:49:17.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.555+0000 7f6dcf81b980 1 -- 172.21.15.170:0/421765113 wait complete. 2024-03-28T16:49:17.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcf81b980 1 Processor -- start 2024-03-28T16:49:17.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcf81b980 1 -- start start 2024-03-28T16:49:17.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcf81b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcf81b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547a4cd620 con 0x56547a531c20 2024-03-28T16:49:17.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6a95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56214/0 (socket says 172.21.15.170:56214) 2024-03-28T16:49:17.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcd336640 1 -- 172.21.15.170:0/1117797643 learned_addr learned my addr 172.21.15.170:0/1117797643 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.567+0000 7f6dcd336640 1 -- 172.21.15.170:0/1117797643 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db800a1c0 con 0x56547a531c20 2024-03-28T16:49:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcd336640 1 --2- 172.21.15.170:0/1117797643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6a95a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6db800a4e0 tx=0x7f6db80033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ce1480401120bf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6cba614640 1 -- 172.21.15.170:0/1117797643 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80ab810 con 0x56547a531c20 2024-03-28T16:49:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6cba614640 1 -- 172.21.15.170:0/1117797643 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db80ab970 con 0x56547a531c20 2024-03-28T16:49:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6cba614640 1 -- 172.21.15.170:0/1117797643 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80abc50 con 0x56547a531c20 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1117797643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 msgr2=0x56547a6a95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1117797643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6a95a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6db800a4e0 tx=0x7f6db80033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1117797643 shutdown_connections 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1117797643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6a95a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1117797643 >> 172.21.15.170:0/1117797643 conn(0x56547a5b8800 msgr2=0x56547a6a5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1117797643 shutdown_connections 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1117797643 wait complete. 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 Processor -- start 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- start start 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547a4c8e00 con 0x56547a531c20 2024-03-28T16:49:17.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56230/0 (socket says 172.21.15.170:56230) 2024-03-28T16:49:17.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcd336640 1 -- 172.21.15.170:0/4077269308 learned_addr learned my addr 172.21.15.170:0/4077269308 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcd336640 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80af040 con 0x56547a531c20 2024-03-28T16:49:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcd336640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6b0330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6db8005cc0 tx=0x7f6db8017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6cb8e11640 1 -- 172.21.15.170:0/4077269308 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80b4740 con 0x56547a531c20 2024-03-28T16:49:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56547a6b0870 con 0x56547a531c20 2024-03-28T16:49:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6cb8e11640 1 -- 172.21.15.170:0/4077269308 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db80b48a0 con 0x56547a531c20 2024-03-28T16:49:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6cb8e11640 1 -- 172.21.15.170:0/4077269308 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80b4b40 con 0x56547a531c20 2024-03-28T16:49:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.571+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56547a6b0ca0 con 0x56547a531c20 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6cb8e11640 1 -- 172.21.15.170:0/4077269308 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f6db80b4cf0 con 0x56547a531c20 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6cb8e11640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6cb4034350 0x7f6cb4036810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6cb8e11640 1 -- 172.21.15.170:0/4077269308 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f6db80e3810 con 0x56547a531c20 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dccb35640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6cb4034350 0x7f6cb4036810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a533130 0x56547a6b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6b74a0 con 0x56547a533130 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcdb37640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a533130 0x56547a6b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcdb37640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a533130 0x56547a6b6f60 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dccb35640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6cb4034350 0x7f6cb4036810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8000f70 tx=0x7f6dc8001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcdb37640 1 -- 172.21.15.170:0/4077269308 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a533130 2024-03-28T16:49:17.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6b7800 con 0x56547a533130 2024-03-28T16:49:17.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcdb37640 1 -- 172.21.15.170:0/4077269308 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a533130 2024-03-28T16:49:17.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6b9520 0x56547a6bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6bbe50 con 0x56547a6b9520 2024-03-28T16:49:17.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcd336640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6b9520 0x56547a6bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.575+0000 7f6dcd336640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6b9520 0x56547a6bb910 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcd336640 1 -- 172.21.15.170:0/4077269308 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f6db80a8070 con 0x56547a6b9520 2024-03-28T16:49:17.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6b8390 con 0x56547a533130 2024-03-28T16:49:17.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcdb37640 1 -- 172.21.15.170:0/4077269308 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a533130 2024-03-28T16:49:17.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56547a6bd210 0x56547a6bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6bfca0 con 0x56547a6bd210 2024-03-28T16:49:17.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dccb35640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56547a6bd210 0x56547a6bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dccb35640 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56547a6bd210 0x56547a6bf600 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dccb35640 1 -- 172.21.15.170:0/4077269308 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f6dc800f6a0 con 0x56547a6bd210 2024-03-28T16:49:17.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 Processor -- start 2024-03-28T16:49:17.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 -- start start 2024-03-28T16:49:17.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6cdbd0 0x56547a6cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547a461e30 con 0x56547a6cdbd0 2024-03-28T16:49:17.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6cdbd0 0x56547a6cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6cdbd0 0x56547a6cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56240/0 (socket says 172.21.15.170:56240) 2024-03-28T16:49:17.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcd336640 1 -- 172.21.15.170:0/1842590177 learned_addr learned my addr 172.21.15.170:0/1842590177 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcd336640 1 -- 172.21.15.170:0/1842590177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56547a6ce530 con 0x56547a6cdbd0 2024-03-28T16:49:17.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcd336640 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6cdbd0 0x56547a6cdff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6db801b070 tx=0x7f6db80aa940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6cb1ffb640 1 -- 172.21.15.170:0/1842590177 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80aadf0 con 0x56547a6cdbd0 2024-03-28T16:49:17.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56547a6ce780 con 0x56547a6cdbd0 2024-03-28T16:49:17.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56547a6cf2d0 con 0x56547a6cdbd0 2024-03-28T16:49:17.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6cb1ffb640 1 -- 172.21.15.170:0/1842590177 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db80da040 con 0x56547a6cdbd0 2024-03-28T16:49:17.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.579+0000 7f6cb1ffb640 1 -- 172.21.15.170:0/1842590177 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80e3c10 con 0x56547a6cdbd0 2024-03-28T16:49:17.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6cb1ffb640 1 -- 172.21.15.170:0/1842590177 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f6db80bf470 con 0x56547a6cdbd0 2024-03-28T16:49:17.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6cb1ffb640 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ca4034260 0x7f6ca4036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6cb1ffb640 1 -- 172.21.15.170:0/1842590177 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f6db80e7810 con 0x56547a6cdbd0 2024-03-28T16:49:17.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dccb35640 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ca4034260 0x7f6ca4036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 Processor -- start 2024-03-28T16:49:17.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 -- start start 2024-03-28T16:49:17.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dccb35640 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ca4034260 0x7f6ca4036720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x56547a6b4a10 tx=0x7f6dc800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547a455790 con 0x56547a6da9e0 2024-03-28T16:49:17.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6dadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56254/0 (socket says 172.21.15.170:56254) 2024-03-28T16:49:17.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcd336640 1 -- 172.21.15.170:0/1261564389 learned_addr learned my addr 172.21.15.170:0/1261564389 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcd336640 1 -- 172.21.15.170:0/1261564389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56547a6db390 con 0x56547a6da9e0 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcd336640 1 --2- 172.21.15.170:0/1261564389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6dadc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6db80b3680 tx=0x7f6db80c6a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dc533e524b7e2a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6cab7fe640 1 -- 172.21.15.170:0/1261564389 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80de3f0 con 0x56547a6da9e0 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6cab7fe640 1 -- 172.21.15.170:0/1261564389 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db80f4070 con 0x56547a6da9e0 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6cab7fe640 1 -- 172.21.15.170:0/1261564389 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80e3ea0 con 0x56547a6da9e0 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1261564389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 msgr2=0x56547a6dadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1261564389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6dadc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6db80b3680 tx=0x7f6db80c6a60 comp rx=0 tx=0).stop 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1261564389 shutdown_connections 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1261564389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6dadc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1261564389 >> 172.21.15.170:0/1261564389 conn(0x56547a6d6d70 msgr2=0x56547a6d4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1261564389 shutdown_connections 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1261564389 wait complete. 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.583+0000 7f6dcf81b980 1 Processor -- start 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 -- start start 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56547a6e1c00 con 0x56547a6da9e0 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcd336640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6e16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56264/0 (socket says 172.21.15.170:56264) 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 learned_addr learned my addr 172.21.15.170:0/1712094319 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56547a6ce530 con 0x56547a6da9e0 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcd336640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6e16c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6db80c6b30 tx=0x7f6db8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6ca9ffb640 1 -- 172.21.15.170:0/1712094319 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db8108070 con 0x56547a6da9e0 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56547a6ce530 con 0x56547a6da9e0 2024-03-28T16:49:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6ca9ffb640 1 -- 172.21.15.170:0/1712094319 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db80c6310 con 0x56547a6da9e0 2024-03-28T16:49:17.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6ca9ffb640 1 -- 172.21.15.170:0/1712094319 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6db80e0d20 con 0x56547a6da9e0 2024-03-28T16:49:17.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56547a6e1fe0 con 0x56547a6da9e0 2024-03-28T16:49:17.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6ca9ffb640 1 -- 172.21.15.170:0/1712094319 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f6db80f6030 con 0x56547a6da9e0 2024-03-28T16:49:17.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6ca9ffb640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6c9c0342b0 0x7f6c9c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6ca9ffb640 1 -- 172.21.15.170:0/1712094319 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f6db8112810 con 0x56547a6da9e0 2024-03-28T16:49:17.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dccb35640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6c9c0342b0 0x7f6c9c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a6ec930 0x56547a6eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6ef520 con 0x56547a6ec930 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcdb37640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a6ec930 0x56547a6eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dccb35640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6c9c0342b0 0x7f6c9c036770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c010 tx=0x7f6dc8010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcdb37640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a6ec930 0x56547a6eedd0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 20 realm 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.587+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6ef8f0 con 0x56547a6ec930 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f04a0 con 0x56547a6ec930 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6f0e50 0x56547a6f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f37c0 con 0x56547a6f0e50 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcd336640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6f0e50 0x56547a6f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcd336640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6f0e50 0x56547a6f3240 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 20 searching for the correct realm 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56547a6f56c0 0x56547a6f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f81e0 con 0x56547a6f56c0 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dccb35640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56547a6f56c0 0x56547a6f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dccb35640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56547a6f56c0 0x56547a6f7b10 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dccb35640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6dc801fc60 0x7f6dc803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f81e0 con 0x7f6dc801fc60 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcdb37640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6dc801fc60 0x7f6dc803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcdb37640 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6dc801fc60 0x7f6dc803a3c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.591+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x7f6dc801fc60 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.595+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x7f6dc801fc60 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x7f6dc801fc60 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.599+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x7f6dc801fc60 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x7f6dc801fc60 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:17.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f4030 con 0x56547a6ec930 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f4030 con 0x56547a6ec930 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.603+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f4030 con 0x7f6dc801fc60 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:17.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 zone default found 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 4 Realm: () 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 zonegroup default 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f4880 con 0x56547a6f0e50 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f4c60 con 0x56547a6f56c0 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fa100 con 0x56547a6f56c0 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 started sync module instance, tier type = 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f98e0 con 0x7f6dc801fc60 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94920831374400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f9c90 con 0x7f6dc801fc60 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fae60 con 0x56547a6ec930 2024-03-28T16:49:17.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.607+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94920831374400] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94920831378720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fae40 con 0x56547a6ec930 2024-03-28T16:49:17.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fb900 con 0x56547a6f56c0 2024-03-28T16:49:17.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94920831378720] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94920831382464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fbcb0 con 0x56547a6f56c0 2024-03-28T16:49:17.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fc620 con 0x7f6dc801fc60 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94920831386256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fccf0 con 0x7f6dc801fc60 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.611+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fd7c0 con 0x56547a6ec930 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94920831382464] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94920831386256] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94920831390368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fd780 con 0x56547a6ec930 2024-03-28T16:49:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fdb70 con 0x56547a6f0e50 2024-03-28T16:49:17.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94920831390368] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94920831379776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fdf20 con 0x56547a6f0e50 2024-03-28T16:49:17.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.615+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f0e50 2024-03-28T16:49:17.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.619+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94920831379776] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.619+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.619+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94920831387472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6ff090 con 0x56547a6f0e50 2024-03-28T16:49:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.619+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6ff8f0 con 0x56547a6f0e50 2024-03-28T16:49:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.619+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94920831387472] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94920831396848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6ffad0 con 0x56547a6f0e50 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=0 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=1 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=2 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=3 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=4 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=5 2024-03-28T16:49:17.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=6 2024-03-28T16:49:17.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94920831396848] v29'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 add_watcher() i=7 2024-03-28T16:49:17.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:17.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a702e90 con 0x56547a6ec930 2024-03-28T16:49:17.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94920831400384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a7002f0 con 0x56547a6ec930 2024-03-28T16:49:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94920831400384] v29'94 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6c827e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 5 note: GC not initialized 2024-03-28T16:49:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6c817e2640 20 reqs_thread_entry: start 2024-03-28T16:49:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6dcf81b980 20 Filter name: none 2024-03-28T16:49:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.623+0000 7f6bf2ffd640 20 reqs_thread_entry: start 2024-03-28T16:49:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a724470 con 0x56547a6f56c0 2024-03-28T16:49:17.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(62 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a724470 con 0x56547a6f56c0 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(63 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a724470 con 0x56547a6ec930 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(64 [pgnls start_epoch 29 out=49b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 [pgnls start_epoch 29 out=49b] v29'11 uv11 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x7f6dc801fc60 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(67 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(69 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x7f6dc801fc60 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 [pgnls start_epoch 29 out=49b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f56c0 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(71 [pgnls start_epoch 29 out=127b] v29'1 uv1 ondisk = 1) v8 ==== 144+0+127 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(72 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6f0e50 2024-03-28T16:49:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(73 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.627+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f0e50 2024-03-28T16:49:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(74 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f56c0 2024-03-28T16:49:17.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(75 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f0e50 2024-03-28T16:49:17.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(76 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f0e50 2024-03-28T16:49:17.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(77 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f0e50 2024-03-28T16:49:17.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(78 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f56c0 2024-03-28T16:49:17.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(79 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x7f6dc801fc60 2024-03-28T16:49:17.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(80 [pgnls start_epoch 29 out=70b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+70 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6ec930 2024-03-28T16:49:17.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(81 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6ec930 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(82 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f56c0 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.631+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(83 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x7f6dc801fc60 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(84 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f0e50 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(85 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6ec930 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(86 [pgnls start_epoch 29 out=49b] v29'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6dc004f040 con 0x56547a6f0e50 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(87 [pgnls start_epoch 29 out=49b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6dc004f040 con 0x56547a6f56c0 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(88 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6dc004f040 con 0x7f6dc801fc60 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(89 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6dc004f040 con 0x56547a6f0e50 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(90 [pgnls start_epoch 29 out=49b] v29'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6dc004f040 con 0x56547a6ec930 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(91 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6dc004f040 con 0x56547a6f56c0 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(92 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6dc004f040 con 0x7f6dc801fc60 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(93 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.635+0000 7f6dcf81b980 20 RGWRados::pool_iterate: got myfoo 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6c827e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94920831400384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0140 con 0x56547a6ec930 2024-03-28T16:49:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94920831400384] v29'95 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94920831374400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x7f6dc801fc60 2024-03-28T16:49:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94920831374400] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6dcf81b980 20 remove_watcher() i=0 2024-03-28T16:49:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6dcf81b980 2 removed watcher, disabling cache 2024-03-28T16:49:17.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.639+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94920831378720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6ec930 2024-03-28T16:49:17.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94920831378720] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcf81b980 20 remove_watcher() i=1 2024-03-28T16:49:17.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94920831382464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fe890 con 0x56547a6f56c0 2024-03-28T16:49:17.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dccb35640 1 -- 172.21.15.170:0/1712094319 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94920831382464] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc8033040 con 0x56547a6f56c0 2024-03-28T16:49:17.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcf81b980 20 remove_watcher() i=2 2024-03-28T16:49:17.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94920831386256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0100 con 0x7f6dc801fc60 2024-03-28T16:49:17.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94920831386256] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x7f6dc801fc60 2024-03-28T16:49:17.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcf81b980 20 remove_watcher() i=3 2024-03-28T16:49:17.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.643+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94920831390368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0100 con 0x56547a6ec930 2024-03-28T16:49:17.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.647+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94920831390368] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6dc00083f0 con 0x56547a6ec930 2024-03-28T16:49:17.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.647+0000 7f6dcf81b980 20 remove_watcher() i=4 2024-03-28T16:49:17.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.647+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94920831379776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f0100 con 0x56547a6f0e50 2024-03-28T16:49:17.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.647+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94920831379776] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.647+0000 7f6dcf81b980 20 remove_watcher() i=5 2024-03-28T16:49:17.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.647+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94920831387472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6f9440 con 0x56547a6f0e50 2024-03-28T16:49:17.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94920831387472] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 20 remove_watcher() i=6 2024-03-28T16:49:17.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94920831396848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56547a6fc290 con 0x56547a6f0e50 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcd336640 1 -- 172.21.15.170:0/1712094319 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94920831396848] v29'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f6db810a070 con 0x56547a6f0e50 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 20 remove_watcher() i=7 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6f0e50 msgr2=0x56547a6f3240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6f0e50 0x56547a6f3240 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a6ec930 msgr2=0x56547a6eedd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a6ec930 0x56547a6eedd0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6dc801fc60 msgr2=0x7f6dc803a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f6dc801fc60 0x7f6dc803a3c0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56547a6f56c0 msgr2=0x56547a6f7b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56547a6f56c0 0x56547a6f7b10 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6c9c0342b0 msgr2=0x7f6c9c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6c9c0342b0 0x7f6c9c036770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800c010 tx=0x7f6dc8010000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 msgr2=0x56547a6e16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcdb37640 1 -- 172.21.15.170:0/1712094319 reap_dead start 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6e16c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6db80c6b30 tx=0x7f6db8009f90 comp rx=0 tx=0).stop 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 shutdown_connections 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1712094319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6da9e0 0x56547a6e16c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 >> 172.21.15.170:0/1712094319 conn(0x56547a6d6d70 msgr2=0x56547a6d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 shutdown_connections 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1712094319 wait complete. 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ca4034260 msgr2=0x7f6ca4036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ca4034260 0x7f6ca4036720 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x56547a6b4a10 tx=0x7f6dc800f000 comp rx=0 tx=0).stop 2024-03-28T16:49:17.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6cdbd0 msgr2=0x56547a6cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6cdbd0 0x56547a6cdff0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6db801b070 tx=0x7f6db80aa940 comp rx=0 tx=0).stop 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 shutdown_connections 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6ca4034260 0x7f6ca4036720 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/1842590177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a6cdbd0 0x56547a6cdff0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 >> 172.21.15.170:0/1842590177 conn(0x56547a6c29e0 msgr2=0x56547a6c34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 shutdown_connections 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/1842590177 wait complete. 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6b9520 msgr2=0x56547a6bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56547a6b9520 0x56547a6bb910 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a533130 msgr2=0x56547a6b6f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56547a533130 0x56547a6b6f60 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56547a6bd210 msgr2=0x56547a6bf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.651+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56547a6bd210 0x56547a6bf600 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6cb4034350 msgr2=0x7f6cb4036810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6cb4034350 0x7f6cb4036810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8000f70 tx=0x7f6dc8001380 comp rx=0 tx=0).stop 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 msgr2=0x56547a6b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 --2- 172.21.15.170:0/4077269308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56547a531c20 0x56547a6b0330 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6db8005cc0 tx=0x7f6db8017e30 comp rx=0 tx=0).stop 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcdb37640 1 -- 172.21.15.170:0/4077269308 reap_dead start 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 shutdown_connections 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 >> 172.21.15.170:0/4077269308 conn(0x56547a5b8800 msgr2=0x56547a6a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 shutdown_connections 2024-03-28T16:49:17.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.655+0000 7f6dcf81b980 1 -- 172.21.15.170:0/4077269308 wait complete. 2024-03-28T16:49:17.663 INFO:teuthology.orchestra.run.smithi170.stdout:["myfoo"] 2024-03-28T16:49:17.663 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-03-28T16:49:17.676 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.692 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.706 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.751 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-03-28T16:49:17.821 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.890 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.962 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:17.962 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-03-28T16:49:17.962 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-03-28T16:49:17.962 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-03-28T16:49:17.999 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:17.999 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:18.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.999+0000 7f43da3d1980 1 Processor -- start 2024-03-28T16:49:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.999+0000 7f43da3d1980 1 -- start start 2024-03-28T16:49:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.999+0000 7f43da3d1980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55a49bcdf070 0x55a49bcdf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.999+0000 7f43da3d1980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a49bcb2520 con 0x55a49bcdcb70 2024-03-28T16:49:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:17.999+0000 7f43da3d1980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a49bc790f0 con 0x55a49bcdf070 2024-03-28T16:49:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d76eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55a49bcdf070 0x55a49bcdf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d7eec640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55a49bcdcb70 0x55a49bcdcf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32830/0 (socket says 172.21.15.170:32830) 2024-03-28T16:49:18.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d76eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55a49bcdf070 0x55a49bcdf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56278/0 (socket says 172.21.15.170:56278) 2024-03-28T16:49:18.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d7eec640 1 -- 172.21.15.170:0/3440493314 learned_addr learned my addr 172.21.15.170:0/3440493314 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:18.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d76eb640 1 -- 172.21.15.170:0/3440493314 >> v1:172.21.15.119:6789/0 conn(0x55a49bcdcb70 legacy=0x55a49bcdcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d76eb640 1 -- 172.21.15.170:0/3440493314 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49bce09d0 con 0x55a49bcdf070 2024-03-28T16:49:18.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d6eea640 1 -- 172.21.15.170:0/3440493314 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690893189 0 0) 0x55a49bcb2520 con 0x55a49bcdcb70 2024-03-28T16:49:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d76eb640 1 --2- 172.21.15.170:0/3440493314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdf070 0x55a49bcdf480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00a480 tx=0x7f43cc01b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cce5a8bec3a804 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d6eea640 1 -- 172.21.15.170:0/3440493314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43cc0ab540 con 0x55a49bcdf070 2024-03-28T16:49:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d6eea640 1 -- 172.21.15.170:0/3440493314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc0ab6a0 con 0x55a49bcdf070 2024-03-28T16:49:18.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.003+0000 7f43d6eea640 1 -- 172.21.15.170:0/3440493314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43cc0ab960 con 0x55a49bcdf070 2024-03-28T16:49:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.007+0000 7f43da3d1980 1 -- 172.21.15.170:0/3440493314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdf070 msgr2=0x55a49bcdf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.007+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3440493314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdf070 0x55a49bcdf480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00a480 tx=0x7f43cc01b280 comp rx=0 tx=0).stop 2024-03-28T16:49:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.007+0000 7f43da3d1980 1 -- 172.21.15.170:0/3440493314 shutdown_connections 2024-03-28T16:49:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.007+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3440493314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdf070 0x55a49bcdf480 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.007+0000 7f43da3d1980 1 -- 172.21.15.170:0/3440493314 >> 172.21.15.170:0/3440493314 conn(0x55a49bcdb850 msgr2=0x55a49bcdbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.007+0000 7f43da3d1980 1 -- 172.21.15.170:0/3440493314 shutdown_connections 2024-03-28T16:49:18.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.007+0000 7f43da3d1980 1 -- 172.21.15.170:0/3440493314 wait complete. 2024-03-28T16:49:18.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 Processor -- start 2024-03-28T16:49:18.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- start start 2024-03-28T16:49:18.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be56260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49bc76620 con 0x55a49bcdcb70 2024-03-28T16:49:18.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be56260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be56260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56294/0 (socket says 172.21.15.170:56294) 2024-03-28T16:49:18.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43d7eec640 1 -- 172.21.15.170:0/3271942555 learned_addr learned my addr 172.21.15.170:0/3271942555 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43d7eec640 1 -- 172.21.15.170:0/3271942555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc005d30 con 0x55a49bcdcb70 2024-03-28T16:49:18.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43d7eec640 1 --2- 172.21.15.170:0/3271942555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be56260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f43c80016c0 tx=0x7f43c8001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5921f0822293dc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f42c5e0b640 1 -- 172.21.15.170:0/3271942555 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c8003460 con 0x55a49bcdcb70 2024-03-28T16:49:18.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f42c5e0b640 1 -- 172.21.15.170:0/3271942555 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c80035c0 con 0x55a49bcdcb70 2024-03-28T16:49:18.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f42c5e0b640 1 -- 172.21.15.170:0/3271942555 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c80038a0 con 0x55a49bcdcb70 2024-03-28T16:49:18.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- 172.21.15.170:0/3271942555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 msgr2=0x55a49be56260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3271942555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be56260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f43c80016c0 tx=0x7f43c8001b80 comp rx=0 tx=0).stop 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- 172.21.15.170:0/3271942555 shutdown_connections 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3271942555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be56260 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- 172.21.15.170:0/3271942555 >> 172.21.15.170:0/3271942555 conn(0x55a49bcdb850 msgr2=0x55a49be51df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- 172.21.15.170:0/3271942555 shutdown_connections 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- 172.21.15.170:0/3271942555 wait complete. 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 Processor -- start 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.023+0000 7f43da3d1980 1 -- start start 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43da3d1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be5d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43da3d1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49bc71e00 con 0x55a49bcdcb70 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be5d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be5d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56310/0 (socket says 172.21.15.170:56310) 2024-03-28T16:49:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444319440 learned_addr learned my addr 172.21.15.170:0/2444319440 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c8006cf0 con 0x55a49bcdcb70 2024-03-28T16:49:18.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d7eec640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be5d140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f43c80012f0 tx=0x7f43c8001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f42bffff640 1 -- 172.21.15.170:0/2444319440 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c8013630 con 0x55a49bcdcb70 2024-03-28T16:49:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f42bffff640 1 -- 172.21.15.170:0/2444319440 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8013790 con 0x55a49bcdcb70 2024-03-28T16:49:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a49be5d680 con 0x55a49bcdcb70 2024-03-28T16:49:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f42bffff640 1 -- 172.21.15.170:0/2444319440 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c8013a30 con 0x55a49bcdcb70 2024-03-28T16:49:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a49be5dab0 con 0x55a49bcdcb70 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f42bffff640 1 -- 172.21.15.170:0/2444319440 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f43c8013b90 con 0x55a49bcdcb70 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f42bffff640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42c00342b0 0x7f42c0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f42bffff640 1 -- 172.21.15.170:0/2444319440 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f43c80493c0 con 0x55a49bcdcb70 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42c00342b0 0x7f42c0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49bcdf070 0x55a49be63d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be642b0 con 0x55a49bcdf070 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d86ed640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49bcdf070 0x55a49be63d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42c00342b0 0x7f42c0036770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ad000 tx=0x7f43cc0a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.027+0000 7f43d86ed640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49bcdf070 0x55a49be63d70 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444319440 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49bcdf070 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be64610 con 0x55a49bcdf070 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444319440 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49bcdf070 2024-03-28T16:49:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be66330 0x55a49be68720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be68c60 con 0x55a49be66330 2024-03-28T16:49:18.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d7eec640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be66330 0x55a49be68720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d7eec640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be66330 0x55a49be68720 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444319440 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be66330 2024-03-28T16:49:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be651a0 con 0x55a49bcdf070 2024-03-28T16:49:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444319440 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49bcdf070 2024-03-28T16:49:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a49be6a020 0x55a49be6c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be6ca70 con 0x55a49be6a020 2024-03-28T16:49:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a49be6a020 0x55a49be6c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a49be6a020 0x55a49be6c410 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.031+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444319440 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49be6a020 2024-03-28T16:49:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 Processor -- start 2024-03-28T16:49:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 -- start start 2024-03-28T16:49:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be7a9a0 0x55a49be7adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49bc0ae30 con 0x55a49be7a9a0 2024-03-28T16:49:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be7a9a0 0x55a49be7adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be7a9a0 0x55a49be7adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56316/0 (socket says 172.21.15.170:56316) 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43d7eec640 1 -- 172.21.15.170:0/3740846133 learned_addr learned my addr 172.21.15.170:0/3740846133 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43d7eec640 1 -- 172.21.15.170:0/3740846133 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49be7b300 con 0x55a49be7a9a0 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43d7eec640 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be7a9a0 0x55a49be7adc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f43c8011480 tx=0x7f43c8047e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f42bd7fa640 1 -- 172.21.15.170:0/3740846133 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c8039070 con 0x55a49be7a9a0 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a49be7b550 con 0x55a49be7a9a0 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f42bd7fa640 1 -- 172.21.15.170:0/3740846133 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8011e90 con 0x55a49be7a9a0 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f42bd7fa640 1 -- 172.21.15.170:0/3740846133 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c80118f0 con 0x55a49be7a9a0 2024-03-28T16:49:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a49be7c0a0 con 0x55a49be7a9a0 2024-03-28T16:49:18.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f42bd7fa640 1 -- 172.21.15.170:0/3740846133 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f43c8041990 con 0x55a49be7a9a0 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f42bd7fa640 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42b00342b0 0x7f42b0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f42bd7fa640 1 -- 172.21.15.170:0/3740846133 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f43c803fb20 con 0x55a49be7a9a0 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43d76eb640 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42b00342b0 0x7f42b0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 Processor -- start 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43da3d1980 1 -- start start 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.035+0000 7f43d76eb640 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42b00342b0 0x7f42b0036770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a49be617e0 tx=0x7f43cc005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be87b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49bbfe790 con 0x55a49be87730 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be87b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be87b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56318/0 (socket says 172.21.15.170:56318) 2024-03-28T16:49:18.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 -- 172.21.15.170:0/110500702 learned_addr learned my addr 172.21.15.170:0/110500702 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 -- 172.21.15.170:0/110500702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49be88120 con 0x55a49be87730 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 --2- 172.21.15.170:0/110500702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be87b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f43c801dfd0 tx=0x7f43c8032530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4435c3933aa00e0e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f42b6ffd640 1 -- 172.21.15.170:0/110500702 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c8034e20 con 0x55a49be87730 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f42b6ffd640 1 -- 172.21.15.170:0/110500702 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8028460 con 0x55a49be87730 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f42b6ffd640 1 -- 172.21.15.170:0/110500702 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c801d950 con 0x55a49be87730 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- 172.21.15.170:0/110500702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 msgr2=0x55a49be87b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 --2- 172.21.15.170:0/110500702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be87b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f43c801dfd0 tx=0x7f43c8032530 comp rx=0 tx=0).stop 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- 172.21.15.170:0/110500702 shutdown_connections 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 --2- 172.21.15.170:0/110500702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be87b50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- 172.21.15.170:0/110500702 >> 172.21.15.170:0/110500702 conn(0x55a49be83ac0 msgr2=0x55a49be81b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- 172.21.15.170:0/110500702 shutdown_connections 2024-03-28T16:49:18.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- 172.21.15.170:0/110500702 wait complete. 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 Processor -- start 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- start start 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be8e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43da3d1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49be8eb10 con 0x55a49be87730 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be8e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be8e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56332/0 (socket says 172.21.15.170:56332) 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 learned_addr learned my addr 172.21.15.170:0/2444574388 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49be7b300 con 0x55a49be87730 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.039+0000 7f43d7eec640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be8e5d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f43c8042010 tx=0x7f43c8015ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f42b57fa640 1 -- 172.21.15.170:0/2444574388 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c8065070 con 0x55a49be87730 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f42b57fa640 1 -- 172.21.15.170:0/2444574388 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c80285d0 con 0x55a49be87730 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a49be7b300 con 0x55a49be87730 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f42b57fa640 1 -- 172.21.15.170:0/2444574388 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43c8008b20 con 0x55a49be87730 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a49be7b300 con 0x55a49be87730 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f42b57fa640 1 -- 172.21.15.170:0/2444574388 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f43c805b020 con 0x55a49be87730 2024-03-28T16:49:18.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f42b57fa640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42a0034300 0x7f42a00367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f42b57fa640 1 -- 172.21.15.170:0/2444574388 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f43c8074af0 con 0x55a49be87730 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42a0034300 0x7f42a00367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49be997d0 0x55a49be9bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9c3c0 con 0x55a49be997d0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d86ed640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49be997d0 0x55a49be9bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42a0034300 0x7f42a00367c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ae040 tx=0x7f43cc0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d86ed640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49be997d0 0x55a49be9bc70 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 realm 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9c790 con 0x55a49be997d0 2024-03-28T16:49:18.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9d340 con 0x55a49be997d0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be9dcf0 0x55a49bea00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.043+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea0660 con 0x55a49be9dcf0 2024-03-28T16:49:18.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d7eec640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be9dcf0 0x55a49bea00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d7eec640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be9dcf0 0x55a49bea00e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:18.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43da3d1980 20 searching for the correct realm 2024-03-28T16:49:18.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a49bea2560 0x55a49bea4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea5060 con 0x55a49bea2560 2024-03-28T16:49:18.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a49bea2560 0x55a49bea4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a49bea2560 0x55a49bea4990 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d76eb640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f43cc0cb040 0x7f43cc0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea5060 con 0x7f43cc0cb040 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d86ed640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f43cc0cb040 0x7f43cc0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d86ed640 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f43cc0cb040 0x7f43cc0b6420 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.047+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x7f43cc0cb040 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.051+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x7f43cc0cb040 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.055+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x7f43cc0cb040 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x7f43cc0cb040 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49bea2560 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.059+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be9dcf0 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x55a49be997d0 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49be9cfe0 con 0x7f43cc0cb040 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea0ed0 con 0x55a49be997d0 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea0ed0 con 0x55a49be997d0 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea0ea0 con 0x7f43cc0cb040 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 zone default found 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 4 Realm: () 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 zonegroup default 2024-03-28T16:49:18.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea1720 con 0x55a49be9dcf0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea1b20 con 0x55a49bea2560 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea6f00 con 0x55a49bea2560 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 started sync module instance, tier type = 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:18.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.063+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea6760 con 0x7f43cc0cb040 2024-03-28T16:49:18.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.067+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.067+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94165478826688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea6b10 con 0x7f43cc0cb040 2024-03-28T16:49:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.067+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea7c60 con 0x55a49be997d0 2024-03-28T16:49:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.067+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94165478826688] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.067+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.067+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94165478830880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea7ca0 con 0x55a49be997d0 2024-03-28T16:49:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.067+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea8720 con 0x55a49bea2560 2024-03-28T16:49:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94165478830880] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94165478834720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea8b00 con 0x55a49bea2560 2024-03-28T16:49:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea9470 con 0x7f43cc0cb040 2024-03-28T16:49:18.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94165478838496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea9b40 con 0x7f43cc0cb040 2024-03-28T16:49:18.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beaa5f0 con 0x55a49be997d0 2024-03-28T16:49:18.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94165478834720] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.071+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94165478838496] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.075+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.075+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94165478842608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beaa5b0 con 0x55a49be997d0 2024-03-28T16:49:18.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.075+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea9710 con 0x55a49be9dcf0 2024-03-28T16:49:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.075+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94165478842608] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.075+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.075+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94165478831936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beaae70 con 0x55a49be9dcf0 2024-03-28T16:49:18.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.075+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beab670 con 0x55a49be9dcf0 2024-03-28T16:49:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94165478848976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beac290 con 0x55a49be9dcf0 2024-03-28T16:49:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beaccc0 con 0x55a49be9dcf0 2024-03-28T16:49:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94165478831936] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94165478848976] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94165478852544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beacc80 con 0x55a49be9dcf0 2024-03-28T16:49:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 20 add_watcher() i=0 2024-03-28T16:49:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 20 add_watcher() i=1 2024-03-28T16:49:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 20 add_watcher() i=2 2024-03-28T16:49:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 20 add_watcher() i=3 2024-03-28T16:49:18.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 20 add_watcher() i=4 2024-03-28T16:49:18.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 20 add_watcher() i=5 2024-03-28T16:49:18.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.079+0000 7f43da3d1980 20 add_watcher() i=6 2024-03-28T16:49:18.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94165478852544] v29'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 add_watcher() i=7 2024-03-28T16:49:18.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:18.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beabee0 con 0x55a49be997d0 2024-03-28T16:49:18.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94165478844800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bead4d0 con 0x55a49be997d0 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94165478844800] v29'96 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f428dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 5 note: GC not initialized 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f428cfe1640 20 reqs_thread_entry: start 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 Filter name: none 2024-03-28T16:49:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f42067fc640 20 reqs_thread_entry: start 2024-03-28T16:49:18.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bed1c00 con 0x7f43cc0cb040 2024-03-28T16:49:18.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:18.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.083+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:63 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bed21a0 con 0x55a49bea2560 2024-03-28T16:49:18.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(63 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bed1240 con 0x7f43cc0cb040 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bed3ac0 con 0x55a49bea2560 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(65 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:18.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 4.28 4:1795f074:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bed3540 con 0x55a49be997d0 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 4.22 4:4580d573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bed8610 con 0x55a49be997d0 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 4.2 4:400bbb2d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bedb1e0 con 0x55a49be997d0 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 4.14 4:2bd00f8d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beddee0 con 0x55a49bea2560 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 4.9 4:93976103:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bee0910 con 0x55a49be997d0 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 4.1 4:81996a80:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bee3680 con 0x7f43cc0cb040 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 4.9 4:916ca974:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bedb1e0 con 0x55a49be997d0 2024-03-28T16:49:18.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 4.f 4:f072f27b:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beddee0 con 0x55a49be997d0 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(69 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(71 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(66 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(67 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:74 4.16 4:69af5b3c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beddee0 con 0x55a49bea2560 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(70 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 4.29 4:9412bb76:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bedb1e0 con 0x55a49bea2560 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:76 4.15 4:a8ab0eea:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beddee0 con 0x55a49bea2560 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(73 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(74 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(72 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(75 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(76 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:77 6.14 6:2bc8c1a7:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bedb1e0 con 0x7f43cc0cb040 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(77 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.087+0000 7f43da3d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f428dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94165478844800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bed2f50 con 0x55a49be997d0 2024-03-28T16:49:18.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94165478844800] v29'97 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94165478826688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beaa980 con 0x7f43cc0cb040 2024-03-28T16:49:18.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94165478826688] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f43da3d1980 20 remove_watcher() i=0 2024-03-28T16:49:18.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f43da3d1980 2 removed watcher, disabling cache 2024-03-28T16:49:18.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.091+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94165478830880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea62c0 con 0x55a49be997d0 2024-03-28T16:49:18.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94165478830880] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43da3d1980 20 remove_watcher() i=1 2024-03-28T16:49:18.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94165478834720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea62c0 con 0x55a49bea2560 2024-03-28T16:49:18.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43d76eb640 1 -- 172.21.15.170:0/2444574388 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94165478834720] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43cc0be1d0 con 0x55a49bea2560 2024-03-28T16:49:18.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43da3d1980 20 remove_watcher() i=2 2024-03-28T16:49:18.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94165478838496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea8220 con 0x7f43cc0cb040 2024-03-28T16:49:18.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94165478838496] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x7f43cc0cb040 2024-03-28T16:49:18.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43da3d1980 20 remove_watcher() i=3 2024-03-28T16:49:18.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.095+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94165478842608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea62c0 con 0x55a49be997d0 2024-03-28T16:49:18.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.099+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94165478842608] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43d00083f0 con 0x55a49be997d0 2024-03-28T16:49:18.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.099+0000 7f43da3d1980 20 remove_watcher() i=4 2024-03-28T16:49:18.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.099+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94165478831936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea90e0 con 0x55a49be9dcf0 2024-03-28T16:49:18.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.099+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94165478831936] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.099+0000 7f43da3d1980 20 remove_watcher() i=5 2024-03-28T16:49:18.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.099+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94165478848976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49beaa0f0 con 0x55a49be9dcf0 2024-03-28T16:49:18.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.103+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94165478848976] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.103+0000 7f43da3d1980 20 remove_watcher() i=6 2024-03-28T16:49:18.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.103+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94165478852544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55a49bea90e0 con 0x55a49be9dcf0 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.103+0000 7f43d7eec640 1 -- 172.21.15.170:0/2444574388 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94165478852544] v29'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f43c8024020 con 0x55a49be9dcf0 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.103+0000 7f43da3d1980 20 remove_watcher() i=7 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be9dcf0 msgr2=0x55a49bea00e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be9dcf0 0x55a49bea00e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49be997d0 msgr2=0x55a49be9bc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49be997d0 0x55a49be9bc70 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f43cc0cb040 msgr2=0x7f43cc0b6420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f43cc0cb040 0x7f43cc0b6420 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a49bea2560 msgr2=0x55a49bea4990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55a49bea2560 0x55a49bea4990 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42a0034300 msgr2=0x7f42a00367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42a0034300 0x7f42a00367c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ae040 tx=0x7f43cc0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 msgr2=0x55a49be8e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444574388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be87730 0x55a49be8e5d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f43c8042010 tx=0x7f43c8015ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 reap_dead start 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444574388 reap_dead start 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 shutdown_connections 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 >> 172.21.15.170:0/2444574388 conn(0x55a49be83ac0 msgr2=0x55a49be81120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 shutdown_connections 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444574388 wait complete. 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42b00342b0 msgr2=0x7f42b0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42b00342b0 0x7f42b0036770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a49be617e0 tx=0x7f43cc005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be7a9a0 msgr2=0x55a49be7adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be7a9a0 0x55a49be7adc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f43c8011480 tx=0x7f43c8047e70 comp rx=0 tx=0).stop 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 shutdown_connections 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42b00342b0 0x7f42b0036770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/3740846133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49be7a9a0 0x55a49be7adc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 >> 172.21.15.170:0/3740846133 conn(0x55a49be6f7b0 msgr2=0x55a49be702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 shutdown_connections 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/3740846133 wait complete. 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be66330 msgr2=0x55a49be68720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55a49be66330 0x55a49be68720 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49bcdf070 msgr2=0x55a49be63d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55a49bcdf070 0x55a49be63d70 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a49be6a020 msgr2=0x55a49be6c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55a49be6a020 0x55a49be6c410 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42c00342b0 msgr2=0x7f42c0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f42c00342b0 0x7f42c0036770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0ad000 tx=0x7f43cc0a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 msgr2=0x55a49be5d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 --2- 172.21.15.170:0/2444319440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a49bcdcb70 0x55a49be5d140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f43c80012f0 tx=0x7f43c8001260 comp rx=0 tx=0).stop 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43d86ed640 1 -- 172.21.15.170:0/2444319440 reap_dead start 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 shutdown_connections 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 >> 172.21.15.170:0/2444319440 conn(0x55a49bcdb850 msgr2=0x55a49be50900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 shutdown_connections 2024-03-28T16:49:18.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.107+0000 7f43da3d1980 1 -- 172.21.15.170:0/2444319440 wait complete. 2024-03-28T16:49:18.118 INFO:teuthology.orchestra.run.smithi170.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"c7513231-bef6-43ae-ade6-ce57a95c9845","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","marker":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-03-28T16:49:17.315750Z","creation_time":"2024-03-28T16:49:17.311675Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"notifications":[]} 2024-03-28T16:49:18.118 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'c7513231-bef6-43ae-ade6-ce57a95c9845', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'marker': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-03-28T16:49:17.315750Z', 'creation_time': '2024-03-28T16:49:17.311675Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'notifications': []} 2024-03-28T16:49:18.118 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-03-28T16:49:18.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-03-28T16:49:18.118 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-03-28T16:49:18.193 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:18.194 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:18.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eccf75980 1 Processor -- start 2024-03-28T16:49:18.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eccf75980 1 -- start start 2024-03-28T16:49:18.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eccf75980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d3e0bea360 0x55d3e0bec750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eccf75980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d3e0b33520 con 0x55d3e0be5e00 2024-03-28T16:49:18.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eccf75980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d3e0afa0f0 con 0x55d3e0bea360 2024-03-28T16:49:18.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eca28f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d3e0bea360 0x55d3e0bec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eca28f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d3e0bea360 0x55d3e0bec750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56348/0 (socket says 172.21.15.170:56348) 2024-03-28T16:49:18.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0ecaa90640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55d3e0be5e00 0x55d3e0be61e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32846/0 (socket says 172.21.15.170:32846) 2024-03-28T16:49:18.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eca28f640 1 -- 172.21.15.170:0/979022880 learned_addr learned my addr 172.21.15.170:0/979022880 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eca28f640 1 -- 172.21.15.170:0/979022880 >> v1:172.21.15.119:6789/0 conn(0x55d3e0be5e00 legacy=0x55d3e0be61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.195+0000 7f0eca28f640 1 -- 172.21.15.170:0/979022880 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e0bedca0 con 0x55d3e0bea360 2024-03-28T16:49:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eca28f640 1 --2- 172.21.15.170:0/979022880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0bea360 0x55d3e0bec750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000a5c0 tx=0x7f0ec001b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d10df3f291e76bc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0ec9a8e640 1 -- 172.21.15.170:0/979022880 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ec00aa470 con 0x55d3e0bea360 2024-03-28T16:49:18.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0ec9a8e640 1 -- 172.21.15.170:0/979022880 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec00aa5d0 con 0x55d3e0bea360 2024-03-28T16:49:18.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0ec9a8e640 1 -- 172.21.15.170:0/979022880 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ec00aa890 con 0x55d3e0bea360 2024-03-28T16:49:18.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eccf75980 1 -- 172.21.15.170:0/979022880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0bea360 msgr2=0x55d3e0bec750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eccf75980 1 --2- 172.21.15.170:0/979022880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0bea360 0x55d3e0bec750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000a5c0 tx=0x7f0ec001b340 comp rx=0 tx=0).stop 2024-03-28T16:49:18.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eccf75980 1 -- 172.21.15.170:0/979022880 shutdown_connections 2024-03-28T16:49:18.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eccf75980 1 --2- 172.21.15.170:0/979022880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0bea360 0x55d3e0bec750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eccf75980 1 -- 172.21.15.170:0/979022880 >> 172.21.15.170:0/979022880 conn(0x55d3e0be16d0 msgr2=0x55d3e0be3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eccf75980 1 -- 172.21.15.170:0/979022880 shutdown_connections 2024-03-28T16:49:18.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.199+0000 7f0eccf75980 1 -- 172.21.15.170:0/979022880 wait complete. 2024-03-28T16:49:18.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0eccf75980 1 Processor -- start 2024-03-28T16:49:18.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0eccf75980 1 -- start start 2024-03-28T16:49:18.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0eccf75980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0eccf75980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e0af7620 con 0x55d3e0be5e00 2024-03-28T16:49:18.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56350/0 (socket says 172.21.15.170:56350) 2024-03-28T16:49:18.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0ecaa90640 1 -- 172.21.15.170:0/359700741 learned_addr learned my addr 172.21.15.170:0/359700741 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0ecaa90640 1 -- 172.21.15.170:0/359700741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0005df0 con 0x55d3e0be5e00 2024-03-28T16:49:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/359700741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd0e60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc001d40 tx=0x7f0ebc00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d6b3ca0612b787 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0db8e09640 1 -- 172.21.15.170:0/359700741 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc014070 con 0x55d3e0be5e00 2024-03-28T16:49:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0db8e09640 1 -- 172.21.15.170:0/359700741 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc00f590 con 0x55d3e0be5e00 2024-03-28T16:49:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.215+0000 7f0db8e09640 1 -- 172.21.15.170:0/359700741 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc007040 con 0x55d3e0be5e00 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- 172.21.15.170:0/359700741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 msgr2=0x55d3e0cd0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 --2- 172.21.15.170:0/359700741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd0e60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc001d40 tx=0x7f0ebc00b610 comp rx=0 tx=0).stop 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- 172.21.15.170:0/359700741 shutdown_connections 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 --2- 172.21.15.170:0/359700741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd0e60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- 172.21.15.170:0/359700741 >> 172.21.15.170:0/359700741 conn(0x55d3e0be16d0 msgr2=0x55d3e0b5a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- 172.21.15.170:0/359700741 shutdown_connections 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- 172.21.15.170:0/359700741 wait complete. 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 Processor -- start 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- start start 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e0af2e00 con 0x55d3e0be5e00 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56354/0 (socket says 172.21.15.170:56354) 2024-03-28T16:49:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1639529178 learned_addr learned my addr 172.21.15.170:0/1639529178 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e0b33520 con 0x55d3e0be5e00 2024-03-28T16:49:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd7ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b650 tx=0x7f0ebc000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0db7606640 1 -- 172.21.15.170:0/1639529178 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc014030 con 0x55d3e0be5e00 2024-03-28T16:49:18.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0db7606640 1 -- 172.21.15.170:0/1639529178 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc001530 con 0x55d3e0be5e00 2024-03-28T16:49:18.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3e0cd8220 con 0x55d3e0be5e00 2024-03-28T16:49:18.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0db7606640 1 -- 172.21.15.170:0/1639529178 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc00eb10 con 0x55d3e0be5e00 2024-03-28T16:49:18.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.219+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e0cd8650 con 0x55d3e0be5e00 2024-03-28T16:49:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0db7606640 1 -- 172.21.15.170:0/1639529178 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0ebc00ed00 con 0x55d3e0be5e00 2024-03-28T16:49:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0db7606640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0db00342b0 0x7f0db0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0db7606640 1 -- 172.21.15.170:0/1639529178 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0ebc006070 con 0x55d3e0be5e00 2024-03-28T16:49:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0db00342b0 0x7f0db0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0bea360 0x55d3e0cde960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0cdeea0 con 0x55d3e0bea360 2024-03-28T16:49:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecb291640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0bea360 0x55d3e0cde960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0db00342b0 0x7f0db0036770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00ae000 tx=0x7f0ec00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecb291640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0bea360 0x55d3e0cde960 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecb291640 1 -- 172.21.15.170:0/1639529178 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0bea360 2024-03-28T16:49:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0cdf200 con 0x55d3e0bea360 2024-03-28T16:49:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecb291640 1 -- 172.21.15.170:0/1639529178 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0bea360 2024-03-28T16:49:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0ce0f20 0x55d3e0ce3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0ce3850 con 0x55d3e0ce0f20 2024-03-28T16:49:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0ce0f20 0x55d3e0ce3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0ce0f20 0x55d3e0ce3310 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1639529178 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0ce0f20 2024-03-28T16:49:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0cdfd60 con 0x55d3e0bea360 2024-03-28T16:49:18.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0ecb291640 1 -- 172.21.15.170:0/1639529178 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0bea360 2024-03-28T16:49:18.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.223+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d3e0ce4be0 0x55d3e0ce6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0ce7670 con 0x55d3e0ce4be0 2024-03-28T16:49:18.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d3e0ce4be0 0x55d3e0ce6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d3e0ce4be0 0x55d3e0ce6fd0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eca28f640 1 -- 172.21.15.170:0/1639529178 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0ce4be0 2024-03-28T16:49:18.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eccf75980 1 Processor -- start 2024-03-28T16:49:18.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eccf75980 1 -- start start 2024-03-28T16:49:18.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eccf75980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0cf5580 0x55d3e0cf59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eccf75980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e0a8be30 con 0x55d3e0cf5580 2024-03-28T16:49:18.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0cf5580 0x55d3e0cf59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0cf5580 0x55d3e0cf59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56370/0 (socket says 172.21.15.170:56370) 2024-03-28T16:49:18.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0ecaa90640 1 -- 172.21.15.170:0/2699850911 learned_addr learned my addr 172.21.15.170:0/2699850911 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0ecaa90640 1 -- 172.21.15.170:0/2699850911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e0cf5ee0 con 0x55d3e0cf5580 2024-03-28T16:49:18.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0cf5580 0x55d3e0cf59a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc017820 tx=0x7f0ebc017d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0db4e01640 1 -- 172.21.15.170:0/2699850911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc0417a0 con 0x55d3e0cf5580 2024-03-28T16:49:18.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3e0cf6130 con 0x55d3e0cf5580 2024-03-28T16:49:18.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0db4e01640 1 -- 172.21.15.170:0/2699850911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc041900 con 0x55d3e0cf5580 2024-03-28T16:49:18.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e0cf6c80 con 0x55d3e0cf5580 2024-03-28T16:49:18.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.227+0000 7f0db4e01640 1 -- 172.21.15.170:0/2699850911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc041ba0 con 0x55d3e0cf5580 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0db4e01640 1 -- 172.21.15.170:0/2699850911 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0ebc0412c0 con 0x55d3e0cf5580 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0db4e01640 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0da40342b0 0x7f0da4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0db4e01640 1 -- 172.21.15.170:0/2699850911 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0ebc010070 con 0x55d3e0cf5580 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eca28f640 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0da40342b0 0x7f0da4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 Processor -- start 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- start start 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eca28f640 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0da40342b0 0x7f0da4036770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55d3e0cdc3b0 tx=0x7f0ec0005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e0a7f790 con 0x55d3e0d02490 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d028b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56386/0 (socket says 172.21.15.170:56386) 2024-03-28T16:49:18.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0ecaa90640 1 -- 172.21.15.170:0/992755263 learned_addr learned my addr 172.21.15.170:0/992755263 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0ecaa90640 1 -- 172.21.15.170:0/992755263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e0d02e80 con 0x55d3e0d02490 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/992755263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d028b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc01b7e0 tx=0x7f0ebc04ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb81eab04ebc422 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0dae7fc640 1 -- 172.21.15.170:0/992755263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc02dc30 con 0x55d3e0d02490 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0dae7fc640 1 -- 172.21.15.170:0/992755263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc02dd90 con 0x55d3e0d02490 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0dae7fc640 1 -- 172.21.15.170:0/992755263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc049600 con 0x55d3e0d02490 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- 172.21.15.170:0/992755263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 msgr2=0x55d3e0d028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 --2- 172.21.15.170:0/992755263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d028b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc01b7e0 tx=0x7f0ebc04ac50 comp rx=0 tx=0).stop 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- 172.21.15.170:0/992755263 shutdown_connections 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 --2- 172.21.15.170:0/992755263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d028b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- 172.21.15.170:0/992755263 >> 172.21.15.170:0/992755263 conn(0x55d3e0cfe820 msgr2=0x55d3e0cfc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- 172.21.15.170:0/992755263 shutdown_connections 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- 172.21.15.170:0/992755263 wait complete. 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 Processor -- start 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.231+0000 7f0eccf75980 1 -- start start 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eccf75980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d09310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eccf75980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3e0d09850 con 0x55d3e0d02490 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d09310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0ecaa90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d09310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56390/0 (socket says 172.21.15.170:56390) 2024-03-28T16:49:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 learned_addr learned my addr 172.21.15.170:0/1218115269 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3e0cf6c80 con 0x55d3e0d02490 2024-03-28T16:49:18.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d09310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc04ac90 tx=0x7f0ebc01b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0dacff9640 1 -- 172.21.15.170:0/1218115269 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc065070 con 0x55d3e0d02490 2024-03-28T16:49:18.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0dacff9640 1 -- 172.21.15.170:0/1218115269 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc01f380 con 0x55d3e0d02490 2024-03-28T16:49:18.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3e0d09a50 con 0x55d3e0d02490 2024-03-28T16:49:18.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0dacff9640 1 -- 172.21.15.170:0/1218115269 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ebc01a570 con 0x55d3e0d02490 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3e0d09a50 con 0x55d3e0d02490 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0dacff9640 1 -- 172.21.15.170:0/1218115269 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0ebc01a6d0 con 0x55d3e0d02490 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0dacff9640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0d940342b0 0x7f0d94036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0dacff9640 1 -- 172.21.15.170:0/1218115269 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0ebc06aac0 con 0x55d3e0d02490 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0d940342b0 0x7f0d94036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0d14720 0x55d3e0d16bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17310 con 0x55d3e0d14720 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0ecb291640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0d14720 0x55d3e0d16bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0d940342b0 0x7f0d94036770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000c10 tx=0x7f0ec00c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.235+0000 7f0ecb291640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0d14720 0x55d3e0d16bc0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 realm 2024-03-28T16:49:18.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d176e0 con 0x55d3e0d14720 2024-03-28T16:49:18.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d182c0 con 0x55d3e0d14720 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0d18c70 0x55d3e0d1b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1b5a0 con 0x55d3e0d18c70 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0d18c70 0x55d3e0d1b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0ecaa90640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0d18c70 0x55d3e0d1b060 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 20 searching for the correct realm 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d3e0d1d430 0x55d3e0d1f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1ffc0 con 0x55d3e0d1d430 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d3e0d1d430 0x55d3e0d1f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.239+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d3e0d1d430 0x55d3e0d1f8f0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0eca28f640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0ec00e0040 0x7f0ec00d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1ffc0 con 0x7f0ec00e0040 2024-03-28T16:49:18.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0ec00e0040 0x7f0ec00d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0ec00e0040 0x7f0ec00d3400 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x7f0ec00e0040 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.243+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x7f0ec00e0040 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.247+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x7f0ec00e0040 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x7f0ec00e0040 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.251+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d1d430 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d18c70 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x55d3e0d14720 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d17f60 con 0x7f0ec00e0040 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1bda0 con 0x55d3e0d14720 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1bda0 con 0x55d3e0d14720 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1bda0 con 0x7f0ec00e0040 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 zone default found 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 4 Realm: () 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 zonegroup default 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1c5f0 con 0x55d3e0d18c70 2024-03-28T16:49:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d1c9f0 con 0x55d3e0d1d430 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.255+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d21ef0 con 0x55d3e0d1d430 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 20 started sync module instance, tier type = 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d21710 con 0x7f0ec00e0040 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94368498324080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d21ac0 con 0x7f0ec00e0040 2024-03-28T16:49:18.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d22c50 con 0x55d3e0d14720 2024-03-28T16:49:18.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94368498324080] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94368498328336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d22c50 con 0x55d3e0d14720 2024-03-28T16:49:18.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.259+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d236f0 con 0x55d3e0d1d430 2024-03-28T16:49:18.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94368498328336] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94368498329392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d23aa0 con 0x55d3e0d1d430 2024-03-28T16:49:18.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d24410 con 0x7f0ec00e0040 2024-03-28T16:49:18.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94368498335872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d24b20 con 0x7f0ec00e0040 2024-03-28T16:49:18.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d255d0 con 0x55d3e0d14720 2024-03-28T16:49:18.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.263+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94368498329392] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94368498335872] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94368498340048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d25590 con 0x55d3e0d14720 2024-03-28T16:49:18.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d259b0 con 0x55d3e0d18c70 2024-03-28T16:49:18.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94368498340048] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94368498332112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d25d60 con 0x55d3e0d18c70 2024-03-28T16:49:18.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.267+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d266d0 con 0x55d3e0d18c70 2024-03-28T16:49:18.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94368498346544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d272f0 con 0x55d3e0d18c70 2024-03-28T16:49:18.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d27ca0 con 0x55d3e0d18c70 2024-03-28T16:49:18.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94368498332112] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94368498346544] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94368498347600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d277c0 con 0x55d3e0d18c70 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 20 add_watcher() i=0 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 20 add_watcher() i=1 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 20 add_watcher() i=2 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 20 add_watcher() i=3 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 20 add_watcher() i=4 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 20 add_watcher() i=5 2024-03-28T16:49:18.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.271+0000 7f0eccf75980 20 add_watcher() i=6 2024-03-28T16:49:18.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94368498347600] v29'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 20 add_watcher() i=7 2024-03-28T16:49:18.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:18.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d2ae90 con 0x55d3e0d14720 2024-03-28T16:49:18.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94368498337152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d28210 con 0x55d3e0d14720 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94368498337152] v29'98 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0d81fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 5 note: GC not initialized 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0d80fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 20 Filter name: none 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0d79ffb640 20 reqs_thread_entry: start 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d4c930 con 0x55d3e0d14720 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.275+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d4c930 con 0x7f0ec00e0040 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv16 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d4b860 con 0x7f0ec00e0040 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:18.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d4bd00 con 0x55d3e0d1d430 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(65 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 4.28 4:1795f074:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d4f990 con 0x55d3e0d14720 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 4.22 4:4580d573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d52440 con 0x55d3e0d14720 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 4.2 4:400bbb2d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d54fe0 con 0x55d3e0d14720 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 4.14 4:2bd00f8d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d57920 con 0x55d3e0d1d430 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 4.9 4:93976103:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d54fe0 con 0x55d3e0d14720 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 4.1 4:81996a80:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d5cc30 con 0x7f0ec00e0040 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 4.9 4:916ca974:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d5f4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 4.f 4:f072f27b:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d61f50 con 0x55d3e0d14720 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:74 4.16 4:69af5b3c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d64770 con 0x55d3e0d1d430 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 4.29 4:9412bb76:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d67090 con 0x55d3e0d1d430 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:76 4.15 4:a8ab0eea:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d69f00 con 0x55d3e0d1d430 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(66 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(69 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(71 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(67 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(70 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(73 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(68 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(74 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(75 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(76 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(72 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:77 6.14 6:2bc8c1a7:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d69e80 con 0x7f0ec00e0040 2024-03-28T16:49:18.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(77 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0d81fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.279+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94368498337152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d2a730 con 0x55d3e0d14720 2024-03-28T16:49:18.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.283+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94368498337152] v29'99 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.283+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94368498324080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d2a730 con 0x7f0ec00e0040 2024-03-28T16:49:18.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.283+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94368498324080] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.283+0000 7f0eccf75980 20 remove_watcher() i=0 2024-03-28T16:49:18.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.283+0000 7f0eccf75980 2 removed watcher, disabling cache 2024-03-28T16:49:18.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.283+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94368498328336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d2a730 con 0x55d3e0d14720 2024-03-28T16:49:18.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94368498328336] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0eccf75980 20 remove_watcher() i=1 2024-03-28T16:49:18.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94368498329392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d22310 con 0x55d3e0d1d430 2024-03-28T16:49:18.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0eca28f640 1 -- 172.21.15.170:0/1218115269 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94368498329392] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec00b0020 con 0x55d3e0d1d430 2024-03-28T16:49:18.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0eccf75980 20 remove_watcher() i=2 2024-03-28T16:49:18.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94368498335872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d2a730 con 0x7f0ec00e0040 2024-03-28T16:49:18.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94368498335872] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x7f0ec00e0040 2024-03-28T16:49:18.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0eccf75980 20 remove_watcher() i=3 2024-03-28T16:49:18.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.287+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94368498340048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d2a730 con 0x55d3e0d14720 2024-03-28T16:49:18.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.291+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94368498340048] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ec400a4d0 con 0x55d3e0d14720 2024-03-28T16:49:18.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.291+0000 7f0eccf75980 20 remove_watcher() i=4 2024-03-28T16:49:18.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.291+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94368498332112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d250d0 con 0x55d3e0d18c70 2024-03-28T16:49:18.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.291+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94368498332112] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.291+0000 7f0eccf75980 20 remove_watcher() i=5 2024-03-28T16:49:18.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.291+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94368498346544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d2a730 con 0x55d3e0d18c70 2024-03-28T16:49:18.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94368498346544] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 20 remove_watcher() i=6 2024-03-28T16:49:18.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94368498347600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d3e0d26a30 con 0x55d3e0d18c70 2024-03-28T16:49:18.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0ecaa90640 1 -- 172.21.15.170:0/1218115269 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94368498347600] v29'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0ebc016070 con 0x55d3e0d18c70 2024-03-28T16:49:18.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 20 remove_watcher() i=7 2024-03-28T16:49:18.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0d18c70 msgr2=0x55d3e0d1b060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0d18c70 0x55d3e0d1b060 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0d14720 msgr2=0x55d3e0d16bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0d14720 0x55d3e0d16bc0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0ec00e0040 msgr2=0x7f0ec00d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0ec00e0040 0x7f0ec00d3400 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d3e0d1d430 msgr2=0x55d3e0d1f8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.295+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d3e0d1d430 0x55d3e0d1f8f0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0d940342b0 msgr2=0x7f0d94036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0d940342b0 0x7f0d94036770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000c10 tx=0x7f0ec00c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 msgr2=0x55d3e0d09310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1218115269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0d02490 0x55d3e0d09310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc04ac90 tx=0x7f0ebc01b4e0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 reap_dead start 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0ecb291640 1 -- 172.21.15.170:0/1218115269 reap_dead start 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 shutdown_connections 2024-03-28T16:49:18.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 >> 172.21.15.170:0/1218115269 conn(0x55d3e0cfe820 msgr2=0x55d3e0cfbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 shutdown_connections 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1218115269 wait complete. 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0da40342b0 msgr2=0x7f0da4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0da40342b0 0x7f0da4036770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55d3e0cdc3b0 tx=0x7f0ec0005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0cf5580 msgr2=0x55d3e0cf59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0cf5580 0x55d3e0cf59a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc017820 tx=0x7f0ebc017d70 comp rx=0 tx=0).stop 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 shutdown_connections 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0da40342b0 0x7f0da4036770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/2699850911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0cf5580 0x55d3e0cf59a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 >> 172.21.15.170:0/2699850911 conn(0x55d3e0cea390 msgr2=0x55d3e0ceaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 shutdown_connections 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/2699850911 wait complete. 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0ce0f20 msgr2=0x55d3e0ce3310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d3e0ce0f20 0x55d3e0ce3310 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0bea360 msgr2=0x55d3e0cde960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d3e0bea360 0x55d3e0cde960 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d3e0ce4be0 msgr2=0x55d3e0ce6fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d3e0ce4be0 0x55d3e0ce6fd0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0db00342b0 msgr2=0x7f0db0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0db00342b0 0x7f0db0036770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00ae000 tx=0x7f0ec00a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 msgr2=0x55d3e0cd7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 --2- 172.21.15.170:0/1639529178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d3e0be5e00 0x55d3e0cd7ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b650 tx=0x7f0ebc000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0ecb291640 1 -- 172.21.15.170:0/1639529178 reap_dead start 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 shutdown_connections 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 >> 172.21.15.170:0/1639529178 conn(0x55d3e0be16d0 msgr2=0x55d3e0cca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 shutdown_connections 2024-03-28T16:49:18.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.299+0000 7f0eccf75980 1 -- 172.21.15.170:0/1639529178 wait complete. 2024-03-28T16:49:18.310 INFO:teuthology.orchestra.run.smithi170.stdout:[{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"c7513231-bef6-43ae-ade6-ce57a95c9845","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","marker":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-03-28T16:49:17.315750Z","creation_time":"2024-03-28T16:49:17.311675Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"notifications":[]}] 2024-03-28T16:49:18.310 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'c7513231-bef6-43ae-ade6-ce57a95c9845', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'marker': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-03-28T16:49:17.315750Z', 'creation_time': '2024-03-28T16:49:17.311675Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'notifications': []}] 2024-03-28T16:49:18.318 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-03-28T16:49:18.318 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-03-28T16:49:18.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-03-28T16:49:18.318 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-03-28T16:49:18.384 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:18.385 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:18.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1ce4dd980 1 Processor -- start 2024-03-28T16:49:18.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1ce4dd980 1 -- start start 2024-03-28T16:49:18.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1ce4dd980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560ee8f60350 0x560ee8f62740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1ce4dd980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560ee8ea9520 con 0x560ee8f5be40 2024-03-28T16:49:18.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1ce4dd980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x560ee8e700f0 con 0x560ee8f60350 2024-03-28T16:49:18.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1cb7f7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560ee8f60350 0x560ee8f62740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1cb7f7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560ee8f60350 0x560ee8f62740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56406/0 (socket says 172.21.15.170:56406) 2024-03-28T16:49:18.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1cbff8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x560ee8f5be40 0x560ee8f5c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32858/0 (socket says 172.21.15.170:32858) 2024-03-28T16:49:18.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/1767730518 learned_addr learned my addr 172.21.15.170:0/1767730518 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/1767730518 >> v1:172.21.15.119:6789/0 conn(0x560ee8f5be40 legacy=0x560ee8f5c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.387+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/1767730518 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ee8f63c90 con 0x560ee8f60350 2024-03-28T16:49:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/1767730518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f60350 0x560ee8f62740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400a5c0 tx=0x7fe1b401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ded349b08bdc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1caff6640 1 -- 172.21.15.170:0/1767730518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1b40aa470 con 0x560ee8f60350 2024-03-28T16:49:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1caff6640 1 -- 172.21.15.170:0/1767730518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b40aa5d0 con 0x560ee8f60350 2024-03-28T16:49:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1caff6640 1 -- 172.21.15.170:0/1767730518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1b40aa890 con 0x560ee8f60350 2024-03-28T16:49:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/1767730518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f60350 msgr2=0x560ee8f62740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/1767730518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f60350 0x560ee8f62740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400a5c0 tx=0x7fe1b401b340 comp rx=0 tx=0).stop 2024-03-28T16:49:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/1767730518 shutdown_connections 2024-03-28T16:49:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/1767730518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f60350 0x560ee8f62740 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/1767730518 >> 172.21.15.170:0/1767730518 conn(0x560ee8f57710 msgr2=0x560ee8f59b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/1767730518 shutdown_connections 2024-03-28T16:49:18.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.391+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/1767730518 wait complete. 2024-03-28T16:49:18.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 Processor -- start 2024-03-28T16:49:18.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- start start 2024-03-28T16:49:18.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee9046e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee8e6d620 con 0x560ee8f5be40 2024-03-28T16:49:18.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee9046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee9046e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56422/0 (socket says 172.21.15.170:56422) 2024-03-28T16:49:18.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1cbff8640 1 -- 172.21.15.170:0/226133375 learned_addr learned my addr 172.21.15.170:0/226133375 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1cbff8640 1 -- 172.21.15.170:0/226133375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4005df0 con 0x560ee8f5be40 2024-03-28T16:49:18.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/226133375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee9046e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc006fc0 tx=0x7fe1bc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2652109512d9d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe0b9e0b640 1 -- 172.21.15.170:0/226133375 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc013420 con 0x560ee8f5be40 2024-03-28T16:49:18.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe0b9e0b640 1 -- 172.21.15.170:0/226133375 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc013580 con 0x560ee8f5be40 2024-03-28T16:49:18.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe0b9e0b640 1 -- 172.21.15.170:0/226133375 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc013880 con 0x560ee8f5be40 2024-03-28T16:49:18.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/226133375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 msgr2=0x560ee9046e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/226133375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee9046e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc006fc0 tx=0x7fe1bc0019b0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/226133375 shutdown_connections 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/226133375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee9046e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/226133375 >> 172.21.15.170:0/226133375 conn(0x560ee8f57710 msgr2=0x560ee8ed1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/226133375 shutdown_connections 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/226133375 wait complete. 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 Processor -- start 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 -- start start 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.407+0000 7fe1ce4dd980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee904dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1ce4dd980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee8e68e00 con 0x560ee8f5be40 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee904dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee904dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56428/0 (socket says 172.21.15.170:56428) 2024-03-28T16:49:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cbff8640 1 -- 172.21.15.170:0/57335181 learned_addr learned my addr 172.21.15.170:0/57335181 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cbff8640 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ee8ea9520 con 0x560ee8f5be40 2024-03-28T16:49:18.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee904dc50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0012f0 tx=0x7fe1bc001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe0b3fff640 1 -- 172.21.15.170:0/57335181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc006320 con 0x560ee8f5be40 2024-03-28T16:49:18.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560ee904e190 con 0x560ee8f5be40 2024-03-28T16:49:18.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe0b3fff640 1 -- 172.21.15.170:0/57335181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc006840 con 0x560ee8f5be40 2024-03-28T16:49:18.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe0b3fff640 1 -- 172.21.15.170:0/57335181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc004c90 con 0x560ee8f5be40 2024-03-28T16:49:18.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ee904e5c0 con 0x560ee8f5be40 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe0b3fff640 1 -- 172.21.15.170:0/57335181 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe1bc01e3f0 con 0x560ee8f5be40 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe0b3fff640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b40342b0 0x7fe0b4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe0b3fff640 1 -- 172.21.15.170:0/57335181 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe1bc049680 con 0x560ee8f5be40 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b40342b0 0x7fe0b4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee8f60350 0x560ee90548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9054e10 con 0x560ee8f60350 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cc7f9640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee8f60350 0x560ee90548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b40342b0 0x7fe0b4036770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40ae000 tx=0x7fe1b40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.411+0000 7fe1cc7f9640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee8f60350 0x560ee90548d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/57335181 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee8f60350 2024-03-28T16:49:18.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9055170 con 0x560ee8f60350 2024-03-28T16:49:18.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/57335181 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee8f60350 2024-03-28T16:49:18.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9056e90 0x560ee9059280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90597c0 con 0x560ee9056e90 2024-03-28T16:49:18.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9056e90 0x560ee9059280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9056e90 0x560ee9059280 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cbff8640 1 -- 172.21.15.170:0/57335181 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9056e90 2024-03-28T16:49:18.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9055d00 con 0x560ee8f60350 2024-03-28T16:49:18.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/57335181 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee8f60350 2024-03-28T16:49:18.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560ee905ab60 0x560ee905cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee905d5f0 con 0x560ee905ab60 2024-03-28T16:49:18.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560ee905ab60 0x560ee905cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560ee905ab60 0x560ee905cf50 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/57335181 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee905ab60 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 Processor -- start 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.415+0000 7fe1ce4dd980 1 -- start start 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee906b520 0x560ee906b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee8e01e30 con 0x560ee906b520 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee906b520 0x560ee906b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee906b520 0x560ee906b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56442/0 (socket says 172.21.15.170:56442) 2024-03-28T16:49:18.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1cbff8640 1 -- 172.21.15.170:0/2340212492 learned_addr learned my addr 172.21.15.170:0/2340212492 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1cbff8640 1 -- 172.21.15.170:0/2340212492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ee906be80 con 0x560ee906b520 2024-03-28T16:49:18.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee906b520 0x560ee906b940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc047c00 tx=0x7fe1bc0269e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe0b17fa640 1 -- 172.21.15.170:0/2340212492 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc01fe80 con 0x560ee906b520 2024-03-28T16:49:18.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560ee906c0d0 con 0x560ee906b520 2024-03-28T16:49:18.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ee906cc20 con 0x560ee906b520 2024-03-28T16:49:18.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe0b17fa640 1 -- 172.21.15.170:0/2340212492 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc041260 con 0x560ee906b520 2024-03-28T16:49:18.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe0b17fa640 1 -- 172.21.15.170:0/2340212492 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc010490 con 0x560ee906b520 2024-03-28T16:49:18.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe0b17fa640 1 -- 172.21.15.170:0/2340212492 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe1bc010c80 con 0x560ee906b520 2024-03-28T16:49:18.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe0b17fa640 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0a4034300 0x7fe0a40367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe0b17fa640 1 -- 172.21.15.170:0/2340212492 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe1bc03cab0 con 0x560ee906b520 2024-03-28T16:49:18.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0a4034300 0x7fe0a40367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 Processor -- start 2024-03-28T16:49:18.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 -- start start 2024-03-28T16:49:18.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0a4034300 0x7fe0a40367c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x560ee9052360 tx=0x7fe1b4005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9042ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.419+0000 7fe1ce4dd980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee8df5790 con 0x560ee9042ac0 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9042ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9042ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56444/0 (socket says 172.21.15.170:56444) 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 -- 172.21.15.170:0/3722156879 learned_addr learned my addr 172.21.15.170:0/3722156879 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 -- 172.21.15.170:0/3722156879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ee9041350 con 0x560ee9042ac0 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/3722156879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9042ee0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc030d20 tx=0x7fe1bc030d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d8fd93d87d7aed server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe0aaffd640 1 -- 172.21.15.170:0/3722156879 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc03cd10 con 0x560ee9042ac0 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe0aaffd640 1 -- 172.21.15.170:0/3722156879 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc03ce70 con 0x560ee9042ac0 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe0aaffd640 1 -- 172.21.15.170:0/3722156879 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc01eba0 con 0x560ee9042ac0 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/3722156879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 msgr2=0x560ee9042ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/3722156879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9042ee0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc030d20 tx=0x7fe1bc030d50 comp rx=0 tx=0).stop 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/3722156879 shutdown_connections 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/3722156879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9042ee0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/3722156879 >> 172.21.15.170:0/3722156879 conn(0x560ee9074800 msgr2=0x560ee9072860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/3722156879 shutdown_connections 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/3722156879 wait complete. 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 Processor -- start 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 -- start start 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9081540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1ce4dd980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee9081a80 con 0x560ee9042ac0 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9081540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9081540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56456/0 (socket says 172.21.15.170:56456) 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 learned_addr learned my addr 172.21.15.170:0/88540994 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ee906cc20 con 0x560ee9042ac0 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.423+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9081540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc004710 tx=0x7fe1bc02afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe0a97fa640 1 -- 172.21.15.170:0/88540994 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc064070 con 0x560ee9042ac0 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe0a97fa640 1 -- 172.21.15.170:0/88540994 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc040e60 con 0x560ee9042ac0 2024-03-28T16:49:18.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560ee906cc20 con 0x560ee9042ac0 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe0a97fa640 1 -- 172.21.15.170:0/88540994 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1bc049bb0 con 0x560ee9042ac0 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ee906cc20 con 0x560ee9042ac0 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe0a97fa640 1 -- 172.21.15.170:0/88540994 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe1bc049d30 con 0x560ee9042ac0 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe0a97fa640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0940342b0 0x7fe094036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe0a97fa640 1 -- 172.21.15.170:0/88540994 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fe1bc05eb20 con 0x560ee9042ac0 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0940342b0 0x7fe094036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee908c730 0x560ee908ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908f340 con 0x560ee908c730 2024-03-28T16:49:18.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cc7f9640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee908c730 0x560ee908ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0940342b0 0x7fe094036770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c10 tx=0x7fe1b40c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cc7f9640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee908c730 0x560ee908ebf0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 realm 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908f710 con 0x560ee908c730 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90902f0 con 0x560ee908c730 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9090ca0 0x560ee9093090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90935d0 con 0x560ee9090ca0 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.427+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9090ca0 0x560ee9093090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cbff8640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9090ca0 0x560ee9093090 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1ce4dd980 20 searching for the correct realm 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560ee90954d0 0x560ee9097960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9098030 con 0x560ee90954d0 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560ee90954d0 0x560ee9097960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560ee90954d0 0x560ee9097960 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cb7f7640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1b40e0040 0x7fe1b40d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9098030 con 0x7fe1b40e0040 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cc7f9640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1b40e0040 0x7fe1b40d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cc7f9640 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1b40e0040 0x7fe1b40d3400 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.431+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x7fe1b40e0040 2024-03-28T16:49:18.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x7fe1b40e0040 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.435+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x7fe1b40e0040 2024-03-28T16:49:18.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.439+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x7fe1b40e0040 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee90954d0 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee9090ca0 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x560ee908c730 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee908ff90 con 0x7fe1b40e0040 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:18.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.443+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9093e40 con 0x560ee908c730 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9093e40 con 0x560ee908c730 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9093e10 con 0x7fe1b40e0040 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 zone default found 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 4 Realm: () 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 zonegroup default 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9094690 con 0x560ee9090ca0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9094a90 con 0x560ee90954d0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9099ef0 con 0x560ee90954d0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 started sync module instance, tier type = 2024-03-28T16:49:18.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:18.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9099730 con 0x7fe1b40e0040 2024-03-28T16:49:18.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94622039249552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9099ae0 con 0x7fe1b40e0040 2024-03-28T16:49:18.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.447+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909ac50 con 0x560ee908c730 2024-03-28T16:49:18.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94622039249552] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94622039253776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909ac90 con 0x560ee908c730 2024-03-28T16:49:18.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909b700 con 0x560ee90954d0 2024-03-28T16:49:18.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94622039253776] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94622039257568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909bab0 con 0x560ee90954d0 2024-03-28T16:49:18.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.451+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909c420 con 0x7fe1b40e0040 2024-03-28T16:49:18.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94622039261328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909cb10 con 0x7fe1b40e0040 2024-03-28T16:49:18.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909d5c0 con 0x560ee908c730 2024-03-28T16:49:18.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94622039257568] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94622039261328] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94622039265472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909d580 con 0x560ee908c730 2024-03-28T16:49:18.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.455+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909d9a0 con 0x560ee9090ca0 2024-03-28T16:49:18.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94622039265472] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94622039254832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909dd50 con 0x560ee9090ca0 2024-03-28T16:49:18.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909e6c0 con 0x560ee9090ca0 2024-03-28T16:49:18.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94622039254832] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94622039262576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909eec0 con 0x560ee9090ca0 2024-03-28T16:49:18.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.459+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909f740 con 0x560ee9090ca0 2024-03-28T16:49:18.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94622039273152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909fde0 con 0x560ee9090ca0 2024-03-28T16:49:18.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=0 2024-03-28T16:49:18.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=1 2024-03-28T16:49:18.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=2 2024-03-28T16:49:18.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=3 2024-03-28T16:49:18.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=4 2024-03-28T16:49:18.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=5 2024-03-28T16:49:18.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94622039262576] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=6 2024-03-28T16:49:18.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94622039273152] v29'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 20 add_watcher() i=7 2024-03-28T16:49:18.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:18.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90a2ce0 con 0x560ee908c730 2024-03-28T16:49:18.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.463+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94622039275536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90a0140 con 0x560ee908c730 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94622039275536] v29'100 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe081fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 5 note: GC not initialized 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe080fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 Filter name: none 2024-03-28T16:49:18.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fdffa7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:18.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90c4bc0 con 0x7fe1b40e0040 2024-03-28T16:49:18.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:63 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90c5110 con 0x560ee90954d0 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(63 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90c4240 con 0x7fe1b40e0040 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90c6a30 con 0x560ee90954d0 2024-03-28T16:49:18.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(65 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 4.28 4:1795f074:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90c6430 con 0x560ee908c730 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 4.22 4:4580d573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90cb440 con 0x560ee908c730 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 4.2 4:400bbb2d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90cdd40 con 0x560ee908c730 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 4.14 4:2bd00f8d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d0650 con 0x560ee90954d0 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 4.9 4:93976103:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90cdd40 con 0x560ee908c730 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 4.1 4:81996a80:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d59f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 4.9 4:916ca974:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d8300 con 0x560ee908c730 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 4.f 4:f072f27b:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90dadd0 con 0x560ee908c730 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(66 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:74 4.16 4:69af5b3c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90dadd0 con 0x560ee90954d0 2024-03-28T16:49:18.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 4.29 4:9412bb76:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d8300 con 0x560ee90954d0 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.467+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:76 4.15 4:a8ab0eea:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90e06b0 con 0x560ee90954d0 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(68 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(67 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(73 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(71 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(70 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(72 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(69 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(74 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(75 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(76 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1ce4dd980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:77 6.14 6:2bc8c1a7:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90e06b0 con 0x7fe1b40e0040 2024-03-28T16:49:18.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(77 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1ce4dd980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe081fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94622039275536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d5970 con 0x560ee908c730 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94622039275536] v29'101 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.471+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94622039249552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d5970 con 0x7fe1b40e0040 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.475+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94622039249552] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.475+0000 7fe1ce4dd980 20 remove_watcher() i=0 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.475+0000 7fe1ce4dd980 2 removed watcher, disabling cache 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.475+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94622039253776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9099290 con 0x560ee908c730 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.475+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94622039253776] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.475+0000 7fe1ce4dd980 20 remove_watcher() i=1 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.475+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94622039257568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d5970 con 0x560ee90954d0 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1cb7f7640 1 -- 172.21.15.170:0/88540994 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94622039257568] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1b40b0020 con 0x560ee90954d0 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1ce4dd980 20 remove_watcher() i=2 2024-03-28T16:49:18.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94622039261328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909a310 con 0x7fe1b40e0040 2024-03-28T16:49:18.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94622039261328] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x7fe1b40e0040 2024-03-28T16:49:18.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1ce4dd980 20 remove_watcher() i=3 2024-03-28T16:49:18.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94622039265472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee9099290 con 0x560ee908c730 2024-03-28T16:49:18.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94622039265472] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1c40083f0 con 0x560ee908c730 2024-03-28T16:49:18.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1ce4dd980 20 remove_watcher() i=4 2024-03-28T16:49:18.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.479+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94622039254832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909b1e0 con 0x560ee9090ca0 2024-03-28T16:49:18.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.483+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94622039254832] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.483+0000 7fe1ce4dd980 20 remove_watcher() i=5 2024-03-28T16:49:18.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.483+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94622039262576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee90d5970 con 0x560ee9090ca0 2024-03-28T16:49:18.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94622039262576] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 20 remove_watcher() i=6 2024-03-28T16:49:18.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94622039273152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x560ee909c090 con 0x560ee9090ca0 2024-03-28T16:49:18.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1cbff8640 1 -- 172.21.15.170:0/88540994 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94622039273152] v29'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fe1bc024070 con 0x560ee9090ca0 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 20 remove_watcher() i=7 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9090ca0 msgr2=0x560ee9093090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9090ca0 0x560ee9093090 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee908c730 msgr2=0x560ee908ebf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee908c730 0x560ee908ebf0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1b40e0040 msgr2=0x7fe1b40d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fe1b40e0040 0x7fe1b40d3400 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560ee90954d0 msgr2=0x560ee9097960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560ee90954d0 0x560ee9097960 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0940342b0 msgr2=0x7fe094036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0940342b0 0x7fe094036770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4000c10 tx=0x7fe1b40c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:18.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/88540994 reap_dead start 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 msgr2=0x560ee9081540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9081540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc004710 tx=0x7fe1bc02afa0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 shutdown_connections 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/88540994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee9042ac0 0x560ee9081540 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 >> 172.21.15.170:0/88540994 conn(0x560ee9074800 msgr2=0x560ee9071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 shutdown_connections 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/88540994 wait complete. 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0a4034300 msgr2=0x7fe0a40367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0a4034300 0x7fe0a40367c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x560ee9052360 tx=0x7fe1b4005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee906b520 msgr2=0x560ee906b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee906b520 0x560ee906b940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc047c00 tx=0x7fe1bc0269e0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 shutdown_connections 2024-03-28T16:49:18.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0a4034300 0x7fe0a40367c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/2340212492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee906b520 0x560ee906b940 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 >> 172.21.15.170:0/2340212492 conn(0x560ee9060330 msgr2=0x560ee9060e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 shutdown_connections 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/2340212492 wait complete. 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9056e90 msgr2=0x560ee9059280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560ee9056e90 0x560ee9059280 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee8f60350 msgr2=0x560ee90548d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560ee8f60350 0x560ee90548d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560ee905ab60 msgr2=0x560ee905cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560ee905ab60 0x560ee905cf50 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b40342b0 msgr2=0x7fe0b4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe0b40342b0 0x7fe0b4036770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40ae000 tx=0x7fe1b40a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:18.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 msgr2=0x560ee904dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 --2- 172.21.15.170:0/57335181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ee8f5be40 0x560ee904dc50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0012f0 tx=0x7fe1bc001260 comp rx=0 tx=0).stop 2024-03-28T16:49:18.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1cc7f9640 1 -- 172.21.15.170:0/57335181 reap_dead start 2024-03-28T16:49:18.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 shutdown_connections 2024-03-28T16:49:18.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 >> 172.21.15.170:0/57335181 conn(0x560ee8f57710 msgr2=0x560ee90403c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 shutdown_connections 2024-03-28T16:49:18.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.487+0000 7fe1ce4dd980 1 -- 172.21.15.170:0/57335181 wait complete. 2024-03-28T16:49:18.500 INFO:teuthology.orchestra.run.smithi170.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"c7513231-bef6-43ae-ade6-ce57a95c9845","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","marker":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-03-28T16:49:17.315750Z","creation_time":"2024-03-28T16:49:17.311675Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":3,"size_actual":4096,"size_utilized":3,"size_kb":1,"size_kb_actual":4,"size_kb_utilized":1,"num_objects":1}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"notifications":[]} 2024-03-28T16:49:18.500 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'c7513231-bef6-43ae-ade6-ce57a95c9845', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'marker': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-03-28T16:49:17.315750Z', 'creation_time': '2024-03-28T16:49:17.311675Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'notifications': []} 2024-03-28T16:49:18.500 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-03-28T16:49:18.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-03-28T16:49:18.501 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-03-28T16:49:18.577 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:18.577 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:18.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c234a8980 1 Processor -- start 2024-03-28T16:49:18.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c234a8980 1 -- start start 2024-03-28T16:49:18.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c234a8980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x561df5e5de10 0x561df5e5e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c234a8980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561df5dab4c0 con 0x561df5e5e730 2024-03-28T16:49:18.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c234a8980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x561df5d720f0 con 0x561df5e5de10 2024-03-28T16:49:18.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c20fc3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x561df5e5de10 0x561df5e5e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c20fc3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x561df5e5de10 0x561df5e5e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56468/0 (socket says 172.21.15.170:56468) 2024-03-28T16:49:18.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c207c2640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x561df5e5e730 0x561df5e5f140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32862/0 (socket says 172.21.15.170:32862) 2024-03-28T16:49:18.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c20fc3640 1 -- 172.21.15.170:0/1548754489 learned_addr learned my addr 172.21.15.170:0/1548754489 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c20fc3640 1 -- 172.21.15.170:0/1548754489 >> v1:172.21.15.119:6789/0 conn(0x561df5e5e730 legacy=0x561df5e5f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.579+0000 7f1c20fc3640 1 -- 172.21.15.170:0/1548754489 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df5e5f970 con 0x561df5e5de10 2024-03-28T16:49:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/1548754489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5e5e1f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000a750 tx=0x7f1c1001d200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103ec0a94ebabfd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c1ffc1640 1 -- 172.21.15.170:0/1548754489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c1000a200 con 0x561df5e5de10 2024-03-28T16:49:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c1ffc1640 1 -- 172.21.15.170:0/1548754489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c100a8050 con 0x561df5e5de10 2024-03-28T16:49:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c1ffc1640 1 -- 172.21.15.170:0/1548754489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c100085a0 con 0x561df5e5de10 2024-03-28T16:49:18.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c234a8980 1 -- 172.21.15.170:0/1548754489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 msgr2=0x561df5e5e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c234a8980 1 --2- 172.21.15.170:0/1548754489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5e5e1f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000a750 tx=0x7f1c1001d200 comp rx=0 tx=0).stop 2024-03-28T16:49:18.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c234a8980 1 -- 172.21.15.170:0/1548754489 shutdown_connections 2024-03-28T16:49:18.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c234a8980 1 --2- 172.21.15.170:0/1548754489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5e5e1f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c234a8980 1 -- 172.21.15.170:0/1548754489 >> 172.21.15.170:0/1548754489 conn(0x561df5e596e0 msgr2=0x561df5e5bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c234a8980 1 -- 172.21.15.170:0/1548754489 shutdown_connections 2024-03-28T16:49:18.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.583+0000 7f1c234a8980 1 -- 172.21.15.170:0/1548754489 wait complete. 2024-03-28T16:49:18.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 Processor -- start 2024-03-28T16:49:18.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- start start 2024-03-28T16:49:18.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f47390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df5d6f620 con 0x561df5e5de10 2024-03-28T16:49:18.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f47390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f47390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56476/0 (socket says 172.21.15.170:56476) 2024-03-28T16:49:18.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2995810724 learned_addr learned my addr 172.21.15.170:0/2995810724 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2995810724 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10005df0 con 0x561df5e5de10 2024-03-28T16:49:18.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2995810724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f47390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c1001ae30 tx=0x7f1c10003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a1e1a405457b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1b0ee0d640 1 -- 172.21.15.170:0/2995810724 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c1000a280 con 0x561df5e5de10 2024-03-28T16:49:18.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1b0ee0d640 1 -- 172.21.15.170:0/2995810724 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1000a3e0 con 0x561df5e5de10 2024-03-28T16:49:18.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1b0ee0d640 1 -- 172.21.15.170:0/2995810724 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c10008710 con 0x561df5e5de10 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- 172.21.15.170:0/2995810724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 msgr2=0x561df5f47390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2995810724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f47390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1c1001ae30 tx=0x7f1c10003030 comp rx=0 tx=0).stop 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- 172.21.15.170:0/2995810724 shutdown_connections 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2995810724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f47390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- 172.21.15.170:0/2995810724 >> 172.21.15.170:0/2995810724 conn(0x561df5e596e0 msgr2=0x561df5f42f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- 172.21.15.170:0/2995810724 shutdown_connections 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- 172.21.15.170:0/2995810724 wait complete. 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 Processor -- start 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.599+0000 7f1c234a8980 1 -- start start 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c234a8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f52530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c234a8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df5d6ae00 con 0x561df5e5de10 2024-03-28T16:49:18.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f52530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56492/0 (socket says 172.21.15.170:56492) 2024-03-28T16:49:18.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2311851609 learned_addr learned my addr 172.21.15.170:0/2311851609 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df5e5f970 con 0x561df5e5de10 2024-03-28T16:49:18.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f52530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c100b8040 tx=0x7f1c10019fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1b0d60a640 1 -- 172.21.15.170:0/2311851609 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c10008ea0 con 0x561df5e5de10 2024-03-28T16:49:18.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561df5f52a70 con 0x561df5e5de10 2024-03-28T16:49:18.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1b0d60a640 1 -- 172.21.15.170:0/2311851609 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c100c2070 con 0x561df5e5de10 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1b0d60a640 1 -- 172.21.15.170:0/2311851609 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c100c7430 con 0x561df5e5de10 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561df5f52ea0 con 0x561df5e5de10 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1b0d60a640 1 -- 172.21.15.170:0/2311851609 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f1c100c4030 con 0x561df5e5de10 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1b0d60a640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1b080342b0 0x7f1b08036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1b0d60a640 1 -- 172.21.15.170:0/2311851609 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1c100e3820 con 0x561df5e5de10 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1b080342b0 0x7f1b08036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5e5e730 0x561df5f59160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f596a0 con 0x561df5e5e730 2024-03-28T16:49:18.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c217c4640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5e5e730 0x561df5f59160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1b080342b0 0x7f1b08036770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1c14001e50 tx=0x7f1c140013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c217c4640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5e5e730 0x561df5f59160 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.603+0000 7f1c217c4640 1 -- 172.21.15.170:0/2311851609 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5e5e730 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f59a00 con 0x561df5e5e730 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c217c4640 1 -- 172.21.15.170:0/2311851609 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5e5e730 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f5b720 0x561df5f5db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f5e050 con 0x561df5f5b720 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f5b720 0x561df5f5db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f5b720 0x561df5f5db10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2311851609 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f5b720 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f5a530 con 0x561df5e5e730 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c217c4640 1 -- 172.21.15.170:0/2311851609 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5e5e730 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561df5f5f3e0 0x561df5f617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f61e20 con 0x561df5f5f3e0 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561df5f5f3e0 0x561df5f617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561df5f5f3e0 0x561df5f617d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c207c2640 1 -- 172.21.15.170:0/2311851609 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f1c1400f700 con 0x561df5f5f3e0 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 Processor -- start 2024-03-28T16:49:18.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 -- start start 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f6fd40 0x561df5f70160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.607+0000 7f1c234a8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df5d03e30 con 0x561df5f6fd40 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f6fd40 0x561df5f70160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f6fd40 0x561df5f70160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56494/0 (socket says 172.21.15.170:56494) 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c20fc3640 1 -- 172.21.15.170:0/4001040552 learned_addr learned my addr 172.21.15.170:0/4001040552 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c20fc3640 1 -- 172.21.15.170:0/4001040552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df5f706a0 con 0x561df5f6fd40 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f6fd40 0x561df5f70160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c100e1da0 tx=0x7f1c100e3470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1b067fc640 1 -- 172.21.15.170:0/4001040552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c100de040 con 0x561df5f6fd40 2024-03-28T16:49:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561df5f708f0 con 0x561df5f6fd40 2024-03-28T16:49:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1b067fc640 1 -- 172.21.15.170:0/4001040552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c100e3c30 con 0x561df5f6fd40 2024-03-28T16:49:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1b067fc640 1 -- 172.21.15.170:0/4001040552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c100c7590 con 0x561df5f6fd40 2024-03-28T16:49:18.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561df5f71440 con 0x561df5f6fd40 2024-03-28T16:49:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1b067fc640 1 -- 172.21.15.170:0/4001040552 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f1c100c7900 con 0x561df5f6fd40 2024-03-28T16:49:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1b067fc640 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1af80342b0 0x7f1af8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1b067fc640 1 -- 172.21.15.170:0/4001040552 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1c100edac0 con 0x561df5f6fd40 2024-03-28T16:49:18.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c207c2640 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1af80342b0 0x7f1af8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c234a8980 1 Processor -- start 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c234a8980 1 -- start start 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c207c2640 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1af80342b0 0x7f1af8036770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x561df5f56bb0 tx=0x7f1c1400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c234a8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f7cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.611+0000 7f1c234a8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df5cf7790 con 0x561df5f7caf0 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f7cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f7cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56502/0 (socket says 172.21.15.170:56502) 2024-03-28T16:49:18.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2882115752 learned_addr learned my addr 172.21.15.170:0/2882115752 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2882115752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df5f7d4e0 con 0x561df5f7caf0 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2882115752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f7cf10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c100c6f80 tx=0x7f1c100ac5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e9acbb66f1948f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1af7fff640 1 -- 172.21.15.170:0/2882115752 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c100c6640 con 0x561df5f7caf0 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1af7fff640 1 -- 172.21.15.170:0/2882115752 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c100e0600 con 0x561df5f7caf0 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1af7fff640 1 -- 172.21.15.170:0/2882115752 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c100e08a0 con 0x561df5f7caf0 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- 172.21.15.170:0/2882115752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 msgr2=0x561df5f7cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2882115752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f7cf10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1c100c6f80 tx=0x7f1c100ac5a0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- 172.21.15.170:0/2882115752 shutdown_connections 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2882115752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f7cf10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- 172.21.15.170:0/2882115752 >> 172.21.15.170:0/2882115752 conn(0x561df5f78e80 msgr2=0x561df5f76ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- 172.21.15.170:0/2882115752 shutdown_connections 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- 172.21.15.170:0/2882115752 wait complete. 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 Processor -- start 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- start start 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f83990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df5f83ed0 con 0x561df5f7caf0 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f83990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f83990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56506/0 (socket says 172.21.15.170:56506) 2024-03-28T16:49:18.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 learned_addr learned my addr 172.21.15.170:0/2371043630 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df5f706a0 con 0x561df5f7caf0 2024-03-28T16:49:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f83990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c100d9fb0 tx=0x7f1c100b3f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1af67fc640 1 -- 172.21.15.170:0/2371043630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c1010d070 con 0x561df5f7caf0 2024-03-28T16:49:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561df5f706a0 con 0x561df5f7caf0 2024-03-28T16:49:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1af67fc640 1 -- 172.21.15.170:0/2371043630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c100d9910 con 0x561df5f7caf0 2024-03-28T16:49:18.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.615+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561df5f842b0 con 0x561df5f7caf0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1af67fc640 1 -- 172.21.15.170:0/2371043630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c100d9bb0 con 0x561df5f7caf0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1af67fc640 1 -- 172.21.15.170:0/2371043630 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f1c100d20c0 con 0x561df5f7caf0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1af67fc640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1aec0342b0 0x7f1aec036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1af67fc640 1 -- 172.21.15.170:0/2371043630 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1c10104070 con 0x561df5f7caf0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1aec0342b0 0x7f1aec036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5f8eae0 0x561df5f90fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f916f0 con 0x561df5f8eae0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c217c4640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5f8eae0 0x561df5f90fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1aec0342b0 0x7f1aec036770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x561df5f621c0 tx=0x7f1c14010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c217c4640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5f8eae0 0x561df5f90fa0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 realm 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f91ac0 con 0x561df5f8eae0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92670 con 0x561df5f8eae0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f93020 0x561df5f95410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f95990 con 0x561df5f93020 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.619+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f93020 0x561df5f95410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c20fc3640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f93020 0x561df5f95410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c234a8980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c234a8980 20 searching for the correct realm 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561df5f97890 0x561df5f99c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9a330 con 0x561df5f97890 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561df5f97890 0x561df5f99c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561df5f97890 0x561df5f99c80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c207c2640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f1c1401fbe0 0x7f1c1403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9a330 con 0x7f1c1401fbe0 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c217c4640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f1c1401fbe0 0x7f1c1403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c217c4640 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f1c1401fbe0 0x7f1c1403a3c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.623+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x7f1c1401fbe0 2024-03-28T16:49:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.627+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x7f1c1401fbe0 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.631+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x7f1c1401fbe0 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x7f1c1401fbe0 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f97890 2024-03-28T16:49:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f93020 2024-03-28T16:49:18.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x561df5f8eae0 2024-03-28T16:49:18.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.635+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f92310 con 0x7f1c1401fbe0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f96200 con 0x561df5f8eae0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f96200 con 0x561df5f8eae0 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f961d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 zone default found 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 4 Realm: () 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 zonegroup default 2024-03-28T16:49:18.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f96a50 con 0x561df5f93020 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f96e60 con 0x561df5f97890 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9c2a0 con 0x561df5f97890 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 started sync module instance, tier type = 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:18.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9ba30 con 0x7f1c1401fbe0 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.639+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94686680823184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9bde0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.643+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9d000 con 0x561df5f8eae0 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.643+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94686680823184] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.643+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.643+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94686680827584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9cfe0 con 0x561df5f8eae0 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.643+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9daa0 con 0x561df5f97890 2024-03-28T16:49:18.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.643+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94686680827584] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94686680831328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9de50 con 0x561df5f97890 2024-03-28T16:49:18.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9e7c0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94686680835120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9f010 con 0x7f1c1401fbe0 2024-03-28T16:49:18.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9f9c0 con 0x561df5f8eae0 2024-03-28T16:49:18.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94686680831328] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94686680835120] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94686680828640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9eaf0 con 0x561df5f8eae0 2024-03-28T16:49:18.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.647+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa0210 con 0x561df5f93020 2024-03-28T16:49:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.651+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94686680828640] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.651+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.651+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94686680841584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9eaf0 con 0x561df5f93020 2024-03-28T16:49:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.651+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa0a10 con 0x561df5f93020 2024-03-28T16:49:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.651+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.651+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94686680845680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa1630 con 0x561df5f93020 2024-03-28T16:49:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa2020 con 0x561df5f93020 2024-03-28T16:49:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94686680841584] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94686680845680] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94686680849152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa11d0 con 0x561df5f93020 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 20 add_watcher() i=0 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 20 add_watcher() i=1 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 20 add_watcher() i=2 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 20 add_watcher() i=3 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 20 add_watcher() i=4 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 20 add_watcher() i=5 2024-03-28T16:49:18.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.655+0000 7f1c234a8980 20 add_watcher() i=6 2024-03-28T16:49:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94686680849152] v29'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 20 add_watcher() i=7 2024-03-28T16:49:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa5120 con 0x561df5f8eae0 2024-03-28T16:49:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94686680839392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa2590 con 0x561df5f8eae0 2024-03-28T16:49:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94686680839392] v29'102 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1ad6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 5 note: GC not initialized 2024-03-28T16:49:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1ad5fe3640 20 reqs_thread_entry: start 2024-03-28T16:49:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:18.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 20 Filter name: none 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1a477fe640 20 reqs_thread_entry: start 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fc69e0 con 0x561df5f8eae0 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.659+0000 7f1c234a8980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fc69e0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv16 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fc5db0 con 0x561df5f97890 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 4.28 4:1795f074:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fc96c0 con 0x561df5f8eae0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 4.22 4:4580d573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fcc320 con 0x561df5f8eae0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 4.2 4:400bbb2d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fcf000 con 0x561df5f8eae0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 4.14 4:2bd00f8d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fcc320 con 0x561df5f97890 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 4.9 4:93976103:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fd4040 con 0x561df5f8eae0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 4.1 4:81996a80:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fd6660 con 0x7f1c1401fbe0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 4.9 4:916ca974:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fd9350 con 0x561df5f8eae0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 4.f 4:f072f27b:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fdbcf0 con 0x561df5f8eae0 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 4.16 4:69af5b3c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fde600 con 0x561df5f97890 2024-03-28T16:49:18.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:74 4.29 4:9412bb76:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fe0ec0 con 0x561df5f97890 2024-03-28T16:49:18.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 4.15 4:a8ab0eea:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fde600 con 0x561df5f97890 2024-03-28T16:49:18.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(67 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(69 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(71 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(72 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(73 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(74 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(75 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(68 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.663+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fce980 con 0x7f1c1401fbe0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(76 foo.buckets [call] v29'17 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fce980 con 0x7f1c1401fbe0 2024-03-28T16:49:18.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(77 foo.buckets [call] v29'18 uv18 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c234a8980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fce980 con 0x561df5f8eae0 2024-03-28T16:49:18.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(78 foo [stat out=16b,read 0~357 out=357b] v0'0 uv11 ondisk = 0) v8 ==== 189+0+373 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c234a8980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:18.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fce980 con 0x7f1c1401fbe0 2024-03-28T16:49:18.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.667+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv18 ondisk = 0) v8 ==== 155+0+58 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1ad6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94686680839392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fb8bd0 con 0x561df5f8eae0 2024-03-28T16:49:18.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 94686680839392] v29'103 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94686680823184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9f4e0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 94686680823184] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1c234a8980 20 remove_watcher() i=0 2024-03-28T16:49:18.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1c234a8980 2 removed watcher, disabling cache 2024-03-28T16:49:18.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.671+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94686680827584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9f4e0 con 0x561df5f8eae0 2024-03-28T16:49:18.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.675+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 94686680827584] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.675+0000 7f1c234a8980 20 remove_watcher() i=1 2024-03-28T16:49:18.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.675+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94686680831328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9c6c0 con 0x561df5f97890 2024-03-28T16:49:18.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.675+0000 7f1c207c2640 1 -- 172.21.15.170:0/2371043630 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 94686680831328] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c14033040 con 0x561df5f97890 2024-03-28T16:49:18.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.675+0000 7f1c234a8980 20 remove_watcher() i=2 2024-03-28T16:49:18.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.675+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94686680835120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9f4e0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.679+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 94686680835120] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x7f1c1401fbe0 2024-03-28T16:49:18.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.679+0000 7f1c234a8980 20 remove_watcher() i=3 2024-03-28T16:49:18.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.679+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94686680828640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9c6c0 con 0x561df5f8eae0 2024-03-28T16:49:18.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.679+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 94686680828640] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c1800a4d0 con 0x561df5f8eae0 2024-03-28T16:49:18.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.679+0000 7f1c234a8980 20 remove_watcher() i=4 2024-03-28T16:49:18.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.679+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94686680841584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9f4e0 con 0x561df5f93020 2024-03-28T16:49:18.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.683+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 94686680841584] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.683+0000 7f1c234a8980 20 remove_watcher() i=5 2024-03-28T16:49:18.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.683+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94686680845680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5f9f4e0 con 0x561df5f93020 2024-03-28T16:49:18.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.683+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 94686680845680] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.683+0000 7f1c234a8980 20 remove_watcher() i=6 2024-03-28T16:49:18.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.683+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94686680849152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x561df5fa0d70 con 0x561df5f93020 2024-03-28T16:49:18.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c20fc3640 1 -- 172.21.15.170:0/2371043630 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 94686680849152] v29'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f1c100b3040 con 0x561df5f93020 2024-03-28T16:49:18.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 20 remove_watcher() i=7 2024-03-28T16:49:18.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f93020 msgr2=0x561df5f95410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f93020 0x561df5f95410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5f8eae0 msgr2=0x561df5f90fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5f8eae0 0x561df5f90fa0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f1c1401fbe0 msgr2=0x7f1c1403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f1c1401fbe0 0x7f1c1403a3c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561df5f97890 msgr2=0x561df5f99c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561df5f97890 0x561df5f99c80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1aec0342b0 msgr2=0x7f1aec036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1aec0342b0 0x7f1aec036770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x561df5f621c0 tx=0x7f1c14010000 comp rx=0 tx=0).stop 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 msgr2=0x561df5f83990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c217c4640 1 -- 172.21.15.170:0/2371043630 reap_dead start 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f83990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c100d9fb0 tx=0x7f1c100b3f50 comp rx=0 tx=0).stop 2024-03-28T16:49:18.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 shutdown_connections 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2371043630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f7caf0 0x561df5f83990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 >> 172.21.15.170:0/2371043630 conn(0x561df5f78e80 msgr2=0x561df5f764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 shutdown_connections 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2371043630 wait complete. 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1af80342b0 msgr2=0x7f1af8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1af80342b0 0x7f1af8036770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x561df5f56bb0 tx=0x7f1c1400f000 comp rx=0 tx=0).stop 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f6fd40 msgr2=0x561df5f70160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f6fd40 0x561df5f70160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c100e1da0 tx=0x7f1c100e3470 comp rx=0 tx=0).stop 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 shutdown_connections 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1af80342b0 0x7f1af8036770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/4001040552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5f6fd40 0x561df5f70160 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 >> 172.21.15.170:0/4001040552 conn(0x561df5f64b50 msgr2=0x561df5f65660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 shutdown_connections 2024-03-28T16:49:18.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/4001040552 wait complete. 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f5b720 msgr2=0x561df5f5db10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561df5f5b720 0x561df5f5db10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5e5e730 msgr2=0x561df5f59160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561df5e5e730 0x561df5f59160 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561df5f5f3e0 msgr2=0x561df5f617d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561df5f5f3e0 0x561df5f617d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1b080342b0 msgr2=0x7f1b08036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1b080342b0 0x7f1b08036770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1c14001e50 tx=0x7f1c140013b0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 msgr2=0x561df5f52530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 --2- 172.21.15.170:0/2311851609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561df5e5de10 0x561df5f52530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c100b8040 tx=0x7f1c10019fa0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c217c4640 1 -- 172.21.15.170:0/2311851609 reap_dead start 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 shutdown_connections 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 >> 172.21.15.170:0/2311851609 conn(0x561df5e596e0 msgr2=0x561df5f41a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 shutdown_connections 2024-03-28T16:49:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.687+0000 7f1c234a8980 1 -- 172.21.15.170:0/2311851609 wait complete. 2024-03-28T16:49:18.698 INFO:teuthology.orchestra.run.smithi170.stdout:{"stats":{"size":3,"size_actual":4096,"size_kb":1,"size_kb_actual":4,"num_objects":1},"last_stats_sync":"2024-03-28T16:49:18.669195Z","last_stats_update":"2024-03-28T16:49:18.667230Z"} 2024-03-28T16:49:18.699 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-03-28T16:49:18.669195Z', 'last_stats_update': '2024-03-28T16:49:18.667230Z'} 2024-03-28T16:49:18.705 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:18.705 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-03-28T16:49:18.705 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-03-28T16:49:18.705 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-03-28T16:49:18.773 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:18.773 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:18.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febbb502980 1 Processor -- start 2024-03-28T16:49:18.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febbb502980 1 -- start start 2024-03-28T16:49:18.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febbb502980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56293a45ae10 0x56293a45b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febbb502980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56293a3a84c0 con 0x56293a45b730 2024-03-28T16:49:18.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febbb502980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56293a36f0f0 con 0x56293a45ae10 2024-03-28T16:49:18.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febb901d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56293a45ae10 0x56293a45b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febb901d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56293a45ae10 0x56293a45b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56520/0 (socket says 172.21.15.170:56520) 2024-03-28T16:49:18.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febb881c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x56293a45b730 0x56293a45c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32868/0 (socket says 172.21.15.170:32868) 2024-03-28T16:49:18.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febb901d640 1 -- 172.21.15.170:0/2150666938 learned_addr learned my addr 172.21.15.170:0/2150666938 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febb901d640 1 -- 172.21.15.170:0/2150666938 >> v1:172.21.15.119:6789/0 conn(0x56293a45b730 legacy=0x56293a45c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.775+0000 7febb901d640 1 -- 172.21.15.170:0/2150666938 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293a45c970 con 0x56293a45ae10 2024-03-28T16:49:18.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febb901d640 1 --2- 172.21.15.170:0/2150666938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a45b1f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7febb400a4e0 tx=0x7febb401b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153f596b0f11f160 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febabfff640 1 -- 172.21.15.170:0/2150666938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40aa540 con 0x56293a45ae10 2024-03-28T16:49:18.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febabfff640 1 -- 172.21.15.170:0/2150666938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40a6070 con 0x56293a45ae10 2024-03-28T16:49:18.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febabfff640 1 -- 172.21.15.170:0/2150666938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40aa850 con 0x56293a45ae10 2024-03-28T16:49:18.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febbb502980 1 -- 172.21.15.170:0/2150666938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 msgr2=0x56293a45b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febbb502980 1 --2- 172.21.15.170:0/2150666938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a45b1f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7febb400a4e0 tx=0x7febb401b090 comp rx=0 tx=0).stop 2024-03-28T16:49:18.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febbb502980 1 -- 172.21.15.170:0/2150666938 shutdown_connections 2024-03-28T16:49:18.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febbb502980 1 --2- 172.21.15.170:0/2150666938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a45b1f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febbb502980 1 -- 172.21.15.170:0/2150666938 >> 172.21.15.170:0/2150666938 conn(0x56293a4566e0 msgr2=0x56293a458b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febbb502980 1 -- 172.21.15.170:0/2150666938 shutdown_connections 2024-03-28T16:49:18.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.779+0000 7febbb502980 1 -- 172.21.15.170:0/2150666938 wait complete. 2024-03-28T16:49:18.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febbb502980 1 Processor -- start 2024-03-28T16:49:18.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febbb502980 1 -- start start 2024-03-28T16:49:18.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febbb502980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a545330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febbb502980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293a36c620 con 0x56293a45ae10 2024-03-28T16:49:18.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a545330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a545330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56534/0 (socket says 172.21.15.170:56534) 2024-03-28T16:49:18.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febb901d640 1 -- 172.21.15.170:0/140965827 learned_addr learned my addr 172.21.15.170:0/140965827 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febb901d640 1 -- 172.21.15.170:0/140965827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb400a1c0 con 0x56293a45ae10 2024-03-28T16:49:18.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.795+0000 7febb901d640 1 --2- 172.21.15.170:0/140965827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a545330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7febb4018cc0 tx=0x7febb40033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acf65d3810f58d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7feaa7606640 1 -- 172.21.15.170:0/140965827 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40aa700 con 0x56293a45ae10 2024-03-28T16:49:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7feaa7606640 1 -- 172.21.15.170:0/140965827 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40aa860 con 0x56293a45ae10 2024-03-28T16:49:18.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7feaa7606640 1 -- 172.21.15.170:0/140965827 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40aab20 con 0x56293a45ae10 2024-03-28T16:49:18.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- 172.21.15.170:0/140965827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 msgr2=0x56293a545330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 --2- 172.21.15.170:0/140965827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a545330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7febb4018cc0 tx=0x7febb40033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- 172.21.15.170:0/140965827 shutdown_connections 2024-03-28T16:49:18.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 --2- 172.21.15.170:0/140965827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a545330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- 172.21.15.170:0/140965827 >> 172.21.15.170:0/140965827 conn(0x56293a4566e0 msgr2=0x56293a540ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- 172.21.15.170:0/140965827 shutdown_connections 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- 172.21.15.170:0/140965827 wait complete. 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 Processor -- start 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- start start 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a3cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293a367e00 con 0x56293a45ae10 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a3cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a3cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56538/0 (socket says 172.21.15.170:56538) 2024-03-28T16:49:18.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febb901d640 1 -- 172.21.15.170:0/2440579557 learned_addr learned my addr 172.21.15.170:0/2440579557 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febb901d640 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293a45c970 con 0x56293a45ae10 2024-03-28T16:49:18.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febb901d640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a3cfd90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febb4018cc0 tx=0x7febb4017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7feaa5e03640 1 -- 172.21.15.170:0/2440579557 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40c4070 con 0x56293a45ae10 2024-03-28T16:49:18.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7feaa5e03640 1 -- 172.21.15.170:0/2440579557 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40b5650 con 0x56293a45ae10 2024-03-28T16:49:18.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7feaa5e03640 1 -- 172.21.15.170:0/2440579557 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40b58f0 con 0x56293a45ae10 2024-03-28T16:49:18.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56293a3d02d0 con 0x56293a45ae10 2024-03-28T16:49:18.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.799+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56293a550120 con 0x56293a45ae10 2024-03-28T16:49:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7feaa5e03640 1 -- 172.21.15.170:0/2440579557 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7febb40a8070 con 0x56293a45ae10 2024-03-28T16:49:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7feaa5e03640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7feaa00342b0 0x7feaa0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7feaa5e03640 1 -- 172.21.15.170:0/2440579557 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7febb40e3420 con 0x56293a45ae10 2024-03-28T16:49:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a45b730 0x56293a553b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febb881c640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7feaa00342b0 0x7feaa0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5540c0 con 0x56293a45b730 2024-03-28T16:49:18.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febb981e640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a45b730 0x56293a553b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febb881c640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7feaa00342b0 0x7feaa0036770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f70 tx=0x7febb0001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febb981e640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a45b730 0x56293a553b40 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febb981e640 1 -- 172.21.15.170:0/2440579557 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a45b730 2024-03-28T16:49:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a551250 con 0x56293a45b730 2024-03-28T16:49:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febb981e640 1 -- 172.21.15.170:0/2440579557 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7febac0083f0 con 0x56293a45b730 2024-03-28T16:49:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a556300 0x56293a5586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a558c30 con 0x56293a556300 2024-03-28T16:49:18.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.803+0000 7febb901d640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a556300 0x56293a5586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb901d640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a556300 0x56293a5586f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb901d640 1 -- 172.21.15.170:0/2440579557 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7febb40bd4a0 con 0x56293a556300 2024-03-28T16:49:18.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a555170 con 0x56293a45b730 2024-03-28T16:49:18.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb981e640 1 -- 172.21.15.170:0/2440579557 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7febac0083f0 con 0x56293a45b730 2024-03-28T16:49:18.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56293a559ff0 0x56293a55c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a55ca30 con 0x56293a559ff0 2024-03-28T16:49:18.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb881c640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56293a559ff0 0x56293a55c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb881c640 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56293a559ff0 0x56293a55c3e0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb881c640 1 -- 172.21.15.170:0/2440579557 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7febb000f6a0 con 0x56293a559ff0 2024-03-28T16:49:18.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febbb502980 1 Processor -- start 2024-03-28T16:49:18.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febbb502980 1 -- start start 2024-03-28T16:49:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febbb502980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a54daa0 0x56293a54bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febbb502980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293a300e30 con 0x56293a54daa0 2024-03-28T16:49:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a54daa0 0x56293a54bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a54daa0 0x56293a54bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56542/0 (socket says 172.21.15.170:56542) 2024-03-28T16:49:18.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.807+0000 7febb901d640 1 -- 172.21.15.170:0/2572552094 learned_addr learned my addr 172.21.15.170:0/2572552094 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febb901d640 1 -- 172.21.15.170:0/2572552094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293a54c480 con 0x56293a54daa0 2024-03-28T16:49:18.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febb901d640 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a54daa0 0x56293a54bf40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7febb4005670 tx=0x7febb40cae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7fea9effd640 1 -- 172.21.15.170:0/2572552094 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40a9930 con 0x56293a54daa0 2024-03-28T16:49:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7fea9effd640 1 -- 172.21.15.170:0/2572552094 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40db040 con 0x56293a54daa0 2024-03-28T16:49:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56293a54c6d0 con 0x56293a54daa0 2024-03-28T16:49:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7fea9effd640 1 -- 172.21.15.170:0/2572552094 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40070a0 con 0x56293a54daa0 2024-03-28T16:49:18.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56293a54cb60 con 0x56293a54daa0 2024-03-28T16:49:18.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7fea9effd640 1 -- 172.21.15.170:0/2572552094 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7febb4007200 con 0x56293a54daa0 2024-03-28T16:49:18.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7fea9effd640 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea94034300 0x7fea940367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7fea9effd640 1 -- 172.21.15.170:0/2572552094 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7febb40d4040 con 0x56293a54daa0 2024-03-28T16:49:18.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febb881c640 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea94034300 0x7fea940367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febbb502980 1 Processor -- start 2024-03-28T16:49:18.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febbb502980 1 -- start start 2024-03-28T16:49:18.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febb881c640 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea94034300 0x7fea940367c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x56293a5554d0 tx=0x7febb000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febbb502980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a579f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.811+0000 7febbb502980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293a2f4790 con 0x56293a579b90 2024-03-28T16:49:18.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a579f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a579f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56550/0 (socket says 172.21.15.170:56550) 2024-03-28T16:49:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 -- 172.21.15.170:0/3352522957 learned_addr learned my addr 172.21.15.170:0/3352522957 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 -- 172.21.15.170:0/3352522957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293a57a560 con 0x56293a579b90 2024-03-28T16:49:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 --2- 172.21.15.170:0/3352522957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a579f90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7febb40b49f0 tx=0x7febb40d3ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d28f2c30244f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7fea9cff9640 1 -- 172.21.15.170:0/3352522957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40d5420 con 0x56293a579b90 2024-03-28T16:49:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7fea9cff9640 1 -- 172.21.15.170:0/3352522957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40ec430 con 0x56293a579b90 2024-03-28T16:49:18.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7fea9cff9640 1 -- 172.21.15.170:0/3352522957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb4007de0 con 0x56293a579b90 2024-03-28T16:49:18.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- 172.21.15.170:0/3352522957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 msgr2=0x56293a579f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 --2- 172.21.15.170:0/3352522957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a579f90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7febb40b49f0 tx=0x7febb40d3ab0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- 172.21.15.170:0/3352522957 shutdown_connections 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 --2- 172.21.15.170:0/3352522957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a579f90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- 172.21.15.170:0/3352522957 >> 172.21.15.170:0/3352522957 conn(0x56293a575ed0 msgr2=0x56293a573f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- 172.21.15.170:0/3352522957 shutdown_connections 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- 172.21.15.170:0/3352522957 wait complete. 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 Processor -- start 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- start start 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a580a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293a580f90 con 0x56293a579b90 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a580a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a580a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56552/0 (socket says 172.21.15.170:56552) 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 learned_addr learned my addr 172.21.15.170:0/3771304908 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293a54c6d0 con 0x56293a579b90 2024-03-28T16:49:18.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febb901d640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a580a50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7febb40d6ba0 tx=0x7febb40ebe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7fea8affd640 1 -- 172.21.15.170:0/3771304908 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40d5710 con 0x56293a579b90 2024-03-28T16:49:18.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56293a54c6d0 con 0x56293a579b90 2024-03-28T16:49:18.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7fea8affd640 1 -- 172.21.15.170:0/3771304908 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40d5870 con 0x56293a579b90 2024-03-28T16:49:18.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.815+0000 7fea8affd640 1 -- 172.21.15.170:0/3771304908 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb40b4bb0 con 0x56293a579b90 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56293a54c6d0 con 0x56293a579b90 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7fea8affd640 1 -- 172.21.15.170:0/3771304908 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7febb40b4d10 con 0x56293a579b90 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7fea8affd640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea840342b0 0x7fea84036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7fea8affd640 1 -- 172.21.15.170:0/3771304908 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7febb410aac0 con 0x56293a579b90 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb881c640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea840342b0 0x7fea84036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a58bbf0 0x56293a58e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58e800 con 0x56293a58bbf0 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb881c640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea840342b0 0x7fea84036770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febb000c010 tx=0x7febb0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb981e640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a58bbf0 0x56293a58e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb981e640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a58bbf0 0x56293a58e0b0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 realm 2024-03-28T16:49:18.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58ebd0 con 0x56293a58bbf0 2024-03-28T16:49:18.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f450 con 0x56293a58bbf0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a5904c0 0x56293a5928b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.819+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a592ec0 con 0x56293a5904c0 2024-03-28T16:49:18.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb901d640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a5904c0 0x56293a5928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb901d640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a5904c0 0x56293a5928b0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febbb502980 20 searching for the correct realm 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56293a5950c0 0x56293a597580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a597c50 con 0x56293a5950c0 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb881c640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56293a5950c0 0x56293a597580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb881c640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56293a5950c0 0x56293a597580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb881c640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7febb001fc60 0x7febb003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a597c50 con 0x7febb001fc60 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb981e640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7febb001fc60 0x7febb003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb981e640 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7febb001fc60 0x7febb003a3c0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.823+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x7febb001fc60 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.827+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x7febb001fc60 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x7febb001fc60 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.831+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x7febb001fc60 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5950c0 2024-03-28T16:49:18.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a5904c0 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x56293a58bbf0 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.835+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a58f7b0 con 0x7febb001fc60 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:18.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5934c0 con 0x56293a58bbf0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5934c0 con 0x56293a58bbf0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a593470 con 0x7febb001fc60 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:18.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 zone default found 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 4 Realm: () 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 zonegroup default 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a593d10 con 0x56293a5904c0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a594140 con 0x56293a5950c0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a599b50 con 0x56293a5950c0 2024-03-28T16:49:18.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:18.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 started sync module instance, tier type = 2024-03-28T16:49:18.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:18.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a599350 con 0x7febb001fc60 2024-03-28T16:49:18.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.839+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.843+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94735072595632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a599700 con 0x7febb001fc60 2024-03-28T16:49:18.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.843+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59a8b0 con 0x56293a58bbf0 2024-03-28T16:49:18.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.843+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94735072595632] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.843+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.843+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94735072599920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59a830 con 0x56293a58bbf0 2024-03-28T16:49:18.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.843+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59b2b0 con 0x56293a5950c0 2024-03-28T16:49:18.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94735072599920] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94735072600976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59a7b0 con 0x56293a5950c0 2024-03-28T16:49:18.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59bfd0 con 0x7febb001fc60 2024-03-28T16:49:18.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94735072607328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59c820 con 0x7febb001fc60 2024-03-28T16:49:18.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59d210 con 0x56293a58bbf0 2024-03-28T16:49:18.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94735072600976] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94735072607328] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94735072611568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59c330 con 0x56293a58bbf0 2024-03-28T16:49:18.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.847+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59da60 con 0x56293a5904c0 2024-03-28T16:49:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.851+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94735072611568] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.851+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.851+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94735072613824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59c330 con 0x56293a5904c0 2024-03-28T16:49:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.851+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59e260 con 0x56293a5904c0 2024-03-28T16:49:18.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94735072613824] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94735072615872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59e680 con 0x56293a5904c0 2024-03-28T16:49:18.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59eff0 con 0x56293a5904c0 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94735072619344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59f450 con 0x56293a5904c0 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 20 add_watcher() i=0 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 20 add_watcher() i=1 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 20 add_watcher() i=2 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 20 add_watcher() i=3 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 20 add_watcher() i=4 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 20 add_watcher() i=5 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94735072615872] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.855+0000 7febbb502980 20 add_watcher() i=6 2024-03-28T16:49:18.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94735072619344] v29'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 20 add_watcher() i=7 2024-03-28T16:49:18.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:18.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59e1e0 con 0x56293a58bbf0 2024-03-28T16:49:18.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94735072620400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59e210 con 0x56293a58bbf0 2024-03-28T16:49:18.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94735072620400] v29'104 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7fea6f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:18.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:18.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 5 note: GC not initialized 2024-03-28T16:49:18.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7fea6e7e4640 20 reqs_thread_entry: start 2024-03-28T16:49:18.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:18.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 20 Filter name: none 2024-03-28T16:49:18.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7fe9dffff640 20 reqs_thread_entry: start 2024-03-28T16:49:18.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:18.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.859+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5c4480 con 0x56293a58bbf0 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5c5fe0 con 0x7febb001fc60 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:18.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5c7710 con 0x56293a5950c0 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-03-28T16:49:18.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5cae50 con 0x7febb001fc60 2024-03-28T16:49:18.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 foo.buckets [call] v29'19 uv19 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 20 rados->read ofs=0 len=0 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.863+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5c9fb0 con 0x7febb001fc60 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:18.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5ca990 con 0x7febb001fc60 2024-03-28T16:49:18.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v29'20 uv20 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-03-28T16:49:18.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:18.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:49:18.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.867+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94735072797808 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5ca970 con 0x56293a5904c0 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== watch-notify(notify (1) cookie 94735072615872 notify 124554051586 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x56293a57a560 con 0x56293a5904c0 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7fea9cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051586 cookie 94735072615872 notifier 4473 bl.length()=338 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7fea9cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7fea9cff9640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7fea9cff9640 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fea7c003ea0 con 0x56293a5904c0 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(68 notify.6 [notify cookie 94735072797808 out=8b] v0'0 uv105 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify_complete (2) cookie 94735072797808 notify 124554051586 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7febb40e7070 con 0x56293a5904c0 2024-03-28T16:49:18.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7fea6f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:18.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94735072620400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5ca910 con 0x56293a58bbf0 2024-03-28T16:49:18.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94735072620400] v29'105 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.871+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94735072595632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a5ca910 con 0x7febb001fc60 2024-03-28T16:49:18.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.875+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94735072595632] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.875+0000 7febbb502980 20 remove_watcher() i=0 2024-03-28T16:49:18.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.875+0000 7febbb502980 2 removed watcher, disabling cache 2024-03-28T16:49:18.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.875+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94735072599920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a598eb0 con 0x56293a58bbf0 2024-03-28T16:49:18.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.875+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94735072599920] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.875+0000 7febbb502980 20 remove_watcher() i=1 2024-03-28T16:49:18.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.875+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94735072600976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a599f70 con 0x56293a5950c0 2024-03-28T16:49:18.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febb881c640 1 -- 172.21.15.170:0/3771304908 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94735072600976] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb0033040 con 0x56293a5950c0 2024-03-28T16:49:18.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febbb502980 20 remove_watcher() i=2 2024-03-28T16:49:18.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94735072607328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a598eb0 con 0x7febb001fc60 2024-03-28T16:49:18.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94735072607328] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x7febb001fc60 2024-03-28T16:49:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febbb502980 20 remove_watcher() i=3 2024-03-28T16:49:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94735072611568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a598eb0 con 0x56293a58bbf0 2024-03-28T16:49:18.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94735072611568] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febac0083f0 con 0x56293a58bbf0 2024-03-28T16:49:18.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febbb502980 20 remove_watcher() i=4 2024-03-28T16:49:18.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.879+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94735072613824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59ccf0 con 0x56293a5904c0 2024-03-28T16:49:18.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.883+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94735072613824] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.883+0000 7febbb502980 20 remove_watcher() i=5 2024-03-28T16:49:18.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.883+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94735072615872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a598eb0 con 0x56293a5904c0 2024-03-28T16:49:18.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.883+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94735072615872] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.883+0000 7febbb502980 20 remove_watcher() i=6 2024-03-28T16:49:18.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94735072619344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56293a59d5c0 con 0x56293a5904c0 2024-03-28T16:49:18.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febb901d640 1 -- 172.21.15.170:0/3771304908 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94735072619344] v29'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7febb40fc040 con 0x56293a5904c0 2024-03-28T16:49:18.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 20 remove_watcher() i=7 2024-03-28T16:49:18.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a5904c0 msgr2=0x56293a5928b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a5904c0 0x56293a5928b0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a58bbf0 msgr2=0x56293a58e0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a58bbf0 0x56293a58e0b0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7febb001fc60 msgr2=0x7febb003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7febb001fc60 0x7febb003a3c0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56293a5950c0 msgr2=0x56293a597580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56293a5950c0 0x56293a597580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea840342b0 msgr2=0x7fea84036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea840342b0 0x7fea84036770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febb000c010 tx=0x7febb0010000 comp rx=0 tx=0).stop 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 msgr2=0x56293a580a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febb981e640 1 -- 172.21.15.170:0/3771304908 reap_dead start 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a580a50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7febb40d6ba0 tx=0x7febb40ebe90 comp rx=0 tx=0).stop 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 shutdown_connections 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/3771304908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a579b90 0x56293a580a50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 >> 172.21.15.170:0/3771304908 conn(0x56293a575ed0 msgr2=0x56293a573530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 shutdown_connections 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/3771304908 wait complete. 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea94034300 msgr2=0x7fea940367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea94034300 0x7fea940367c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x56293a5554d0 tx=0x7febb000f000 comp rx=0 tx=0).stop 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a54daa0 msgr2=0x56293a54bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a54daa0 0x56293a54bf40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7febb4005670 tx=0x7febb40cae00 comp rx=0 tx=0).stop 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 shutdown_connections 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fea94034300 0x7fea940367c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2572552094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a54daa0 0x56293a54bf40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 >> 172.21.15.170:0/2572552094 conn(0x56293a55f770 msgr2=0x56293a560280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 shutdown_connections 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2572552094 wait complete. 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a556300 msgr2=0x56293a5586f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56293a556300 0x56293a5586f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a45b730 msgr2=0x56293a553b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56293a45b730 0x56293a553b40 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56293a559ff0 msgr2=0x56293a55c3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56293a559ff0 0x56293a55c3e0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7feaa00342b0 msgr2=0x7feaa0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7feaa00342b0 0x7feaa0036770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febb0000f70 tx=0x7febb0001380 comp rx=0 tx=0).stop 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 msgr2=0x56293a3cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 --2- 172.21.15.170:0/2440579557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56293a45ae10 0x56293a3cfd90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febb4018cc0 tx=0x7febb4017e30 comp rx=0 tx=0).stop 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febb981e640 1 -- 172.21.15.170:0/2440579557 reap_dead start 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 shutdown_connections 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 >> 172.21.15.170:0/2440579557 conn(0x56293a4566e0 msgr2=0x56293a53f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 shutdown_connections 2024-03-28T16:49:18.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.887+0000 7febbb502980 1 -- 172.21.15.170:0/2440579557 wait complete. 2024-03-28T16:49:18.900 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-03-28T16:49:18.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-03-28T16:49:18.901 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-03-28T16:49:18.982 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:18.982 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f832a0ba980 1 Processor -- start 2024-03-28T16:49:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f832a0ba980 1 -- start start 2024-03-28T16:49:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f832a0ba980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x555f85f58170 0x555f85f58550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f832a0ba980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555f85ea24c0 con 0x555f85f58a90 2024-03-28T16:49:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f832a0ba980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x555f85e690f0 con 0x555f85f58170 2024-03-28T16:49:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f8327bd5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x555f85f58170 0x555f85f58550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f8327bd5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x555f85f58170 0x555f85f58550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56556/0 (socket says 172.21.15.170:56556) 2024-03-28T16:49:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f8327bd5640 1 -- 172.21.15.170:0/39205036 learned_addr learned my addr 172.21.15.170:0/39205036 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f83273d4640 1 --1- 172.21.15.170:0/39205036 >> v1:172.21.15.119:6789/0 conn(0x555f85f58a90 0x555f85f59440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:32874/0 (socket says 172.21.15.170:32874) 2024-03-28T16:49:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f8327bd5640 1 -- 172.21.15.170:0/39205036 >> v1:172.21.15.119:6789/0 conn(0x555f85f58a90 legacy=0x555f85f59440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.983+0000 7f8327bd5640 1 -- 172.21.15.170:0/39205036 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f85f59c70 con 0x555f85f58170 2024-03-28T16:49:18.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f8327bd5640 1 --2- 172.21.15.170:0/39205036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58170 0x555f85f58550 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f832000a4e0 tx=0x7f832001b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c442afd5db446e04 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:18.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f8326bd3640 1 -- 172.21.15.170:0/39205036 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200aa540 con 0x555f85f58170 2024-03-28T16:49:18.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f8326bd3640 1 -- 172.21.15.170:0/39205036 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200a6070 con 0x555f85f58170 2024-03-28T16:49:18.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f8326bd3640 1 -- 172.21.15.170:0/39205036 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200aa850 con 0x555f85f58170 2024-03-28T16:49:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f832a0ba980 1 -- 172.21.15.170:0/39205036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58170 msgr2=0x555f85f58550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f832a0ba980 1 --2- 172.21.15.170:0/39205036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58170 0x555f85f58550 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f832000a4e0 tx=0x7f832001b090 comp rx=0 tx=0).stop 2024-03-28T16:49:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f832a0ba980 1 -- 172.21.15.170:0/39205036 shutdown_connections 2024-03-28T16:49:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f832a0ba980 1 --2- 172.21.15.170:0/39205036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58170 0x555f85f58550 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f832a0ba980 1 -- 172.21.15.170:0/39205036 >> 172.21.15.170:0/39205036 conn(0x555f85ecb890 msgr2=0x555f85ecbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f832a0ba980 1 -- 172.21.15.170:0/39205036 shutdown_connections 2024-03-28T16:49:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:18.987+0000 7f832a0ba980 1 -- 172.21.15.170:0/39205036 wait complete. 2024-03-28T16:49:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f832a0ba980 1 Processor -- start 2024-03-28T16:49:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f832a0ba980 1 -- start start 2024-03-28T16:49:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f832a0ba980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86042860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f832a0ba980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f85e66620 con 0x555f85f58a90 2024-03-28T16:49:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86042860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86042860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56568/0 (socket says 172.21.15.170:56568) 2024-03-28T16:49:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f8327bd5640 1 -- 172.21.15.170:0/745686288 learned_addr learned my addr 172.21.15.170:0/745686288 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.003+0000 7f8327bd5640 1 -- 172.21.15.170:0/745686288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832000a1c0 con 0x555f85f58a90 2024-03-28T16:49:19.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8327bd5640 1 --2- 172.21.15.170:0/745686288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86042860 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f832000a4e0 tx=0x7f83200033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf18cba22d93f80 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8214e19640 1 -- 172.21.15.170:0/745686288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200aa640 con 0x555f85f58a90 2024-03-28T16:49:19.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8214e19640 1 -- 172.21.15.170:0/745686288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200aa7a0 con 0x555f85f58a90 2024-03-28T16:49:19.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8214e19640 1 -- 172.21.15.170:0/745686288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200aaa60 con 0x555f85f58a90 2024-03-28T16:49:19.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- 172.21.15.170:0/745686288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 msgr2=0x555f86042860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 --2- 172.21.15.170:0/745686288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86042860 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f832000a4e0 tx=0x7f83200033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- 172.21.15.170:0/745686288 shutdown_connections 2024-03-28T16:49:19.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 --2- 172.21.15.170:0/745686288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86042860 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- 172.21.15.170:0/745686288 >> 172.21.15.170:0/745686288 conn(0x555f85ecb890 msgr2=0x555f8603e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- 172.21.15.170:0/745686288 shutdown_connections 2024-03-28T16:49:19.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- 172.21.15.170:0/745686288 wait complete. 2024-03-28T16:49:19.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 Processor -- start 2024-03-28T16:49:19.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- start start 2024-03-28T16:49:19.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86049650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f85e61e00 con 0x555f85f58a90 2024-03-28T16:49:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86049650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86049650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56576/0 (socket says 172.21.15.170:56576) 2024-03-28T16:49:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8327bd5640 1 -- 172.21.15.170:0/740033857 learned_addr learned my addr 172.21.15.170:0/740033857 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8327bd5640 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f85f59c70 con 0x555f85f58a90 2024-03-28T16:49:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8327bd5640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86049650 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f832001b0d0 tx=0x7f8320017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8213616640 1 -- 172.21.15.170:0/740033857 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200aa640 con 0x555f85f58a90 2024-03-28T16:49:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f86049b90 con 0x555f85f58a90 2024-03-28T16:49:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8213616640 1 -- 172.21.15.170:0/740033857 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200b5440 con 0x555f85f58a90 2024-03-28T16:49:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f8213616640 1 -- 172.21.15.170:0/740033857 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200b5740 con 0x555f85f58a90 2024-03-28T16:49:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.007+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f86049fc0 con 0x555f85f58a90 2024-03-28T16:49:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f8213616640 1 -- 172.21.15.170:0/740033857 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f83200a8070 con 0x555f85f58a90 2024-03-28T16:49:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f8213616640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f820c0342b0 0x7f820c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f8213616640 1 -- 172.21.15.170:0/740033857 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f83200e4230 con 0x555f85f58a90 2024-03-28T16:49:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f83273d4640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f820c0342b0 0x7f820c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f8604e2c0 0x555f860506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86050bf0 con 0x555f8604e2c0 2024-03-28T16:49:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f83283d6640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f8604e2c0 0x555f860506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f83273d4640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f820c0342b0 0x7f820c036770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8318000f70 tx=0x7f8318001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f83283d6640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f8604e2c0 0x555f860506b0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f83283d6640 1 -- 172.21.15.170:0/740033857 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f8604e2c0 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86050f50 con 0x555f8604e2c0 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f83283d6640 1 -- 172.21.15.170:0/740033857 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f83100083f0 con 0x555f8604e2c0 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f860528a0 0x555f86054cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86055430 con 0x555f860528a0 2024-03-28T16:49:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.011+0000 7f8327bd5640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f860528a0 0x555f86054cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f8327bd5640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f860528a0 0x555f86054cc0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f8327bd5640 1 -- 172.21.15.170:0/740033857 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f83200c5020 con 0x555f860528a0 2024-03-28T16:49:19.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86051750 con 0x555f8604e2c0 2024-03-28T16:49:19.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f83283d6640 1 -- 172.21.15.170:0/740033857 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f83100083f0 con 0x555f8604e2c0 2024-03-28T16:49:19.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555f86056850 0x555f86058c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86059300 con 0x555f86056850 2024-03-28T16:49:19.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f83273d4640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555f86056850 0x555f86058c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f83273d4640 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555f86056850 0x555f86058c40 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f83273d4640 1 -- 172.21.15.170:0/740033857 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f831800f6a0 con 0x555f86056850 2024-03-28T16:49:19.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f832a0ba980 1 Processor -- start 2024-03-28T16:49:19.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f832a0ba980 1 -- start start 2024-03-28T16:49:19.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f832a0ba980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86067230 0x555f86067650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f832a0ba980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f85dfae30 con 0x555f86067230 2024-03-28T16:49:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86067230 0x555f86067650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86067230 0x555f86067650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56586/0 (socket says 172.21.15.170:56586) 2024-03-28T16:49:19.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f8327bd5640 1 -- 172.21.15.170:0/3622594854 learned_addr learned my addr 172.21.15.170:0/3622594854 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.015+0000 7f8327bd5640 1 -- 172.21.15.170:0/3622594854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f86067b90 con 0x555f86067230 2024-03-28T16:49:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8327bd5640 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86067230 0x555f86067650 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83200e2770 tx=0x7f83200b4860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8210e11640 1 -- 172.21.15.170:0/3622594854 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200d9040 con 0x555f86067230 2024-03-28T16:49:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f86067de0 con 0x555f86067230 2024-03-28T16:49:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f86068930 con 0x555f86067230 2024-03-28T16:49:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8210e11640 1 -- 172.21.15.170:0/3622594854 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200bcc70 con 0x555f86067230 2024-03-28T16:49:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8210e11640 1 -- 172.21.15.170:0/3622594854 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200b4bc0 con 0x555f86067230 2024-03-28T16:49:19.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8210e11640 1 -- 172.21.15.170:0/3622594854 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f83200b4d20 con 0x555f86067230 2024-03-28T16:49:19.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8210e11640 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f82000342b0 0x7f8200036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8210e11640 1 -- 172.21.15.170:0/3622594854 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f83200f0af0 con 0x555f86067230 2024-03-28T16:49:19.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f83273d4640 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f82000342b0 0x7f8200036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f832a0ba980 1 Processor -- start 2024-03-28T16:49:19.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f832a0ba980 1 -- start start 2024-03-28T16:49:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f83273d4640 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f82000342b0 0x7f8200036770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x555f86051ab0 tx=0x7f831800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f832a0ba980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f86074580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f832a0ba980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f85dee790 con 0x555f86074160 2024-03-28T16:49:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f86074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f86074580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56602/0 (socket says 172.21.15.170:56602) 2024-03-28T16:49:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8327bd5640 1 -- 172.21.15.170:0/1650729633 learned_addr learned my addr 172.21.15.170:0/1650729633 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8327bd5640 1 -- 172.21.15.170:0/1650729633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f86074b50 con 0x555f86074160 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.019+0000 7f8327bd5640 1 --2- 172.21.15.170:0/1650729633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f86074580 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f83200bdff0 tx=0x7f8320009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3acef0a2e52a081 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f820a7fc640 1 -- 172.21.15.170:0/1650729633 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200f0d80 con 0x555f86074160 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f820a7fc640 1 -- 172.21.15.170:0/1650729633 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83200e2b70 con 0x555f86074160 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f820a7fc640 1 -- 172.21.15.170:0/1650729633 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200cc120 con 0x555f86074160 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- 172.21.15.170:0/1650729633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 msgr2=0x555f86074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 --2- 172.21.15.170:0/1650729633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f86074580 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f83200bdff0 tx=0x7f8320009fc0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- 172.21.15.170:0/1650729633 shutdown_connections 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 --2- 172.21.15.170:0/1650729633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f86074580 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- 172.21.15.170:0/1650729633 >> 172.21.15.170:0/1650729633 conn(0x555f860704f0 msgr2=0x555f8606e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- 172.21.15.170:0/1650729633 shutdown_connections 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- 172.21.15.170:0/1650729633 wait complete. 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 Processor -- start 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- start start 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f8607b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8607b540 con 0x555f86074160 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f8607b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8327bd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f8607b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56610/0 (socket says 172.21.15.170:56610) 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 learned_addr learned my addr 172.21.15.170:0/3053350969 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f86068930 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8327bd5640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f8607b000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f83200e1540 tx=0x7f83200c9f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8208ff9640 1 -- 172.21.15.170:0/3053350969 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8320106040 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8208ff9640 1 -- 172.21.15.170:0/3053350969 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8320009720 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555f86068930 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8208ff9640 1 -- 172.21.15.170:0/3053350969 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83200a8af0 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f86068930 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8208ff9640 1 -- 172.21.15.170:0/3053350969 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f83200a93c0 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.023+0000 7f8208ff9640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f81f0034300 0x7f81f00367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f8208ff9640 1 -- 172.21.15.170:0/3053350969 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f832010daf0 con 0x555f86074160 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83273d4640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f81f0034300 0x7f81f00367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f860861f0 0x555f860886b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86088e00 con 0x555f860861f0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83273d4640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f81f0034300 0x7f81f00367c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f831800c010 tx=0x7f8318010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83283d6640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f860861f0 0x555f860886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83283d6640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f860861f0 0x555f860886b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 realm 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860891d0 con 0x555f860861f0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089db0 con 0x555f860861f0 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f8608a760 0x555f8608cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f8608d160 con 0x555f8608a760 2024-03-28T16:49:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f8327bd5640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f8608a760 0x555f8608cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f8327bd5640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f8608a760 0x555f8608cb50 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.027+0000 7f832a0ba980 20 searching for the correct realm 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555f8608f360 0x555f86091820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86091ef0 con 0x555f8608f360 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83273d4640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555f8608f360 0x555f86091820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83273d4640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555f8608f360 0x555f86091820 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83273d4640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f831801fc60 0x7f831803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86091ef0 con 0x7f831801fc60 2024-03-28T16:49:19.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f831801fc60 0x7f831803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f831801fc60 0x7f831803a3c0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.031+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x7f831801fc60 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x7f831801fc60 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.035+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x7f831801fc60 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.039+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x7f831801fc60 2024-03-28T16:49:19.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608f360 2024-03-28T16:49:19.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f8608a760 2024-03-28T16:49:19.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x555f860861f0 2024-03-28T16:49:19.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86089a50 con 0x7f831801fc60 2024-03-28T16:49:19.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f8608d760 con 0x555f860861f0 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.043+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f8608d760 con 0x555f860861f0 2024-03-28T16:49:19.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f8608d710 con 0x7f831801fc60 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 zone default found 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 4 Realm: () 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 zonegroup default 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f8608dfb0 con 0x555f8608a760 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86093d10 con 0x555f8608f360 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86094070 con 0x555f8608f360 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 started sync module instance, tier type = 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:19.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860935f0 con 0x7f831801fc60 2024-03-28T16:49:19.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93868759003472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860939a0 con 0x7f831801fc60 2024-03-28T16:49:19.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.047+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860935b0 con 0x555f860861f0 2024-03-28T16:49:19.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.051+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93868759003472] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.051+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.051+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93868759008208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86094d60 con 0x555f860861f0 2024-03-28T16:49:19.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.051+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860956d0 con 0x555f8608f360 2024-03-28T16:49:19.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.051+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.051+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93868759011888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86095ed0 con 0x555f8608f360 2024-03-28T16:49:19.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.051+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86096840 con 0x7f831801fc60 2024-03-28T16:49:19.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93868759008208] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93868759016352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860967c0 con 0x7f831801fc60 2024-03-28T16:49:19.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86097530 con 0x555f860861f0 2024-03-28T16:49:19.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93868759011888] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93868759016352] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93868759013936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86097010 con 0x555f860861f0 2024-03-28T16:49:19.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.055+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86097d80 con 0x555f8608a760 2024-03-28T16:49:19.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93868759013936] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93868759021792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86096f50 con 0x555f8608a760 2024-03-28T16:49:19.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86098580 con 0x555f8608a760 2024-03-28T16:49:19.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93868759024320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86098d20 con 0x555f8608a760 2024-03-28T16:49:19.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860997f0 con 0x555f8608a760 2024-03-28T16:49:19.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.059+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 93868759021792] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93868759028432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860997b0 con 0x555f8608a760 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=0 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=1 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=2 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=3 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=4 2024-03-28T16:49:19.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=5 2024-03-28T16:49:19.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93868759024320] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=6 2024-03-28T16:49:19.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93868759028432] v29'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 20 add_watcher() i=7 2024-03-28T16:49:19.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:19.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860987c0 con 0x555f860861f0 2024-03-28T16:49:19.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.063+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93868759025536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86099c70 con 0x555f860861f0 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93868759025536] v29'106 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f81dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 5 note: GC not initialized 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f8157fff640 20 reqs_thread_entry: start 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 Filter name: none 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f8155ffb640 20 reqs_thread_entry: start 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860be720 con 0x555f8608a760 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860be720 con 0x555f8608a760 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:49:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:49:19.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.067+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860bd410 con 0x555f8608a760 2024-03-28T16:49:19.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v29'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f832a0ba980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-03-28T16:49:19.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f832a0ba980 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:19.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f832a0ba980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-03-28T16:49:19.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 93868759177168 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860c1c80 con 0x555f8608a760 2024-03-28T16:49:19.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify (1) cookie 93868759021792 notify 124554051587 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x555f86074b50 con 0x555f8608a760 2024-03-28T16:49:19.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f820a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051587 cookie 93868759021792 notifier 4480 bl.length()=522 2024-03-28T16:49:19.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(65 notify.5 [notify cookie 93868759177168 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f820a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-03-28T16:49:19.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f820a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:19.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f820a7fc640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f81ec003ea0 con 0x555f8608a760 2024-03-28T16:49:19.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== watch-notify(notify_complete (2) cookie 93868759177168 notify 124554051587 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f83200f9040 con 0x555f8608a760 2024-03-28T16:49:19.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.071+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860bd7d0 con 0x555f8608a760 2024-03-28T16:49:19.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v29'3 uv3 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f832a0ba980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:49:19.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f832a0ba980 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:19.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f832a0ba980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 93868759177168 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860c1c30 con 0x555f8608f360 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== watch-notify(notify (1) cookie 93868759011888 notify 124554051592 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7f8318046040 con 0x555f8608f360 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f820a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051592 cookie 93868759011888 notifier 4480 bl.length()=177 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f820a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f820a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(68 notify.2 [notify cookie 93868759177168 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f820a7fc640 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f81ec003ea0 con 0x555f8608f360 2024-03-28T16:49:19.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== watch-notify(notify_complete (2) cookie 93868759177168 notify 124554051592 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555f85ea24c0 con 0x555f8608f360 2024-03-28T16:49:19.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.077 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:19.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f81dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.075+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93868759025536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86083950 con 0x555f860861f0 2024-03-28T16:49:19.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.079+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93868759025536] v29'107 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.079+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93868759003472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86098780 con 0x7f831801fc60 2024-03-28T16:49:19.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.079+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93868759003472] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.079+0000 7f832a0ba980 20 remove_watcher() i=0 2024-03-28T16:49:19.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.079+0000 7f832a0ba980 2 removed watcher, disabling cache 2024-03-28T16:49:19.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.079+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93868759008208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f86093150 con 0x555f860861f0 2024-03-28T16:49:19.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93868759008208] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f832a0ba980 20 remove_watcher() i=1 2024-03-28T16:49:19.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93868759011888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860943d0 con 0x555f8608f360 2024-03-28T16:49:19.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f83273d4640 1 -- 172.21.15.170:0/3053350969 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93868759011888] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8318033040 con 0x555f8608f360 2024-03-28T16:49:19.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f832a0ba980 20 remove_watcher() i=2 2024-03-28T16:49:19.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93868759016352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860943d0 con 0x7f831801fc60 2024-03-28T16:49:19.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93868759016352] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x7f831801fc60 2024-03-28T16:49:19.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f832a0ba980 20 remove_watcher() i=3 2024-03-28T16:49:19.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.083+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93868759013936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860963a0 con 0x555f860861f0 2024-03-28T16:49:19.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.087+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93868759013936] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83100083f0 con 0x555f860861f0 2024-03-28T16:49:19.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.087+0000 7f832a0ba980 20 remove_watcher() i=4 2024-03-28T16:49:19.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.087+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93868759021792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860943d0 con 0x555f8608a760 2024-03-28T16:49:19.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.087+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93868759021792] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.087+0000 7f832a0ba980 20 remove_watcher() i=5 2024-03-28T16:49:19.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.087+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93868759024320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860963a0 con 0x555f8608a760 2024-03-28T16:49:19.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.091+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93868759024320] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.091+0000 7f832a0ba980 20 remove_watcher() i=6 2024-03-28T16:49:19.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.091+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93868759028432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x555f860943d0 con 0x555f8608a760 2024-03-28T16:49:19.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.091+0000 7f8327bd5640 1 -- 172.21.15.170:0/3053350969 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93868759028432] v29'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f83200c5020 con 0x555f8608a760 2024-03-28T16:49:19.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.091+0000 7f832a0ba980 20 remove_watcher() i=7 2024-03-28T16:49:19.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f8608a760 msgr2=0x555f8608cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f8608a760 0x555f8608cb50 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f860861f0 msgr2=0x555f860886b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f860861f0 0x555f860886b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f831801fc60 msgr2=0x7f831803a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f831801fc60 0x7f831803a3c0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555f8608f360 msgr2=0x555f86091820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x555f8608f360 0x555f86091820 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f81f0034300 msgr2=0x7f81f00367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f81f0034300 0x7f81f00367c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f831800c010 tx=0x7f8318010000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 msgr2=0x555f8607b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f83283d6640 1 -- 172.21.15.170:0/3053350969 reap_dead start 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f8607b000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f83200e1540 tx=0x7f83200c9f20 comp rx=0 tx=0).stop 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 shutdown_connections 2024-03-28T16:49:19.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3053350969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86074160 0x555f8607b000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 >> 172.21.15.170:0/3053350969 conn(0x555f860704f0 msgr2=0x555f8606db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 shutdown_connections 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3053350969 wait complete. 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f82000342b0 msgr2=0x7f8200036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f82000342b0 0x7f8200036770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x555f86051ab0 tx=0x7f831800f000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86067230 msgr2=0x555f86067650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86067230 0x555f86067650 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83200e2770 tx=0x7f83200b4860 comp rx=0 tx=0).stop 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 shutdown_connections 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f82000342b0 0x7f8200036770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/3622594854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f86067230 0x555f86067650 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 >> 172.21.15.170:0/3622594854 conn(0x555f8605c040 msgr2=0x555f8605cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 shutdown_connections 2024-03-28T16:49:19.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/3622594854 wait complete. 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f860528a0 msgr2=0x555f86054cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x555f860528a0 0x555f86054cc0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f8604e2c0 msgr2=0x555f860506b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x555f8604e2c0 0x555f860506b0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555f86056850 msgr2=0x555f86058c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x555f86056850 0x555f86058c40 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f820c0342b0 msgr2=0x7f820c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f820c0342b0 0x7f820c036770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8318000f70 tx=0x7f8318001380 comp rx=0 tx=0).stop 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 msgr2=0x555f86049650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 --2- 172.21.15.170:0/740033857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555f85f58a90 0x555f86049650 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f832001b0d0 tx=0x7f8320017e30 comp rx=0 tx=0).stop 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f83283d6640 1 -- 172.21.15.170:0/740033857 reap_dead start 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 shutdown_connections 2024-03-28T16:49:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 >> 172.21.15.170:0/740033857 conn(0x555f85ecb890 msgr2=0x555f8603ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 shutdown_connections 2024-03-28T16:49:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.095+0000 7f832a0ba980 1 -- 172.21.15.170:0/740033857 wait complete. 2024-03-28T16:49:19.106 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:19.114 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-03-28T16:49:19.120 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:19.120 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-03-28T16:49:19.120 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-03-28T16:49:19.120 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-03-28T16:49:19.183 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:19.183 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff88b7c7980 1 Processor -- start 2024-03-28T16:49:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff88b7c7980 1 -- start start 2024-03-28T16:49:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff88b7c7980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5650f3c27130 0x5650f3c27510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff88b7c7980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5650f3b71520 con 0x5650f3c27a50 2024-03-28T16:49:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff88b7c7980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5650f3b380f0 con 0x5650f3c27130 2024-03-28T16:49:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff8892e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5650f3c27130 0x5650f3c27510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff8892e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5650f3c27130 0x5650f3c27510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56626/0 (socket says 172.21.15.170:56626) 2024-03-28T16:49:19.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.183+0000 7ff8892e2640 1 -- 172.21.15.170:0/1823689620 learned_addr learned my addr 172.21.15.170:0/1823689620 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff8892e2640 1 -- 172.21.15.170:0/1823689620 >> v1:172.21.15.119:6789/0 conn(0x5650f3c27a50 legacy=0x5650f3c28400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff8892e2640 1 -- 172.21.15.170:0/1823689620 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f3c28c30 con 0x5650f3c27130 2024-03-28T16:49:19.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff8892e2640 1 --2- 172.21.15.170:0/1823689620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3c27510 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00a5c0 tx=0x7ff87c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca58b60362cbc78f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff87bfff640 1 -- 172.21.15.170:0/1823689620 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0aa470 con 0x5650f3c27130 2024-03-28T16:49:19.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff87bfff640 1 -- 172.21.15.170:0/1823689620 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0aa5d0 con 0x5650f3c27130 2024-03-28T16:49:19.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff87bfff640 1 -- 172.21.15.170:0/1823689620 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0aa890 con 0x5650f3c27130 2024-03-28T16:49:19.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff88b7c7980 1 -- 172.21.15.170:0/1823689620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 msgr2=0x5650f3c27510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/1823689620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3c27510 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00a5c0 tx=0x7ff87c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:19.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff88b7c7980 1 -- 172.21.15.170:0/1823689620 shutdown_connections 2024-03-28T16:49:19.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/1823689620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3c27510 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff88b7c7980 1 -- 172.21.15.170:0/1823689620 >> 172.21.15.170:0/1823689620 conn(0x5650f3b9a850 msgr2=0x5650f3b9ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff88b7c7980 1 -- 172.21.15.170:0/1823689620 shutdown_connections 2024-03-28T16:49:19.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.187+0000 7ff88b7c7980 1 -- 172.21.15.170:0/1823689620 wait complete. 2024-03-28T16:49:19.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.203+0000 7ff88b7c7980 1 Processor -- start 2024-03-28T16:49:19.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.203+0000 7ff88b7c7980 1 -- start start 2024-03-28T16:49:19.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.203+0000 7ff88b7c7980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3d11630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.203+0000 7ff88b7c7980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f3b35620 con 0x5650f3c27130 2024-03-28T16:49:19.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3d11630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3d11630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56636/0 (socket says 172.21.15.170:56636) 2024-03-28T16:49:19.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 -- 172.21.15.170:0/2942494511 learned_addr learned my addr 172.21.15.170:0/2942494511 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 -- 172.21.15.170:0/2942494511 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c005df0 con 0x5650f3c27130 2024-03-28T16:49:19.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 --2- 172.21.15.170:0/2942494511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3d11630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0ac040 tx=0x7ff87c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed2b17279b8ba1e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff777606640 1 -- 172.21.15.170:0/2942494511 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0aa9b0 con 0x5650f3c27130 2024-03-28T16:49:19.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff777606640 1 -- 172.21.15.170:0/2942494511 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0aab30 con 0x5650f3c27130 2024-03-28T16:49:19.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff777606640 1 -- 172.21.15.170:0/2942494511 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0aae10 con 0x5650f3c27130 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2942494511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 msgr2=0x5650f3d11630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2942494511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3d11630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0ac040 tx=0x7ff87c003030 comp rx=0 tx=0).stop 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2942494511 shutdown_connections 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2942494511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27130 0x5650f3d11630 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2942494511 >> 172.21.15.170:0/2942494511 conn(0x5650f3b9a850 msgr2=0x5650f3d0d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2942494511 shutdown_connections 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2942494511 wait complete. 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 Processor -- start 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 -- start start 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27a50 0x5650f3d18580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff88b7c7980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f3b30e00 con 0x5650f3c27a50 2024-03-28T16:49:19.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27a50 0x5650f3d18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27a50 0x5650f3d18580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56644/0 (socket says 172.21.15.170:56644) 2024-03-28T16:49:19.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 -- 172.21.15.170:0/2139815343 learned_addr learned my addr 172.21.15.170:0/2139815343 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.207+0000 7ff8892e2640 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f3c28c30 con 0x5650f3c27a50 2024-03-28T16:49:19.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff8892e2640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27a50 0x5650f3d18580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005940 tx=0x7ff87c0180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff775e03640 1 -- 172.21.15.170:0/2139815343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0aa950 con 0x5650f3c27a50 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650f3d18ac0 con 0x5650f3c27a50 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff775e03640 1 -- 172.21.15.170:0/2139815343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0c6690 con 0x5650f3c27a50 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff775e03640 1 -- 172.21.15.170:0/2139815343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0c6930 con 0x5650f3c27a50 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f3d18ef0 con 0x5650f3c27a50 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff775e03640 1 -- 172.21.15.170:0/2139815343 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff87c0c6a90 con 0x5650f3c27a50 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff775e03640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7700342b0 0x7ff770036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff775e03640 1 -- 172.21.15.170:0/2139815343 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7ff87c0e4200 con 0x5650f3c27a50 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff888ae1640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7700342b0 0x7ff770036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d1d1a0 0x5650f3d1f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d1fad0 con 0x5650f3d1d1a0 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff889ae3640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d1d1a0 0x5650f3d1f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff888ae1640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7700342b0 0x7ff770036770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff880001e20 tx=0x7ff8800013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff889ae3640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d1d1a0 0x5650f3d1f590 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff889ae3640 1 -- 172.21.15.170:0/2139815343 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d1d1a0 2024-03-28T16:49:19.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d1fe30 con 0x5650f3d1d1a0 2024-03-28T16:49:19.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff889ae3640 1 -- 172.21.15.170:0/2139815343 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d1d1a0 2024-03-28T16:49:19.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d21b50 0x5650f3d23f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d24480 con 0x5650f3d21b50 2024-03-28T16:49:19.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.211+0000 7ff8892e2640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d21b50 0x5650f3d23f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff8892e2640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d21b50 0x5650f3d23f40 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff8892e2640 1 -- 172.21.15.170:0/2139815343 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7ff87c0b0020 con 0x5650f3d21b50 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d20990 con 0x5650f3d1d1a0 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff889ae3640 1 -- 172.21.15.170:0/2139815343 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d1d1a0 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5650f3d257f0 0x5650f3d27be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d28230 con 0x5650f3d257f0 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff888ae1640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5650f3d257f0 0x5650f3d27be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff888ae1640 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5650f3d257f0 0x5650f3d27be0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff888ae1640 1 -- 172.21.15.170:0/2139815343 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7ff880010630 con 0x5650f3d257f0 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff88b7c7980 1 Processor -- start 2024-03-28T16:49:19.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff88b7c7980 1 -- start start 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff88b7c7980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d36160 0x5650f3d36580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff88b7c7980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f3ac9e30 con 0x5650f3d36160 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d36160 0x5650f3d36580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d36160 0x5650f3d36580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56658/0 (socket says 172.21.15.170:56658) 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff8892e2640 1 -- 172.21.15.170:0/325899136 learned_addr learned my addr 172.21.15.170:0/325899136 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.215+0000 7ff8892e2640 1 -- 172.21.15.170:0/325899136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f3d36ac0 con 0x5650f3d36160 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff8892e2640 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d36160 0x5650f3d36580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0e2830 tx=0x7ff87c0cf5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff76effd640 1 -- 172.21.15.170:0/325899136 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0e2480 con 0x5650f3d36160 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff76effd640 1 -- 172.21.15.170:0/325899136 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0df040 con 0x5650f3d36160 2024-03-28T16:49:19.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650f3d36cb0 con 0x5650f3d36160 2024-03-28T16:49:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff76effd640 1 -- 172.21.15.170:0/325899136 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0c53e0 con 0x5650f3d36160 2024-03-28T16:49:19.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f3d36cb0 con 0x5650f3d36160 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff76effd640 1 -- 172.21.15.170:0/325899136 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff87c0c5540 con 0x5650f3d36160 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff76effd640 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7600342b0 0x7ff760036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff76effd640 1 -- 172.21.15.170:0/325899136 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7ff87c0d20c0 con 0x5650f3d36160 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff888ae1640 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7600342b0 0x7ff760036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff88b7c7980 1 Processor -- start 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff88b7c7980 1 -- start start 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff888ae1640 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7600342b0 0x7ff760036770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5650f3d20cf0 tx=0x7ff880006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff88b7c7980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d43160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff88b7c7980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f3abd790 con 0x5650f3d42d40 2024-03-28T16:49:19.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d43160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d43160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56660/0 (socket says 172.21.15.170:56660) 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff8892e2640 1 -- 172.21.15.170:0/2323337752 learned_addr learned my addr 172.21.15.170:0/2323337752 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff8892e2640 1 -- 172.21.15.170:0/2323337752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f3d43730 con 0x5650f3d42d40 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.219+0000 7ff8892e2640 1 --2- 172.21.15.170:0/2323337752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d43160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0cfa60 tx=0x7ff87c0d0440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb41a0f8365b8532 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff76cff9640 1 -- 172.21.15.170:0/2323337752 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0fc070 con 0x5650f3d42d40 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff76cff9640 1 -- 172.21.15.170:0/2323337752 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0cecb0 con 0x5650f3d42d40 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff76cff9640 1 -- 172.21.15.170:0/2323337752 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0e28e0 con 0x5650f3d42d40 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2323337752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 msgr2=0x5650f3d43160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2323337752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d43160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0cfa60 tx=0x7ff87c0d0440 comp rx=0 tx=0).stop 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2323337752 shutdown_connections 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2323337752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d43160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2323337752 >> 172.21.15.170:0/2323337752 conn(0x5650f3d3f0d0 msgr2=0x5650f3d3d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2323337752 shutdown_connections 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2323337752 wait complete. 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 Processor -- start 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- start start 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d49c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f3d4a140 con 0x5650f3d42d40 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d49c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff8892e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d49c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56670/0 (socket says 172.21.15.170:56670) 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 learned_addr learned my addr 172.21.15.170:0/3389070756 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0e8030 con 0x5650f3d42d40 2024-03-28T16:49:19.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff8892e2640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d49c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff87c103040 tx=0x7ff87c0eb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff75effd640 1 -- 172.21.15.170:0/3389070756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c112070 con 0x5650f3d42d40 2024-03-28T16:49:19.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650f3d4a340 con 0x5650f3d42d40 2024-03-28T16:49:19.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff75effd640 1 -- 172.21.15.170:0/3389070756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0e2dd0 con 0x5650f3d42d40 2024-03-28T16:49:19.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff75effd640 1 -- 172.21.15.170:0/3389070756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87c0efe10 con 0x5650f3d42d40 2024-03-28T16:49:19.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650f3d4a340 con 0x5650f3d42d40 2024-03-28T16:49:19.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.223+0000 7ff75effd640 1 -- 172.21.15.170:0/3389070756 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff87c107050 con 0x5650f3d42d40 2024-03-28T16:49:19.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff75effd640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7540342b0 0x7ff754036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff75effd640 1 -- 172.21.15.170:0/3389070756 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7ff87c118b00 con 0x5650f3d42d40 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff888ae1640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7540342b0 0x7ff754036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d54e80 0x5650f3d57320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d57a70 con 0x5650f3d54e80 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff889ae3640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d54e80 0x5650f3d57320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff888ae1640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7540342b0 0x7ff754036770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5650f3d285f0 tx=0x7ff880011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff889ae3640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d54e80 0x5650f3d57320 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 realm 2024-03-28T16:49:19.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d57e40 con 0x5650f3d54e80 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d58a20 con 0x5650f3d54e80 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d593d0 0x5650f3d5b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d5bd00 con 0x5650f3d593d0 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff8892e2640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d593d0 0x5650f3d5b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.227+0000 7ff8892e2640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d593d0 0x5650f3d5b7c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7ff87c114070 con 0x5650f3d593d0 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff88b7c7980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff88b7c7980 20 searching for the correct realm 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5650f3d5db90 0x5650f3d60050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d60720 con 0x5650f3d5db90 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff888ae1640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5650f3d5db90 0x5650f3d60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff888ae1640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5650f3d5db90 0x5650f3d60050 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff888ae1640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff88000f3d0 0x7ff88000f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d60720 con 0x7ff88000f3d0 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff88000f3d0 0x7ff88000f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff88000f3d0 0x7ff88000f820 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.231+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x7ff88000f3d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x7ff88000f3d0 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.235+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x7ff88000f3d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.239+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x7ff88000f3d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d5db90 2024-03-28T16:49:19.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d593d0 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x5650f3d54e80 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d586c0 con 0x7ff88000f3d0 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:19.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d5c500 con 0x5650f3d54e80 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d5c500 con 0x5650f3d54e80 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.243+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d5c500 con 0x7ff88000f3d0 2024-03-28T16:49:19.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 zone default found 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 4 Realm: () 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 zonegroup default 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d5cd50 con 0x5650f3d593d0 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d5d130 con 0x5650f3d5db90 2024-03-28T16:49:19.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d62690 con 0x5650f3d5db90 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 started sync module instance, tier type = 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d61e70 con 0x7ff88000f3d0 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94905688267216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d62220 con 0x7ff88000f3d0 2024-03-28T16:49:19.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.247+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d633f0 con 0x5650f3d54e80 2024-03-28T16:49:19.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94905688267216] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94905688271536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d63410 con 0x5650f3d54e80 2024-03-28T16:49:19.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d63e80 con 0x5650f3d5db90 2024-03-28T16:49:19.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94905688271536] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94905688275296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d64230 con 0x5650f3d5db90 2024-03-28T16:49:19.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.251+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d64ba0 con 0x7ff88000f3d0 2024-03-28T16:49:19.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94905688279088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d653f0 con 0x7ff88000f3d0 2024-03-28T16:49:19.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d65de0 con 0x5650f3d54e80 2024-03-28T16:49:19.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94905688275296] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94905688279088] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94905688283328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d64f00 con 0x5650f3d54e80 2024-03-28T16:49:19.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.255+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d66630 con 0x5650f3d593d0 2024-03-28T16:49:19.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94905688283328] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94905688285584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d64f00 con 0x5650f3d593d0 2024-03-28T16:49:19.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d66e30 con 0x5650f3d593d0 2024-03-28T16:49:19.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94905688285584] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94905688272592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d67630 con 0x5650f3d593d0 2024-03-28T16:49:19.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.259+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d67fa0 con 0x5650f3d593d0 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94905688272592] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94905688290160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d67fa0 con 0x5650f3d593d0 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=0 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=1 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=2 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=3 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=4 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=5 2024-03-28T16:49:19.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=6 2024-03-28T16:49:19.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94905688290160] v29'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 20 add_watcher() i=7 2024-03-28T16:49:19.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:19.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d6b390 con 0x5650f3d54e80 2024-03-28T16:49:19.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.263+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94905688292560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d68870 con 0x5650f3d54e80 2024-03-28T16:49:19.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94905688292560] v29'108 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff73f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:19.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:19.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 5 note: GC not initialized 2024-03-28T16:49:19.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff73e7e4640 20 reqs_thread_entry: start 2024-03-28T16:49:19.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:19.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 20 Filter name: none 2024-03-28T16:49:19.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff6affff640 20 reqs_thread_entry: start 2024-03-28T16:49:19.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d8cbf0 con 0x7ff88000f3d0 2024-03-28T16:49:19.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:19.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff73f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.267+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94905688292560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d4eaf0 con 0x5650f3d54e80 2024-03-28T16:49:19.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94905688292560] v29'109 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94905688267216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d67ca0 con 0x7ff88000f3d0 2024-03-28T16:49:19.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94905688267216] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff88b7c7980 20 remove_watcher() i=0 2024-03-28T16:49:19.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff88b7c7980 2 removed watcher, disabling cache 2024-03-28T16:49:19.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94905688271536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d619d0 con 0x5650f3d54e80 2024-03-28T16:49:19.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94905688271536] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff88b7c7980 20 remove_watcher() i=1 2024-03-28T16:49:19.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.271+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94905688275296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d619d0 con 0x5650f3d5db90 2024-03-28T16:49:19.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.275+0000 7ff888ae1640 1 -- 172.21.15.170:0/3389070756 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94905688275296] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff880032040 con 0x5650f3d5db90 2024-03-28T16:49:19.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.275+0000 7ff88b7c7980 20 remove_watcher() i=2 2024-03-28T16:49:19.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.275+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94905688279088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d63960 con 0x7ff88000f3d0 2024-03-28T16:49:19.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.275+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94905688279088] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x7ff88000f3d0 2024-03-28T16:49:19.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.275+0000 7ff88b7c7980 20 remove_watcher() i=3 2024-03-28T16:49:19.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.275+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94905688283328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d62ab0 con 0x5650f3d54e80 2024-03-28T16:49:19.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.279+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94905688283328] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff88400a4d0 con 0x5650f3d54e80 2024-03-28T16:49:19.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.279+0000 7ff88b7c7980 20 remove_watcher() i=4 2024-03-28T16:49:19.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.279+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94905688285584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d62ab0 con 0x5650f3d593d0 2024-03-28T16:49:19.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.279+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94905688285584] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.279+0000 7ff88b7c7980 20 remove_watcher() i=5 2024-03-28T16:49:19.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.279+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94905688272592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d66190 con 0x5650f3d593d0 2024-03-28T16:49:19.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94905688272592] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 20 remove_watcher() i=6 2024-03-28T16:49:19.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94905688290160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5650f3d66190 con 0x5650f3d593d0 2024-03-28T16:49:19.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff8892e2640 1 -- 172.21.15.170:0/3389070756 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94905688290160] v29'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff87c11d040 con 0x5650f3d593d0 2024-03-28T16:49:19.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 20 remove_watcher() i=7 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d593d0 msgr2=0x5650f3d5b7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d593d0 0x5650f3d5b7c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d54e80 msgr2=0x5650f3d57320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d54e80 0x5650f3d57320 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff88000f3d0 msgr2=0x7ff88000f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff88000f3d0 0x7ff88000f820 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5650f3d5db90 msgr2=0x5650f3d60050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5650f3d5db90 0x5650f3d60050 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7540342b0 msgr2=0x7ff754036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7540342b0 0x7ff754036770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5650f3d285f0 tx=0x7ff880011000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 msgr2=0x5650f3d49c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/3389070756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d42d40 0x5650f3d49c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff87c103040 tx=0x7ff87c0eb000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 reap_dead start 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff889ae3640 1 -- 172.21.15.170:0/3389070756 reap_dead start 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 shutdown_connections 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 >> 172.21.15.170:0/3389070756 conn(0x5650f3d3f0d0 msgr2=0x5650f3d3c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 shutdown_connections 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/3389070756 wait complete. 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7600342b0 msgr2=0x7ff760036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7600342b0 0x7ff760036770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5650f3d20cf0 tx=0x7ff880006000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d36160 msgr2=0x5650f3d36580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.283+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d36160 0x5650f3d36580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0e2830 tx=0x7ff87c0cf5c0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 shutdown_connections 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7600342b0 0x7ff760036770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/325899136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3d36160 0x5650f3d36580 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 >> 172.21.15.170:0/325899136 conn(0x5650f3d2af70 msgr2=0x5650f3d2ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 shutdown_connections 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/325899136 wait complete. 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d21b50 msgr2=0x5650f3d23f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5650f3d21b50 0x5650f3d23f40 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d1d1a0 msgr2=0x5650f3d1f590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5650f3d1d1a0 0x5650f3d1f590 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5650f3d257f0 msgr2=0x5650f3d27be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5650f3d257f0 0x5650f3d27be0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7700342b0 msgr2=0x7ff770036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7700342b0 0x7ff770036770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff880001e20 tx=0x7ff8800013a0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27a50 msgr2=0x5650f3d18580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 --2- 172.21.15.170:0/2139815343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650f3c27a50 0x5650f3d18580 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005940 tx=0x7ff87c0180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff889ae3640 1 -- 172.21.15.170:0/2139815343 reap_dead start 2024-03-28T16:49:19.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 shutdown_connections 2024-03-28T16:49:19.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 >> 172.21.15.170:0/2139815343 conn(0x5650f3b9a850 msgr2=0x5650f3d0bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 shutdown_connections 2024-03-28T16:49:19.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.287+0000 7ff88b7c7980 1 -- 172.21.15.170:0/2139815343 wait complete. 2024-03-28T16:49:19.297 INFO:teuthology.orchestra.run.smithi170.stdout:{"key":"bucket:myfoo","ver":{"tag":"_LXvi00GMHyHMS1-z9v2t5rH","ver":2},"mtime":"2024-03-28T16:49:18.869052Z","data":{"bucket":{"name":"myfoo","marker":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","bucket_id":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","tenant":"","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"foo","creation_time":"2024-03-28T16:49:17.311675Z","linked":false,"has_bucket_info":false}} 2024-03-28T16:49:19.297 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_LXvi00GMHyHMS1-z9v2t5rH', 'ver': 2}, 'mtime': '2024-03-28T16:49:18.869052Z', 'data': {'bucket': {'name': 'myfoo', 'marker': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'bucket_id': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-03-28T16:49:17.311675Z', 'linked': False, 'has_bucket_info': False}} 2024-03-28T16:49:19.297 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1'] 2024-03-28T16:49:19.297 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1'] 2024-03-28T16:49:19.298 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:19.376 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:19.376 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:19.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.375+0000 7efe40bc3980 1 Processor -- start 2024-03-28T16:49:19.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.375+0000 7efe40bc3980 1 -- start start 2024-03-28T16:49:19.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.375+0000 7efe40bc3980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5649d7c076b0 0x5649d7c09aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.375+0000 7efe40bc3980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5649d7b4d4c0 con 0x5649d7b75010 2024-03-28T16:49:19.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.375+0000 7efe40bc3980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5649d7b140f0 con 0x5649d7c076b0 2024-03-28T16:49:19.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3dedd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5649d7c076b0 0x5649d7c09aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3e6de640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5649d7b75010 0x5649d7c05f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38196/0 (socket says 172.21.15.170:38196) 2024-03-28T16:49:19.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3e6de640 1 -- 172.21.15.170:0/3687212647 learned_addr learned my addr 172.21.15.170:0/3687212647 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:19.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3dedd640 1 -- 172.21.15.170:0/3687212647 >> v1:172.21.15.119:6789/0 conn(0x5649d7b75010 legacy=0x5649d7c05f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3dedd640 1 -- 172.21.15.170:0/3687212647 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d7c0aff0 con 0x5649d7c076b0 2024-03-28T16:49:19.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3dedd640 1 --2- 172.21.15.170:0/3687212647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7c076b0 0x5649d7c09aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe3400a480 tx=0x7efe3401b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b6c97ed1f3038 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3d6dc640 1 -- 172.21.15.170:0/3687212647 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe340ab540 con 0x5649d7c076b0 2024-03-28T16:49:19.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3d6dc640 1 -- 172.21.15.170:0/3687212647 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe340ab6a0 con 0x5649d7c076b0 2024-03-28T16:49:19.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe3d6dc640 1 -- 172.21.15.170:0/3687212647 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe340ab960 con 0x5649d7c076b0 2024-03-28T16:49:19.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe40bc3980 1 -- 172.21.15.170:0/3687212647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7c076b0 msgr2=0x5649d7c09aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.379+0000 7efe40bc3980 1 --2- 172.21.15.170:0/3687212647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7c076b0 0x5649d7c09aa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe3400a480 tx=0x7efe3401b280 comp rx=0 tx=0).stop 2024-03-28T16:49:19.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.383+0000 7efe40bc3980 1 -- 172.21.15.170:0/3687212647 shutdown_connections 2024-03-28T16:49:19.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.383+0000 7efe40bc3980 1 --2- 172.21.15.170:0/3687212647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7c076b0 0x5649d7c09aa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.383+0000 7efe40bc3980 1 -- 172.21.15.170:0/3687212647 >> 172.21.15.170:0/3687212647 conn(0x5649d7bfb670 msgr2=0x5649d7bfda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.383+0000 7efe40bc3980 1 -- 172.21.15.170:0/3687212647 shutdown_connections 2024-03-28T16:49:19.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.383+0000 7efe40bc3980 1 -- 172.21.15.170:0/3687212647 wait complete. 2024-03-28T16:49:19.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.395+0000 7efe40bc3980 1 Processor -- start 2024-03-28T16:49:19.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.395+0000 7efe40bc3980 1 -- start start 2024-03-28T16:49:19.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7ced250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d7b11620 con 0x5649d7b75010 2024-03-28T16:49:19.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7ced250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7ced250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56558/0 (socket says 172.21.15.170:56558) 2024-03-28T16:49:19.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 -- 172.21.15.170:0/204226116 learned_addr learned my addr 172.21.15.170:0/204226116 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 -- 172.21.15.170:0/204226116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe34005d30 con 0x5649d7b75010 2024-03-28T16:49:19.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 --2- 172.21.15.170:0/204226116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7ced250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efe38001c90 tx=0x7efe3800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e88c937f6739c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efd2c608640 1 -- 172.21.15.170:0/204226116 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe3800e690 con 0x5649d7b75010 2024-03-28T16:49:19.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efd2c608640 1 -- 172.21.15.170:0/204226116 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3800e7f0 con 0x5649d7b75010 2024-03-28T16:49:19.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efd2c608640 1 -- 172.21.15.170:0/204226116 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe3800eaf0 con 0x5649d7b75010 2024-03-28T16:49:19.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- 172.21.15.170:0/204226116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 msgr2=0x5649d7ced250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 --2- 172.21.15.170:0/204226116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7ced250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efe38001c90 tx=0x7efe3800b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- 172.21.15.170:0/204226116 shutdown_connections 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 --2- 172.21.15.170:0/204226116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7ced250 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- 172.21.15.170:0/204226116 >> 172.21.15.170:0/204226116 conn(0x5649d7bfb670 msgr2=0x5649d7ce8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- 172.21.15.170:0/204226116 shutdown_connections 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- 172.21.15.170:0/204226116 wait complete. 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 Processor -- start 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- start start 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7cf42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe40bc3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d7b0ce00 con 0x5649d7b75010 2024-03-28T16:49:19.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7cf42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7cf42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56560/0 (socket says 172.21.15.170:56560) 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.399+0000 7efe3e6de640 1 -- 172.21.15.170:0/365063403 learned_addr learned my addr 172.21.15.170:0/365063403 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3e6de640 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d7b4d4c0 con 0x5649d7b75010 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3e6de640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7cf42f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efe38001ee0 tx=0x7efe38000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efd2ae05640 1 -- 172.21.15.170:0/365063403 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe3801f070 con 0x5649d7b75010 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5649d7cf4830 con 0x5649d7b75010 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efd2ae05640 1 -- 172.21.15.170:0/365063403 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3801a4d0 con 0x5649d7b75010 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efd2ae05640 1 -- 172.21.15.170:0/365063403 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe3801a770 con 0x5649d7b75010 2024-03-28T16:49:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649d7cf4c60 con 0x5649d7b75010 2024-03-28T16:49:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efd2ae05640 1 -- 172.21.15.170:0/365063403 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7efe3800a040 con 0x5649d7b75010 2024-03-28T16:49:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efd2ae05640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd240342b0 0x7efd24036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efd2ae05640 1 -- 172.21.15.170:0/365063403 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7efe380494b0 con 0x5649d7b75010 2024-03-28T16:49:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3dedd640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd240342b0 0x7efd24036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7c076b0 0x5649d7cfaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7cfb4b0 con 0x5649d7c076b0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3eedf640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7c076b0 0x5649d7cfaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3dedd640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd240342b0 0x7efd24036770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe340ad000 tx=0x7efe340a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3eedf640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7c076b0 0x5649d7cfaf70 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3eedf640 1 -- 172.21.15.170:0/365063403 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7c076b0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7cfb810 con 0x5649d7c076b0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe3eedf640 1 -- 172.21.15.170:0/365063403 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7c076b0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7cfd1a0 0x5649d7cff590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.403+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7cffd00 con 0x5649d7cfd1a0 2024-03-28T16:49:19.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3e6de640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7cfd1a0 0x5649d7cff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3e6de640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7cfd1a0 0x5649d7cff590 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3e6de640 1 -- 172.21.15.170:0/365063403 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7efe38014070 con 0x5649d7cfd1a0 2024-03-28T16:49:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7cfc010 con 0x5649d7c076b0 2024-03-28T16:49:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3eedf640 1 -- 172.21.15.170:0/365063403 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7c076b0 2024-03-28T16:49:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5649d7d01120 0x5649d7d03510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d03bd0 con 0x5649d7d01120 2024-03-28T16:49:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3dedd640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5649d7d01120 0x5649d7d03510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3dedd640 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5649d7d01120 0x5649d7d03510 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3dedd640 1 -- 172.21.15.170:0/365063403 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d01120 2024-03-28T16:49:19.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe40bc3980 1 Processor -- start 2024-03-28T16:49:19.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe40bc3980 1 -- start start 2024-03-28T16:49:19.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe40bc3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d11aa0 0x5649d7d11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe40bc3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d7aa5e30 con 0x5649d7d11aa0 2024-03-28T16:49:19.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d11aa0 0x5649d7d11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d11aa0 0x5649d7d11ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56572/0 (socket says 172.21.15.170:56572) 2024-03-28T16:49:19.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.407+0000 7efe3e6de640 1 -- 172.21.15.170:0/4104593792 learned_addr learned my addr 172.21.15.170:0/4104593792 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe3e6de640 1 -- 172.21.15.170:0/4104593792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d7d12400 con 0x5649d7d11aa0 2024-03-28T16:49:19.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe3e6de640 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d11aa0 0x5649d7d11ec0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe38047a80 tx=0x7efe38016a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efd23fff640 1 -- 172.21.15.170:0/4104593792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe38017de0 con 0x5649d7d11aa0 2024-03-28T16:49:19.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5649d7d12650 con 0x5649d7d11aa0 2024-03-28T16:49:19.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efd23fff640 1 -- 172.21.15.170:0/4104593792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38049280 con 0x5649d7d11aa0 2024-03-28T16:49:19.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efd23fff640 1 -- 172.21.15.170:0/4104593792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe38047d00 con 0x5649d7d11aa0 2024-03-28T16:49:19.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649d7d131a0 con 0x5649d7d11aa0 2024-03-28T16:49:19.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efd23fff640 1 -- 172.21.15.170:0/4104593792 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7efe38042a90 con 0x5649d7d11aa0 2024-03-28T16:49:19.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efd23fff640 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd180342b0 0x7efd18036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efd23fff640 1 -- 172.21.15.170:0/4104593792 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7efe38032070 con 0x5649d7d11aa0 2024-03-28T16:49:19.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe3dedd640 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd180342b0 0x7efd18036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe40bc3980 1 Processor -- start 2024-03-28T16:49:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe40bc3980 1 -- start start 2024-03-28T16:49:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe40bc3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d1edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe40bc3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d7a99790 con 0x5649d7d1e9f0 2024-03-28T16:49:19.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe3dedd640 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd180342b0 0x7efd18036770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x5649d7d00090 tx=0x7efe34005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d1edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d1edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56588/0 (socket says 172.21.15.170:56588) 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.411+0000 7efe3e6de640 1 -- 172.21.15.170:0/2995000836 learned_addr learned my addr 172.21.15.170:0/2995000836 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe3e6de640 1 -- 172.21.15.170:0/2995000836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d7d1f3c0 con 0x5649d7d1e9f0 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe3e6de640 1 --2- 172.21.15.170:0/2995000836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d1edf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efe3803e040 tx=0x7efe38023be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946531282d6a1152 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efd21ffb640 1 -- 172.21.15.170:0/2995000836 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe38017650 con 0x5649d7d1e9f0 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efd21ffb640 1 -- 172.21.15.170:0/2995000836 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe380177b0 con 0x5649d7d1e9f0 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efd21ffb640 1 -- 172.21.15.170:0/2995000836 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe38017920 con 0x5649d7d1e9f0 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- 172.21.15.170:0/2995000836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 msgr2=0x5649d7d1edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2995000836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d1edf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efe3803e040 tx=0x7efe38023be0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- 172.21.15.170:0/2995000836 shutdown_connections 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2995000836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d1edf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- 172.21.15.170:0/2995000836 >> 172.21.15.170:0/2995000836 conn(0x5649d7d1ad80 msgr2=0x5649d7d18de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- 172.21.15.170:0/2995000836 shutdown_connections 2024-03-28T16:49:19.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- 172.21.15.170:0/2995000836 wait complete. 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 Processor -- start 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- start start 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d258f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d7d25e30 con 0x5649d7d1e9f0 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe3e6de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56592/0 (socket says 172.21.15.170:56592) 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 learned_addr learned my addr 172.21.15.170:0/2944063978 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d7d131a0 con 0x5649d7d1e9f0 2024-03-28T16:49:19.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe3e6de640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d258f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efe38045d50 tx=0x7efe3804ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efd0ffff640 1 -- 172.21.15.170:0/2944063978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe38065070 con 0x5649d7d1e9f0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5649d7d131a0 con 0x5649d7d1e9f0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efd0ffff640 1 -- 172.21.15.170:0/2944063978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38025c40 con 0x5649d7d1e9f0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efd0ffff640 1 -- 172.21.15.170:0/2944063978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe38017650 con 0x5649d7d1e9f0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.415+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649d7d131a0 con 0x5649d7d1e9f0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efd0ffff640 1 -- 172.21.15.170:0/2944063978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7efe3800d3e0 con 0x5649d7d1e9f0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efd0ffff640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd080342b0 0x7efd08036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efd0ffff640 1 -- 172.21.15.170:0/2944063978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7efe38075af0 con 0x5649d7d1e9f0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3dedd640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd080342b0 0x7efd08036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7d30aa0 0x5649d7d32f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d336b0 con 0x5649d7d30aa0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3eedf640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7d30aa0 0x5649d7d32f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3dedd640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd080342b0 0x7efd08036770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe3401b2c0 tx=0x7efe340a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3eedf640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7d30aa0 0x5649d7d32f60 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 realm 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d33a80 con 0x5649d7d30aa0 2024-03-28T16:49:19.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34660 con 0x5649d7d30aa0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7d35010 0x5649d7d37400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d37a10 con 0x5649d7d35010 2024-03-28T16:49:19.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3e6de640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7d35010 0x5649d7d37400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.419+0000 7efe3e6de640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7d35010 0x5649d7d37400 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe40bc3980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe40bc3980 20 searching for the correct realm 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5649d7d39bf0 0x5649d7d3c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3c780 con 0x5649d7d39bf0 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3dedd640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5649d7d39bf0 0x5649d7d3c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3dedd640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5649d7d39bf0 0x5649d7d3c0b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3dedd640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe340da040 0x7efe340cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3c780 con 0x7efe340da040 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3eedf640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe340da040 0x7efe340cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3eedf640 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe340da040 0x7efe340cc420 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.423+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x7efe340da040 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x7efe340da040 2024-03-28T16:49:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.427+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x7efe340da040 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.431+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x7efe340da040 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d39bf0 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d35010 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x5649d7d30aa0 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d34300 con 0x7efe340da040 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d37ff0 con 0x5649d7d30aa0 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d37ff0 con 0x5649d7d30aa0 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d37fc0 con 0x7efe340da040 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 zone default found 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 4 Realm: () 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:19.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 zonegroup default 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d38840 con 0x5649d7d35010 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.435+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d38c70 con 0x5649d7d39bf0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3e720 con 0x5649d7d39bf0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 20 started sync module instance, tier type = 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3de80 con 0x7efe340da040 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94875153586656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3e230 con 0x7efe340da040 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3f480 con 0x5649d7d30aa0 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94875153586656] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94875153591104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3f400 con 0x5649d7d30aa0 2024-03-28T16:49:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.439+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3fe60 con 0x5649d7d39bf0 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.443+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94875153591104] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.443+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.443+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94875153592160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d3f380 con 0x5649d7d39bf0 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.443+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d40b50 con 0x7efe340da040 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.443+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.443+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94875153598432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d413a0 con 0x7efe340da040 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.443+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d41d50 con 0x5649d7d30aa0 2024-03-28T16:49:19.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94875153592160] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94875153598432] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94875153596944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d40eb0 con 0x5649d7d30aa0 2024-03-28T16:49:19.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d425a0 con 0x5649d7d35010 2024-03-28T16:49:19.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94875153596944] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94875153604864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d40eb0 con 0x5649d7d35010 2024-03-28T16:49:19.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.447+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d42da0 con 0x5649d7d35010 2024-03-28T16:49:19.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.451+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94875153604864] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.451+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.451+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94875153602672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d435a0 con 0x5649d7d35010 2024-03-28T16:49:19.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.451+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d43f10 con 0x5649d7d35010 2024-03-28T16:49:19.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.451+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94875153602672] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94875153611376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d44330 con 0x5649d7d35010 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=0 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=1 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=2 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=3 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=4 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=5 2024-03-28T16:49:19.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=6 2024-03-28T16:49:19.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94875153611376] v29'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 add_watcher() i=7 2024-03-28T16:49:19.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:19.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d47620 con 0x5649d7d30aa0 2024-03-28T16:49:19.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94875153612800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d448c0 con 0x5649d7d30aa0 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94875153612800] v29'110 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efcf4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.455+0000 7efe40bc3980 5 note: GC not initialized 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efc6f7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 Filter name: none 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efc6d7fa640 20 reqs_thread_entry: start 2024-03-28T16:49:19.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:49:19.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d69610 con 0x5649d7d35010 2024-03-28T16:49:19.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6b170 con 0x7efe340da040 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6c9a0 con 0x5649d7d39bf0 2024-03-28T16:49:19.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-03-28T16:49:19.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.459+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d710a0 con 0x7efe340da040 2024-03-28T16:49:19.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 foo.buckets [call] v29'21 uv19 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d76ec0 con 0x5649d7d39bf0 2024-03-28T16:49:19.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:19.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 10 moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 7.9 7:93e5b521:::notify.7:head [notify cookie 94875153780240 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d71490 con 0x5649d7d35010 2024-03-28T16:49:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== watch-notify(notify (1) cookie 94875153611376 notify 124554051588 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5649d7d1f3c0 con 0x5649d7d35010 2024-03-28T16:49:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efd21ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051588 cookie 94875153611376 notifier 4494 bl.length()=725 2024-03-28T16:49:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efd21ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:19.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efd21ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:19.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efd21ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:19.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(67 notify.7 [notify cookie 94875153780240 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efd21ffb640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7efd04003ea0 con 0x5649d7d35010 2024-03-28T16:49:19.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify_complete (2) cookie 94875153780240 notify 124554051588 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5649d7d12400 con 0x5649d7d35010 2024-03-28T16:49:19.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(68 notify.7 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_sources= new_sources= 2024-03-28T16:49:19.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential sources added=[] removed=[] 2024-03-28T16:49:19.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_dests= new_dests= 2024-03-28T16:49:19.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential dests added=[] removed=[] 2024-03-28T16:49:19.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.463+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d35010 2024-03-28T16:49:19.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.467+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(69 fud.buckets [call] v29'3 uv3 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7efe38035030 con 0x5649d7d35010 2024-03-28T16:49:19.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.467+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x7efe340da040 2024-03-28T16:49:19.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v29'22 uv22 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe40bc3980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-03-28T16:49:19.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe40bc3980 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:19.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe40bc3980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:49:19.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94875153819536 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d35010 2024-03-28T16:49:19.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== watch-notify(notify (1) cookie 94875153602672 notify 124554051589 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x5649d7d12400 con 0x5649d7d35010 2024-03-28T16:49:19.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(71 notify.6 [notify cookie 94875153819536 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efe3808d040 con 0x5649d7d35010 2024-03-28T16:49:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efd21ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051589 cookie 94875153602672 notifier 4494 bl.length()=314 2024-03-28T16:49:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efd21ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-03-28T16:49:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efd21ffb640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efd21ffb640 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7efd04004200 con 0x5649d7d35010 2024-03-28T16:49:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== watch-notify(notify_complete (2) cookie 94875153819536 notify 124554051589 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5649d7cf4c60 con 0x5649d7d35010 2024-03-28T16:49:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3808d040 con 0x5649d7d35010 2024-03-28T16:49:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efcf4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94875153612800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d30aa0 2024-03-28T16:49:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.471+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94875153612800] v29'111 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94875153586656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x7efe340da040 2024-03-28T16:49:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94875153586656] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe40bc3980 20 remove_watcher() i=0 2024-03-28T16:49:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe40bc3980 2 removed watcher, disabling cache 2024-03-28T16:49:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94875153591104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d435a0 con 0x5649d7d30aa0 2024-03-28T16:49:19.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94875153591104] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe40bc3980 20 remove_watcher() i=1 2024-03-28T16:49:19.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.475+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94875153592160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d39bf0 2024-03-28T16:49:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe3dedd640 1 -- 172.21.15.170:0/2944063978 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94875153592160] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe340abbe0 con 0x5649d7d39bf0 2024-03-28T16:49:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe40bc3980 20 remove_watcher() i=2 2024-03-28T16:49:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94875153598432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x7efe340da040 2024-03-28T16:49:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94875153598432] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x7efe340da040 2024-03-28T16:49:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe40bc3980 20 remove_watcher() i=3 2024-03-28T16:49:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94875153596944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d30aa0 2024-03-28T16:49:19.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94875153596944] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3000a4d0 con 0x5649d7d30aa0 2024-03-28T16:49:19.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe40bc3980 20 remove_watcher() i=4 2024-03-28T16:49:19.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.479+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94875153604864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d35010 2024-03-28T16:49:19.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.483+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94875153604864] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3808d040 con 0x5649d7d35010 2024-03-28T16:49:19.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.483+0000 7efe40bc3980 20 remove_watcher() i=5 2024-03-28T16:49:19.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.483+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94875153602672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d35010 2024-03-28T16:49:19.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.483+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94875153602672] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3808d040 con 0x5649d7d35010 2024-03-28T16:49:19.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.483+0000 7efe40bc3980 20 remove_watcher() i=6 2024-03-28T16:49:19.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.483+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94875153611376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5649d7d6ce10 con 0x5649d7d35010 2024-03-28T16:49:19.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe3e6de640 1 -- 172.21.15.170:0/2944063978 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94875153611376] v29'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe3808d040 con 0x5649d7d35010 2024-03-28T16:49:19.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 20 remove_watcher() i=7 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7d35010 msgr2=0x5649d7d37400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7d35010 0x5649d7d37400 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7d30aa0 msgr2=0x5649d7d32f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7d30aa0 0x5649d7d32f60 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe340da040 msgr2=0x7efe340cc420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe340da040 0x7efe340cc420 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5649d7d39bf0 msgr2=0x5649d7d3c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5649d7d39bf0 0x5649d7d3c0b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd080342b0 msgr2=0x7efd08036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd080342b0 0x7efd08036770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe3401b2c0 tx=0x7efe340a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe3eedf640 1 -- 172.21.15.170:0/2944063978 reap_dead start 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 msgr2=0x5649d7d258f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d258f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efe38045d50 tx=0x7efe3804ae80 comp rx=0 tx=0).stop 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 shutdown_connections 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/2944063978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d1e9f0 0x5649d7d258f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 >> 172.21.15.170:0/2944063978 conn(0x5649d7d1ad80 msgr2=0x5649d7d183e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 shutdown_connections 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/2944063978 wait complete. 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd180342b0 msgr2=0x7efd18036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd180342b0 0x7efd18036770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x5649d7d00090 tx=0x7efe34005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d11aa0 msgr2=0x5649d7d11ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d11aa0 0x5649d7d11ec0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efe38047a80 tx=0x7efe38016a00 comp rx=0 tx=0).stop 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 shutdown_connections 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd180342b0 0x7efd18036770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/4104593792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7d11aa0 0x5649d7d11ec0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 >> 172.21.15.170:0/4104593792 conn(0x5649d7d068b0 msgr2=0x5649d7d073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 shutdown_connections 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/4104593792 wait complete. 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7cfd1a0 msgr2=0x5649d7cff590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5649d7cfd1a0 0x5649d7cff590 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7c076b0 msgr2=0x5649d7cfaf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5649d7c076b0 0x5649d7cfaf70 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5649d7d01120 msgr2=0x5649d7d03510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5649d7d01120 0x5649d7d03510 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd240342b0 msgr2=0x7efd24036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd240342b0 0x7efd24036770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe340ad000 tx=0x7efe340a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 msgr2=0x5649d7cf42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 --2- 172.21.15.170:0/365063403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5649d7b75010 0x5649d7cf42f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efe38001ee0 tx=0x7efe38000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe3eedf640 1 -- 172.21.15.170:0/365063403 reap_dead start 2024-03-28T16:49:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 shutdown_connections 2024-03-28T16:49:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 >> 172.21.15.170:0/365063403 conn(0x5649d7bfb670 msgr2=0x5649d7ce7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 shutdown_connections 2024-03-28T16:49:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.487+0000 7efe40bc3980 1 -- 172.21.15.170:0/365063403 wait complete. 2024-03-28T16:49:19.498 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-03-28T16:49:19.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-03-28T16:49:19.498 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-03-28T16:49:19.574 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:19.574 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3de0ce980 1 Processor -- start 2024-03-28T16:49:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3de0ce980 1 -- start start 2024-03-28T16:49:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3de0ce980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55702afbe630 0x55702afc0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3de0ce980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55702af04520 con 0x55702afba130 2024-03-28T16:49:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3de0ce980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55702aecb0f0 con 0x55702afbe630 2024-03-28T16:49:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3db3e8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55702afbe630 0x55702afc0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3dbbe9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55702afba130 0x55702afba510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38212/0 (socket says 172.21.15.170:38212) 2024-03-28T16:49:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3db3e8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55702afbe630 0x55702afc0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56604/0 (socket says 172.21.15.170:56604) 2024-03-28T16:49:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3691285775 learned_addr learned my addr 172.21.15.170:0/3691285775 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3db3e8640 1 -- 172.21.15.170:0/3691285775 >> v1:172.21.15.119:6789/0 conn(0x55702afba130 legacy=0x55702afba510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.575+0000 7fb3db3e8640 1 -- 172.21.15.170:0/3691285775 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702afc1f70 con 0x55702afbe630 2024-03-28T16:49:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/3691285775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afbe630 0x55702afc0a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000a480 tx=0x7fb3d001b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e567d30971049ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3dabe7640 1 -- 172.21.15.170:0/3691285775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3d00ab540 con 0x55702afbe630 2024-03-28T16:49:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3dabe7640 1 -- 172.21.15.170:0/3691285775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d00ab6a0 con 0x55702afbe630 2024-03-28T16:49:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3dabe7640 1 -- 172.21.15.170:0/3691285775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3d00ab960 con 0x55702afbe630 2024-03-28T16:49:19.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3691285775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afbe630 msgr2=0x55702afc0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3691285775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afbe630 0x55702afc0a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000a480 tx=0x7fb3d001b280 comp rx=0 tx=0).stop 2024-03-28T16:49:19.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3691285775 shutdown_connections 2024-03-28T16:49:19.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3691285775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afbe630 0x55702afc0a20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3691285775 >> 172.21.15.170:0/3691285775 conn(0x55702af2d850 msgr2=0x55702af2dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3691285775 shutdown_connections 2024-03-28T16:49:19.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.579+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3691285775 wait complete. 2024-03-28T16:49:19.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.591+0000 7fb3de0ce980 1 Processor -- start 2024-03-28T16:49:19.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.591+0000 7fb3de0ce980 1 -- start start 2024-03-28T16:49:19.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.593 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702aec8620 con 0x55702afba130 2024-03-28T16:49:19.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0a3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56608/0 (socket says 172.21.15.170:56608) 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/666571710 learned_addr learned my addr 172.21.15.170:0/666571710 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/666571710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0005d30 con 0x55702afba130 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/666571710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0a3fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc001c00 tx=0x7fb3cc00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5716427d10768f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb2c9e0b640 1 -- 172.21.15.170:0/666571710 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc00e690 con 0x55702afba130 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb2c9e0b640 1 -- 172.21.15.170:0/666571710 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc00e810 con 0x55702afba130 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb2c9e0b640 1 -- 172.21.15.170:0/666571710 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc00eb30 con 0x55702afba130 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- 172.21.15.170:0/666571710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 msgr2=0x55702b0a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/666571710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0a3fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc001c00 tx=0x7fb3cc00b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- 172.21.15.170:0/666571710 shutdown_connections 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/666571710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0a3fe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- 172.21.15.170:0/666571710 >> 172.21.15.170:0/666571710 conn(0x55702af2d850 msgr2=0x55702b09fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- 172.21.15.170:0/666571710 shutdown_connections 2024-03-28T16:49:19.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- 172.21.15.170:0/666571710 wait complete. 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 Processor -- start 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- start start 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3de0ce980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702aec3e00 con 0x55702afba130 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0aae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56616/0 (socket says 172.21.15.170:56616) 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3886879750 learned_addr learned my addr 172.21.15.170:0/3886879750 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702af04520 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.595+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0aae30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00b710 tx=0x7fb3cc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb2c3fff640 1 -- 172.21.15.170:0/3886879750 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc01f070 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55702b0ab370 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb2c3fff640 1 -- 172.21.15.170:0/3886879750 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc01a910 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb2c3fff640 1 -- 172.21.15.170:0/3886879750 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc01abf0 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55702b0ab7a0 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb2c3fff640 1 -- 172.21.15.170:0/3886879750 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb3cc00a040 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb2c3fff640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2c40342b0 0x7fb2c4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb2c3fff640 1 -- 172.21.15.170:0/3886879750 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb3cc049e50 con 0x55702afba130 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2c40342b0 0x7fb2c4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702afbe630 0x55702b0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0b1ff0 con 0x55702afbe630 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3dc3ea640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702afbe630 0x55702b0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2c40342b0 0x7fb2c4036770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00ad000 tx=0x7fb3d00a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3dc3ea640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702afbe630 0x55702b0b1ab0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/3886879750 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702afbe630 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0b2350 con 0x55702afbe630 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/3886879750 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702afbe630 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0b4070 0x55702b0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0b69a0 con 0x55702b0b4070 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0b4070 0x55702b0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.599+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0b4070 0x55702b0b6460 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3886879750 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0b4070 2024-03-28T16:49:19.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0b2eb0 con 0x55702afbe630 2024-03-28T16:49:19.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/3886879750 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702afbe630 2024-03-28T16:49:19.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55702b0b7da0 0x55702b0ba190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ba830 con 0x55702b0b7da0 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55702b0b7da0 0x55702b0ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55702b0b7da0 0x55702b0ba190 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3db3e8640 1 -- 172.21.15.170:0/3886879750 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0b7da0 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 Processor -- start 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 -- start start 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0c8760 0x55702b0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702ae5ce30 con 0x55702b0c8760 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0c8760 0x55702b0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0c8760 0x55702b0c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56628/0 (socket says 172.21.15.170:56628) 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3838106535 learned_addr learned my addr 172.21.15.170:0/3838106535 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3838106535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702b0c90c0 con 0x55702b0c8760 2024-03-28T16:49:19.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0c8760 0x55702b0c8b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc014040 tx=0x7fb3cc018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb2c17fa640 1 -- 172.21.15.170:0/3838106535 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc042120 con 0x55702b0c8760 2024-03-28T16:49:19.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55702b0c9310 con 0x55702b0c8760 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.603+0000 7fb2c17fa640 1 -- 172.21.15.170:0/3838106535 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc0225a0 con 0x55702b0c8760 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55702b0c9e60 con 0x55702b0c8760 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb2c17fa640 1 -- 172.21.15.170:0/3838106535 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc022840 con 0x55702b0c8760 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb2c17fa640 1 -- 172.21.15.170:0/3838106535 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb3cc0229a0 con 0x55702b0c8760 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb2c17fa640 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2b4034300 0x7fb2b40367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb2c17fa640 1 -- 172.21.15.170:0/3838106535 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb3cc01c0a0 con 0x55702b0c8760 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2b4034300 0x7fb2b40367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 Processor -- start 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- start start 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2b4034300 0x7fb2b40367c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55702b0b3210 tx=0x7fb3d0005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0d58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702ae50790 con 0x55702b0d54f0 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0d58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0d58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56634/0 (socket says 172.21.15.170:56634) 2024-03-28T16:49:19.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3572248304 learned_addr learned my addr 172.21.15.170:0/3572248304 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/3572248304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702b0d5ea0 con 0x55702b0d54f0 2024-03-28T16:49:19.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/3572248304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0d58d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc036010 tx=0x7fb3cc03a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b9d90f63ccb660 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb2baffd640 1 -- 172.21.15.170:0/3572248304 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc041350 con 0x55702b0d54f0 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb2baffd640 1 -- 172.21.15.170:0/3572248304 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc0414b0 con 0x55702b0d54f0 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb2baffd640 1 -- 172.21.15.170:0/3572248304 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc049520 con 0x55702b0d54f0 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3572248304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 msgr2=0x55702b0d58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3572248304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0d58d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc036010 tx=0x7fb3cc03a740 comp rx=0 tx=0).stop 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3572248304 shutdown_connections 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3572248304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0d58d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3572248304 >> 172.21.15.170:0/3572248304 conn(0x55702b0d1880 msgr2=0x55702b0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3572248304 shutdown_connections 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.607+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3572248304 wait complete. 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 Processor -- start 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 -- start start 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55702b0dc700 con 0x55702b0d54f0 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dbbe9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0dc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56638/0 (socket says 172.21.15.170:56638) 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 learned_addr learned my addr 172.21.15.170:0/1416777490 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55702b0c90c0 con 0x55702b0d54f0 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0dc1c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0471b0 tx=0x7fb3cc040f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb2b97fa640 1 -- 172.21.15.170:0/1416777490 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc026580 con 0x55702b0d54f0 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55702b0c90c0 con 0x55702b0d54f0 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb2b97fa640 1 -- 172.21.15.170:0/1416777490 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc0266e0 con 0x55702b0d54f0 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55702b0dcb40 con 0x55702b0d54f0 2024-03-28T16:49:19.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb2b97fa640 1 -- 172.21.15.170:0/1416777490 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc03a8b0 con 0x55702b0d54f0 2024-03-28T16:49:19.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb2b97fa640 1 -- 172.21.15.170:0/1416777490 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb3cc03aa10 con 0x55702b0d54f0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb2b97fa640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2a40342b0 0x7fb2a4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb2b97fa640 1 -- 172.21.15.170:0/1416777490 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb3cc004190 con 0x55702b0d54f0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2a40342b0 0x7fb2a4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702b0e7480 0x55702b0e9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ea090 con 0x55702b0e7480 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dc3ea640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702b0e7480 0x55702b0e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2a40342b0 0x7fb2a4036770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00abfd0 tx=0x7fb3d00bff90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dc3ea640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702b0e7480 0x55702b0e9940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.611+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 realm 2024-03-28T16:49:19.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ea460 con 0x55702b0e7480 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eb040 con 0x55702b0e7480 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0eb9f0 0x55702b0edde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ee3f0 con 0x55702b0eb9f0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0eb9f0 0x55702b0edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dbbe9640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0eb9f0 0x55702b0edde0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 20 searching for the correct realm 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55702b0f05d0 0x55702b0f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f3160 con 0x55702b0f05d0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55702b0f05d0 0x55702b0f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55702b0f05d0 0x55702b0f2a90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3db3e8640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb3d00cc050 0x7fb3d00cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f3160 con 0x7fb3d00cc050 2024-03-28T16:49:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dc3ea640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb3d00cc050 0x7fb3d00cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.615+0000 7fb3dc3ea640 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb3d00cc050 0x7fb3d00cb3b0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x7fb3d00cc050 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.619+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x7fb3d00cc050 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.623+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x7fb3d00cc050 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x7fb3d00cc050 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0f05d0 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0eb9f0 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x55702b0e7480 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0eace0 con 0x7fb3d00cc050 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.627+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ee9d0 con 0x55702b0e7480 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ee9d0 con 0x55702b0e7480 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ee9a0 con 0x7fb3d00cc050 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 zone default found 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 4 Realm: () 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 zonegroup default 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ef220 con 0x55702b0eb9f0 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0ef620 con 0x55702b0f05d0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f5050 con 0x55702b0f05d0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 started sync module instance, tier type = 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f4860 con 0x7fb3d00cc050 2024-03-28T16:49:19.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93940247118784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f4c10 con 0x7fb3d00cc050 2024-03-28T16:49:19.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.631+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f5db0 con 0x55702b0e7480 2024-03-28T16:49:19.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93940247118784] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93940247123056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f5d30 con 0x55702b0e7480 2024-03-28T16:49:19.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f6800 con 0x55702b0f05d0 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93940247126752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f7050 con 0x55702b0f05d0 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f7a20 con 0x7fb3d00cc050 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93940247123056] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93940247131424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f7dd0 con 0x7fb3d00cc050 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.635+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f8740 con 0x55702b0e7480 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93940247131424] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93940247126752] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93940247129008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f82a0 con 0x55702b0e7480 2024-03-28T16:49:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f8f90 con 0x55702b0eb9f0 2024-03-28T16:49:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93940247129008] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93940247137008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f82a0 con 0x55702b0eb9f0 2024-03-28T16:49:19.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.639+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f9790 con 0x55702b0eb9f0 2024-03-28T16:49:19.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.643+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93940247137008] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.643+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.643+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93940247124112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0f9f90 con 0x55702b0eb9f0 2024-03-28T16:49:19.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.643+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0fa900 con 0x55702b0eb9f0 2024-03-28T16:49:19.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93940247124112] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93940247143520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0fad20 con 0x55702b0eb9f0 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=0 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=1 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=2 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=3 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=4 2024-03-28T16:49:19.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=5 2024-03-28T16:49:19.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=6 2024-03-28T16:49:19.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93940247143520] v29'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 20 add_watcher() i=7 2024-03-28T16:49:19.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:19.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0fdfa0 con 0x55702b0e7480 2024-03-28T16:49:19.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.647+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93940247144944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b0fb2b0 con 0x55702b0e7480 2024-03-28T16:49:19.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.651+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93940247144944] v29'112 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.651+0000 7fb291fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.651+0000 7fb3de0ce980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:19.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.651+0000 7fb3de0ce980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:19.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.651+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b106610 con 0x55702b0e7480 2024-03-28T16:49:19.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.651+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.651+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b106690 con 0x55702b0eb9f0 2024-03-28T16:49:19.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b106af0 con 0x55702b0f05d0 2024-03-28T16:49:19.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1074c0 con 0x55702b0e7480 2024-03-28T16:49:19.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'113 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b104660 con 0x7fb3d00cc050 2024-03-28T16:49:19.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'32 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.655+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b104660 con 0x55702b0f05d0 2024-03-28T16:49:19.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.659+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.659+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b108200 con 0x55702b0e7480 2024-03-28T16:49:19.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.659+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.659+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b108200 con 0x55702b0eb9f0 2024-03-28T16:49:19.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.659+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'21 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.659+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b108730 con 0x7fb3d00cc050 2024-03-28T16:49:19.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.663+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.663+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b108730 con 0x55702b0f05d0 2024-03-28T16:49:19.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.663+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.663+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b108c60 con 0x7fb3d00cc050 2024-03-28T16:49:19.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.663+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'33 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.663+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b108c60 con 0x55702b0eb9f0 2024-03-28T16:49:19.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.667+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'16 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.667+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b109190 con 0x7fb3d00cc050 2024-03-28T16:49:19.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.667+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.667+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b109190 con 0x55702b0eb9f0 2024-03-28T16:49:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.671+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'28 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.671+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1092d0 con 0x55702b0e7480 2024-03-28T16:49:19.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.671+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'114 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.671+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b109100 con 0x55702b0f05d0 2024-03-28T16:49:19.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.671+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'26 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.671+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103650 con 0x7fb3d00cc050 2024-03-28T16:49:19.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.675+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'34 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.675+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.675+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.675+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0e7480 2024-03-28T16:49:19.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.675+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'22 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.675+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0e7480 2024-03-28T16:49:19.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.675+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'115 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.679+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.679+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'27 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.679+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x7fb3d00cc050 2024-03-28T16:49:19.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.679+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'24 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.679+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'20 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'22 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.683+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0eb9f0 2024-03-28T16:49:19.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.687+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.687+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.687+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.687+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0eb9f0 2024-03-28T16:49:19.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.691+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'22 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.691+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0f05d0 2024-03-28T16:49:19.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.691+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'28 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.691+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x55702b0eb9f0 2024-03-28T16:49:19.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.695+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'29 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.695+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b103c40 con 0x7fb3d00cc050 2024-03-28T16:49:19.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.695+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.695+0000 7fb290fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:19.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:19.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 20 Filter name: none 2024-03-28T16:49:19.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb20a7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:19.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:49:19.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b105a10 con 0x55702b0eb9f0 2024-03-28T16:49:19.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(94 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:49:19.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-03-28T16:49:19.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:19.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:19.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0eb9f0 2024-03-28T16:49:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(95 fud.buckets [call out=183b] v0'0 uv3 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-03-28T16:49:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb291fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.699+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93940247144944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0e7480 2024-03-28T16:49:19.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 93940247144944] v29'116 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93940247118784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 93940247118784] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3de0ce980 20 remove_watcher() i=0 2024-03-28T16:49:19.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3de0ce980 2 removed watcher, disabling cache 2024-03-28T16:49:19.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93940247123056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0e7480 2024-03-28T16:49:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 93940247123056] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3de0ce980 20 remove_watcher() i=1 2024-03-28T16:49:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.703+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93940247126752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0f05d0 2024-03-28T16:49:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.707+0000 7fb3db3e8640 1 -- 172.21.15.170:0/1416777490 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 93940247126752] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d00abc30 con 0x55702b0f05d0 2024-03-28T16:49:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.707+0000 7fb3de0ce980 20 remove_watcher() i=2 2024-03-28T16:49:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.707+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93940247131424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.707+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 93940247131424] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x7fb3d00cc050 2024-03-28T16:49:19.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.707+0000 7fb3de0ce980 20 remove_watcher() i=3 2024-03-28T16:49:19.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.707+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93940247129008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0e7480 2024-03-28T16:49:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.711+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 93940247129008] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3d400a4d0 con 0x55702b0e7480 2024-03-28T16:49:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.711+0000 7fb3de0ce980 20 remove_watcher() i=4 2024-03-28T16:49:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.711+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93940247137008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0eb9f0 2024-03-28T16:49:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.711+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 93940247137008] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.711+0000 7fb3de0ce980 20 remove_watcher() i=5 2024-03-28T16:49:19.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.711+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93940247124112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0eb9f0 2024-03-28T16:49:19.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 93940247124112] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 20 remove_watcher() i=6 2024-03-28T16:49:19.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93940247143520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55702b1059d0 con 0x55702b0eb9f0 2024-03-28T16:49:19.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3dbbe9640 1 -- 172.21.15.170:0/1416777490 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 93940247143520] v29'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb3cc017900 con 0x55702b0eb9f0 2024-03-28T16:49:19.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 20 remove_watcher() i=7 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0eb9f0 msgr2=0x55702b0edde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0eb9f0 0x55702b0edde0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702b0e7480 msgr2=0x55702b0e9940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702b0e7480 0x55702b0e9940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb3d00cc050 msgr2=0x7fb3d00cb3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb3d00cc050 0x7fb3d00cb3b0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55702b0f05d0 msgr2=0x55702b0f2a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55702b0f05d0 0x55702b0f2a90 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2a40342b0 msgr2=0x7fb2a4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2a40342b0 0x7fb2a4036770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00abfd0 tx=0x7fb3d00bff90 comp rx=0 tx=0).stop 2024-03-28T16:49:19.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 msgr2=0x55702b0dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/1416777490 reap_dead start 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0dc1c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0471b0 tx=0x7fb3cc040f10 comp rx=0 tx=0).stop 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 shutdown_connections 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/1416777490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0d54f0 0x55702b0dc1c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 >> 172.21.15.170:0/1416777490 conn(0x55702b0d1880 msgr2=0x55702b0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 shutdown_connections 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/1416777490 wait complete. 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2b4034300 msgr2=0x7fb2b40367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2b4034300 0x7fb2b40367c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55702b0b3210 tx=0x7fb3d0005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0c8760 msgr2=0x55702b0c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0c8760 0x55702b0c8b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc014040 tx=0x7fb3cc018f90 comp rx=0 tx=0).stop 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 shutdown_connections 2024-03-28T16:49:19.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2b4034300 0x7fb2b40367c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3838106535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702b0c8760 0x55702b0c8b80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 >> 172.21.15.170:0/3838106535 conn(0x55702b0bd570 msgr2=0x55702b0be080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 shutdown_connections 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3838106535 wait complete. 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0b4070 msgr2=0x55702b0b6460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55702b0b4070 0x55702b0b6460 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702afbe630 msgr2=0x55702b0b1ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55702afbe630 0x55702b0b1ab0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55702b0b7da0 msgr2=0x55702b0ba190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55702b0b7da0 0x55702b0ba190 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2c40342b0 msgr2=0x7fb2c4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb2c40342b0 0x7fb2c4036770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00ad000 tx=0x7fb3d00a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 msgr2=0x55702b0aae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3de0ce980 1 --2- 172.21.15.170:0/3886879750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55702afba130 0x55702b0aae30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00b710 tx=0x7fb3cc000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.715+0000 7fb3dc3ea640 1 -- 172.21.15.170:0/3886879750 reap_dead start 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.719+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 shutdown_connections 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.719+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 >> 172.21.15.170:0/3886879750 conn(0x55702af2d850 msgr2=0x55702b09e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.719+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 shutdown_connections 2024-03-28T16:49:19.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.719+0000 7fb3de0ce980 1 -- 172.21.15.170:0/3886879750 wait complete. 2024-03-28T16:49:19.726 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-03-28T16:49:19.726 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-03-28T16:49:19.727 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-03-28T16:49:19.727 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-03-28T16:49:19.804 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:19.804 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.803+0000 7f70ef59c980 1 Processor -- start 2024-03-28T16:49:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.803+0000 7f70ef59c980 1 -- start start 2024-03-28T16:49:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.803+0000 7f70ef59c980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e291f598c0 0x55e291fe7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.803+0000 7f70ef59c980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e291f314c0 con 0x55e291fe8180 2024-03-28T16:49:19.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.803+0000 7f70ef59c980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e291ef80f0 con 0x55e291f598c0 2024-03-28T16:49:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70ed0b7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e291f598c0 0x55e291fe7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70ed0b7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e291f598c0 0x55e291fe7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56650/0 (socket says 172.21.15.170:56650) 2024-03-28T16:49:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70ec8b6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55e291fe8180 0x55e291fea570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38228/0 (socket says 172.21.15.170:38228) 2024-03-28T16:49:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70ed0b7640 1 -- 172.21.15.170:0/1245478929 learned_addr learned my addr 172.21.15.170:0/1245478929 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70ed0b7640 1 -- 172.21.15.170:0/1245478929 >> v1:172.21.15.119:6789/0 conn(0x55e291fe8180 legacy=0x55e291fea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70ed0b7640 1 -- 172.21.15.170:0/1245478929 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e291feccd0 con 0x55e291f598c0 2024-03-28T16:49:19.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/1245478929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e291fe7c40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a5c0 tx=0x7f70e001b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650fd39dcfde73ef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70dffff640 1 -- 172.21.15.170:0/1245478929 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00aa470 con 0x55e291f598c0 2024-03-28T16:49:19.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70dffff640 1 -- 172.21.15.170:0/1245478929 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00aa5d0 con 0x55e291f598c0 2024-03-28T16:49:19.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.807+0000 7f70dffff640 1 -- 172.21.15.170:0/1245478929 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00aa890 con 0x55e291f598c0 2024-03-28T16:49:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.811+0000 7f70ef59c980 1 -- 172.21.15.170:0/1245478929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 msgr2=0x55e291fe7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.811+0000 7f70ef59c980 1 --2- 172.21.15.170:0/1245478929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e291fe7c40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f70e000a5c0 tx=0x7f70e001b340 comp rx=0 tx=0).stop 2024-03-28T16:49:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.811+0000 7f70ef59c980 1 -- 172.21.15.170:0/1245478929 shutdown_connections 2024-03-28T16:49:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.811+0000 7f70ef59c980 1 --2- 172.21.15.170:0/1245478929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e291fe7c40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.811+0000 7f70ef59c980 1 -- 172.21.15.170:0/1245478929 >> 172.21.15.170:0/1245478929 conn(0x55e291fdf720 msgr2=0x55e291fe1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.811+0000 7f70ef59c980 1 -- 172.21.15.170:0/1245478929 shutdown_connections 2024-03-28T16:49:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.811+0000 7f70ef59c980 1 -- 172.21.15.170:0/1245478929 wait complete. 2024-03-28T16:49:19.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ef59c980 1 Processor -- start 2024-03-28T16:49:19.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ef59c980 1 -- start start 2024-03-28T16:49:19.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ef59c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ef59c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e291ef5620 con 0x55e291f598c0 2024-03-28T16:49:19.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56662/0 (socket says 172.21.15.170:56662) 2024-03-28T16:49:19.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ed0b7640 1 -- 172.21.15.170:0/1312758315 learned_addr learned my addr 172.21.15.170:0/1312758315 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ed0b7640 1 -- 172.21.15.170:0/1312758315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e0005df0 con 0x55e291f598c0 2024-03-28T16:49:19.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/1312758315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920ceec0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ac040 tx=0x7f70e0003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7474c07d6b0d97 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f6fdb606640 1 -- 172.21.15.170:0/1312758315 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00aa9b0 con 0x55e291f598c0 2024-03-28T16:49:19.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f6fdb606640 1 -- 172.21.15.170:0/1312758315 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00aab30 con 0x55e291f598c0 2024-03-28T16:49:19.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.823+0000 7f6fdb606640 1 -- 172.21.15.170:0/1312758315 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00aae10 con 0x55e291f598c0 2024-03-28T16:49:19.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/1312758315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 msgr2=0x55e2920ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 --2- 172.21.15.170:0/1312758315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920ceec0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ac040 tx=0x7f70e0003030 comp rx=0 tx=0).stop 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/1312758315 shutdown_connections 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 --2- 172.21.15.170:0/1312758315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920ceec0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/1312758315 >> 172.21.15.170:0/1312758315 conn(0x55e291fdf720 msgr2=0x55e2920caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/1312758315 shutdown_connections 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/1312758315 wait complete. 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 Processor -- start 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- start start 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920d5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e291ef0e00 con 0x55e291f598c0 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920d5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56678/0 (socket says 172.21.15.170:56678) 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2225119456 learned_addr learned my addr 172.21.15.170:0/2225119456 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e291feccd0 con 0x55e291f598c0 2024-03-28T16:49:19.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920d5c10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f70e0005940 tx=0x7f70e00180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f6fd9e03640 1 -- 172.21.15.170:0/2225119456 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00c6750 con 0x55e291f598c0 2024-03-28T16:49:19.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2920d6150 con 0x55e291f598c0 2024-03-28T16:49:19.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f6fd9e03640 1 -- 172.21.15.170:0/2225119456 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00c68b0 con 0x55e291f598c0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f6fd9e03640 1 -- 172.21.15.170:0/2225119456 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00c6b70 con 0x55e291f598c0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2920d6580 con 0x55e291f598c0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f6fd9e03640 1 -- 172.21.15.170:0/2225119456 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f70e00c6cd0 con 0x55e291f598c0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f6fd9e03640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fd40342b0 0x7f6fd4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f6fd9e03640 1 -- 172.21.15.170:0/2225119456 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f70e00e4170 con 0x55e291f598c0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fd40342b0 0x7f6fd4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e291fe8180 0x55e2920dc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.827+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2920dcd80 con 0x55e291fe8180 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed8b8640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e291fe8180 0x55e2920dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fd40342b0 0x7f6fd4036770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000f70 tx=0x7f70e4001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed8b8640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e291fe8180 0x55e2920dc840 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2225119456 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e291fe8180 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2920dd0e0 con 0x55e291fe8180 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2225119456 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e291fe8180 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e2920dee00 0x55e2920e11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2920e1730 con 0x55e2920dee00 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e2920dee00 0x55e2920e11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e2920dee00 0x55e2920e11f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2225119456 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f70e00b0020 con 0x55e2920dee00 2024-03-28T16:49:19.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2920ddc10 con 0x55e291fe8180 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2225119456 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e291fe8180 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e2920e2aa0 0x55e2920e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2920e54e0 con 0x55e2920e2aa0 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e2920e2aa0 0x55e2920e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e2920e2aa0 0x55e2920e4e90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2225119456 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f70e400f6a0 con 0x55e2920e2aa0 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.831+0000 7f70ef59c980 1 Processor -- start 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 -- start start 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2920f3410 0x55e2920f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e291e89e30 con 0x55e2920f3410 2024-03-28T16:49:19.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2920f3410 0x55e2920f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2920f3410 0x55e2920f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56680/0 (socket says 172.21.15.170:56680) 2024-03-28T16:49:19.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ed0b7640 1 -- 172.21.15.170:0/395308381 learned_addr learned my addr 172.21.15.170:0/395308381 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ed0b7640 1 -- 172.21.15.170:0/395308381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2920f3d70 con 0x55e2920f3410 2024-03-28T16:49:19.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2920f3410 0x55e2920f3830 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f70e00e27e0 tx=0x7f70e00e40b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f6fd2ffd640 1 -- 172.21.15.170:0/395308381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00b33f0 con 0x55e2920f3410 2024-03-28T16:49:19.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2920f3f60 con 0x55e2920f3410 2024-03-28T16:49:19.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f6fd2ffd640 1 -- 172.21.15.170:0/395308381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00b3550 con 0x55e2920f3410 2024-03-28T16:49:19.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2920f49f0 con 0x55e2920f3410 2024-03-28T16:49:19.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f6fd2ffd640 1 -- 172.21.15.170:0/395308381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00b37f0 con 0x55e2920f3410 2024-03-28T16:49:19.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f6fd2ffd640 1 -- 172.21.15.170:0/395308381 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f70e00e2050 con 0x55e2920f3410 2024-03-28T16:49:19.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f6fd2ffd640 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fc40342b0 0x7f6fc4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f6fd2ffd640 1 -- 172.21.15.170:0/395308381 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f70e00a4190 con 0x55e2920f3410 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fc40342b0 0x7f6fc4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 Processor -- start 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 -- start start 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fc40342b0 0x7f6fc4036770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55e2920ddf70 tx=0x7f70e400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e291fe55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.835+0000 7f70ef59c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e291e7d790 con 0x55e291fe5180 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e291fe55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e291fe55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56682/0 (socket says 172.21.15.170:56682) 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 -- 172.21.15.170:0/76984570 learned_addr learned my addr 172.21.15.170:0/76984570 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 -- 172.21.15.170:0/76984570 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e291fe3a20 con 0x55e291fe5180 2024-03-28T16:49:19.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/76984570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e291fe55a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f70e00b3970 tx=0x7f70e00e48d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fd827623100f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f6fd0ff9640 1 -- 172.21.15.170:0/76984570 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00fc070 con 0x55e291fe5180 2024-03-28T16:49:19.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f6fd0ff9640 1 -- 172.21.15.170:0/76984570 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00be050 con 0x55e291fe5180 2024-03-28T16:49:19.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f6fd0ff9640 1 -- 172.21.15.170:0/76984570 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00e4cd0 con 0x55e291fe5180 2024-03-28T16:49:19.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- 172.21.15.170:0/76984570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 msgr2=0x55e291fe55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 --2- 172.21.15.170:0/76984570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e291fe55a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f70e00b3970 tx=0x7f70e00e48d0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- 172.21.15.170:0/76984570 shutdown_connections 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 --2- 172.21.15.170:0/76984570 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e291fe55a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- 172.21.15.170:0/76984570 >> 172.21.15.170:0/76984570 conn(0x55e2920fc390 msgr2=0x55e2920fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- 172.21.15.170:0/76984570 shutdown_connections 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- 172.21.15.170:0/76984570 wait complete. 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 Processor -- start 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- start start 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e292109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2921095b0 con 0x55e291fe5180 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e292109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e292109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56684/0 (socket says 172.21.15.170:56684) 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 learned_addr learned my addr 172.21.15.170:0/2711934410 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00e8030 con 0x55e291fe5180 2024-03-28T16:49:19.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e292109070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f70e00cf6c0 tx=0x7f70e00cded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f6fc2ffd640 1 -- 172.21.15.170:0/2711934410 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e0111070 con 0x55e291fe5180 2024-03-28T16:49:19.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2921097b0 con 0x55e291fe5180 2024-03-28T16:49:19.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f6fc2ffd640 1 -- 172.21.15.170:0/2711934410 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e00be970 con 0x55e291fe5180 2024-03-28T16:49:19.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f6fc2ffd640 1 -- 172.21.15.170:0/2711934410 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70e00bc5b0 con 0x55e291fe5180 2024-03-28T16:49:19.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.839+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2921097b0 con 0x55e291fe5180 2024-03-28T16:49:19.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f6fc2ffd640 1 -- 172.21.15.170:0/2711934410 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f70e00f2200 con 0x55e291fe5180 2024-03-28T16:49:19.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f6fc2ffd640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fb8034300 0x7f6fb80367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f6fc2ffd640 1 -- 172.21.15.170:0/2711934410 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f70e0009d40 con 0x55e291fe5180 2024-03-28T16:49:19.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fb8034300 0x7f6fb80367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e292114490 0x55e292116950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921170a0 con 0x55e292114490 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed8b8640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e292114490 0x55e292116950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fb8034300 0x7f6fb80367c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c010 tx=0x7f70e4010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed8b8640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e292114490 0x55e292116950 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 realm 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117470 con 0x55e292114490 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292118050 con 0x55e292114490 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e292118a00 0x55e29211adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29211b400 con 0x55e292118a00 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e292118a00 0x55e29211adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.843+0000 7f70ed0b7640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e292118a00 0x55e29211adf0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:19.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ef59c980 20 searching for the correct realm 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e29211d5e0 0x55e29211faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292120170 con 0x55e29211d5e0 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e29211d5e0 0x55e29211faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e29211d5e0 0x55e29211faa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ec8b6640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70e402e910 0x7f70e403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292120170 con 0x7f70e402e910 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70e402e910 0x7f70e403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70e402e910 0x7f70e403a3c0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.847+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x7f70e402e910 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x7f70e402e910 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.851+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x7f70e402e910 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x7f70e402e910 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e29211d5e0 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.855+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292118a00 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x55e292114490 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292117cf0 con 0x7f70e402e910 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:19.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29211b9e0 con 0x55e292114490 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29211b9e0 con 0x55e292114490 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29211b9b0 con 0x7f70e402e910 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 zone default found 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 4 Realm: () 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 zonegroup default 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29211c230 con 0x55e292118a00 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29211c630 con 0x55e29211d5e0 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292122110 con 0x55e29211d5e0 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 started sync module instance, tier type = 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292121870 con 0x7f70e402e910 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94431601628112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921217f0 con 0x7f70e402e910 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.859+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292122e70 con 0x55e292114490 2024-03-28T16:49:19.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.863+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94431601628112] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.863+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.863+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94431601632560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921217f0 con 0x55e292114490 2024-03-28T16:49:19.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.863+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292123510 con 0x55e29211d5e0 2024-03-28T16:49:19.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.863+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94431601632560] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94431601635344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921238c0 con 0x55e29211d5e0 2024-03-28T16:49:19.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292124230 con 0x7f70e402e910 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94431601639104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292124a80 con 0x7f70e402e910 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292125450 con 0x55e292114490 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94431601635344] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94431601639104] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94431601643344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292124560 con 0x55e292114490 2024-03-28T16:49:19.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.867+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292125ca0 con 0x55e292118a00 2024-03-28T16:49:19.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94431601645568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292124560 con 0x55e292118a00 2024-03-28T16:49:19.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921269c0 con 0x55e292118a00 2024-03-28T16:49:19.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94431601643344] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94431601645568] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94431601633616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921271c0 con 0x55e292118a00 2024-03-28T16:49:19.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.871+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292126000 con 0x55e292118a00 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94431601633616] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94431601647616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292127b30 con 0x55e292118a00 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=0 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=1 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=2 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=3 2024-03-28T16:49:19.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=4 2024-03-28T16:49:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=5 2024-03-28T16:49:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=6 2024-03-28T16:49:19.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94431601647616] v29'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 20 add_watcher() i=7 2024-03-28T16:49:19.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:19.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29212ab60 con 0x55e292114490 2024-03-28T16:49:19.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.875+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94431601653392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292127f80 con 0x55e292114490 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94431601653392] v29'117 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f6fa37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 5 note: GC not initialized 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f6fa27e4640 20 reqs_thread_entry: start 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 Filter name: none 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f6f13fff640 20 reqs_thread_entry: start 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:49:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29214c980 con 0x55e292118a00 2024-03-28T16:49:19.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:49:19.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-03-28T16:49:19.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29214e4e0 con 0x7f70e402e910 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv22 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e29214fce0 con 0x55e29211d5e0 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv2 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:19.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:19.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:19.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:19.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:19.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:19.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-03-28T16:49:19.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.879+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292153480 con 0x55e292118a00 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(65 fud.buckets [call] v29'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292152430 con 0x7f70e402e910 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv22 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:19.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:19.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.883+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292152dd0 con 0x7f70e402e910 2024-03-28T16:49:19.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v29'23 uv23 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ef59c980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-03-28T16:49:19.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ef59c980 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:19.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ef59c980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:49:19.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94431601830192 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292152d90 con 0x55e292118a00 2024-03-28T16:49:19.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== watch-notify(notify (1) cookie 94431601633616 notify 124554051590 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x55e291fe3a20 con 0x55e292118a00 2024-03-28T16:49:19.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(68 notify.6 [notify cookie 94431601830192 out=8b] v0'0 uv120 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f6fd0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051590 cookie 94431601633616 notifier 4508 bl.length()=338 2024-03-28T16:49:19.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f6fd0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-03-28T16:49:19.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f6fd0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:19.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f6fd0ff9640 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f6fb0003ea0 con 0x55e292118a00 2024-03-28T16:49:19.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0113070 con 0x55e292118a00 2024-03-28T16:49:19.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== watch-notify(notify_complete (2) cookie 94431601830192 notify 124554051590 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f70e012e020 con 0x55e292118a00 2024-03-28T16:49:19.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f6fa37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:19.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.887+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94431601653392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292152dd0 con 0x55e292114490 2024-03-28T16:49:19.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94431601653392] v29'118 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94431601628112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292152dd0 con 0x7f70e402e910 2024-03-28T16:49:19.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94431601628112] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ef59c980 20 remove_watcher() i=0 2024-03-28T16:49:19.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ef59c980 2 removed watcher, disabling cache 2024-03-28T16:49:19.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94431601632560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921213d0 con 0x55e292114490 2024-03-28T16:49:19.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94431601632560] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ef59c980 20 remove_watcher() i=1 2024-03-28T16:49:19.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.891+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94431601635344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292122530 con 0x55e29211d5e0 2024-03-28T16:49:19.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.895+0000 7f70ec8b6640 1 -- 172.21.15.170:0/2711934410 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94431601635344] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e4033040 con 0x55e29211d5e0 2024-03-28T16:49:19.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.895+0000 7f70ef59c980 20 remove_watcher() i=2 2024-03-28T16:49:19.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.895+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94431601639104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921213d0 con 0x7f70e402e910 2024-03-28T16:49:19.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.895+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94431601639104] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x7f70e402e910 2024-03-28T16:49:19.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.895+0000 7f70ef59c980 20 remove_watcher() i=3 2024-03-28T16:49:19.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.895+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94431601643344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292122530 con 0x55e292114490 2024-03-28T16:49:19.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94431601643344] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e800a4d0 con 0x55e292114490 2024-03-28T16:49:19.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ef59c980 20 remove_watcher() i=4 2024-03-28T16:49:19.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94431601645568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921213d0 con 0x55e292118a00 2024-03-28T16:49:19.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94431601645568] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0130040 con 0x55e292118a00 2024-03-28T16:49:19.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ef59c980 20 remove_watcher() i=5 2024-03-28T16:49:19.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94431601633616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e292124f50 con 0x55e292118a00 2024-03-28T16:49:19.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94431601633616] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0130040 con 0x55e292118a00 2024-03-28T16:49:19.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ef59c980 20 remove_watcher() i=6 2024-03-28T16:49:19.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.899+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94431601647616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55e2921213d0 con 0x55e292118a00 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ed0b7640 1 -- 172.21.15.170:0/2711934410 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94431601647616] v29'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70e0130040 con 0x55e292118a00 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 20 remove_watcher() i=7 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e292118a00 msgr2=0x55e29211adf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e292118a00 0x55e29211adf0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e292114490 msgr2=0x55e292116950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e292114490 0x55e292116950 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70e402e910 msgr2=0x7f70e403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70e402e910 0x7f70e403a3c0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e29211d5e0 msgr2=0x55e29211faa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e29211d5e0 0x55e29211faa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fb8034300 msgr2=0x7f6fb80367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fb8034300 0x7f6fb80367c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c010 tx=0x7f70e4010000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 msgr2=0x55e292109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e292109070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f70e00cf6c0 tx=0x7f70e00cded0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2711934410 reap_dead start 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 shutdown_connections 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2711934410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291fe5180 0x55e292109070 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 >> 172.21.15.170:0/2711934410 conn(0x55e2920fc390 msgr2=0x55e2920f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 shutdown_connections 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2711934410 wait complete. 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fc40342b0 msgr2=0x7f6fc4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fc40342b0 0x7f6fc4036770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55e2920ddf70 tx=0x7f70e400f000 comp rx=0 tx=0).stop 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2920f3410 msgr2=0x55e2920f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2920f3410 0x55e2920f3830 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f70e00e27e0 tx=0x7f70e00e40b0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 shutdown_connections 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fc40342b0 0x7f6fc4036770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/395308381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e2920f3410 0x55e2920f3830 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 >> 172.21.15.170:0/395308381 conn(0x55e2920e8220 msgr2=0x55e2920e8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 shutdown_connections 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/395308381 wait complete. 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e2920dee00 msgr2=0x55e2920e11f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e2920dee00 0x55e2920e11f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e291fe8180 msgr2=0x55e2920dc840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e291fe8180 0x55e2920dc840 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e2920e2aa0 msgr2=0x55e2920e4e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e2920e2aa0 0x55e2920e4e90 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fd40342b0 msgr2=0x7f6fd4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6fd40342b0 0x7f6fd4036770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000f70 tx=0x7f70e4001380 comp rx=0 tx=0).stop 2024-03-28T16:49:19.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 msgr2=0x55e2920d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 --2- 172.21.15.170:0/2225119456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e291f598c0 0x55e2920d5c10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f70e0005940 tx=0x7f70e00180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ed8b8640 1 -- 172.21.15.170:0/2225119456 reap_dead start 2024-03-28T16:49:19.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 shutdown_connections 2024-03-28T16:49:19.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 >> 172.21.15.170:0/2225119456 conn(0x55e291fdf720 msgr2=0x55e2920c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 shutdown_connections 2024-03-28T16:49:19.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.903+0000 7f70ef59c980 1 -- 172.21.15.170:0/2225119456 wait complete. 2024-03-28T16:49:19.914 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1'] 2024-03-28T16:49:19.914 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1'] 2024-03-28T16:49:19.914 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de4513980 1 Processor -- start 2024-03-28T16:49:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de4513980 1 -- start start 2024-03-28T16:49:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de4513980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d51070a060 0x55d51070c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de4513980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d5106544c0 con 0x55d51067c050 2024-03-28T16:49:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de4513980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d51061b0f0 con 0x55d51070a060 2024-03-28T16:49:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de182d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d51070a060 0x55d51070c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de202e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55d51067c050 0x55d510708940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38236/0 (socket says 172.21.15.170:38236) 2024-03-28T16:49:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de202e640 1 -- 172.21.15.170:0/46417007 learned_addr learned my addr 172.21.15.170:0/46417007 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de182d640 1 -- 172.21.15.170:0/46417007 >> v1:172.21.15.119:6789/0 conn(0x55d51067c050 legacy=0x55d510708940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.991+0000 7f7de182d640 1 -- 172.21.15.170:0/46417007 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d51070ca50 con 0x55d51070a060 2024-03-28T16:49:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de182d640 1 --2- 172.21.15.170:0/46417007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51070a060 0x55d51070c480 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800a480 tx=0x7f7dd801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561f3529dd3b3b14 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de102c640 1 -- 172.21.15.170:0/46417007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd80ab540 con 0x55d51070a060 2024-03-28T16:49:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de102c640 1 -- 172.21.15.170:0/46417007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80ab6a0 con 0x55d51070a060 2024-03-28T16:49:19.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de102c640 1 -- 172.21.15.170:0/46417007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd80ab960 con 0x55d51070a060 2024-03-28T16:49:19.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de4513980 1 -- 172.21.15.170:0/46417007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51070a060 msgr2=0x55d51070c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:19.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de4513980 1 --2- 172.21.15.170:0/46417007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51070a060 0x55d51070c480 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800a480 tx=0x7f7dd801b280 comp rx=0 tx=0).stop 2024-03-28T16:49:19.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de4513980 1 -- 172.21.15.170:0/46417007 shutdown_connections 2024-03-28T16:49:19.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de4513980 1 --2- 172.21.15.170:0/46417007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51070a060 0x55d51070c480 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:19.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de4513980 1 -- 172.21.15.170:0/46417007 >> 172.21.15.170:0/46417007 conn(0x55d510702670 msgr2=0x55d510704a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:19.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de4513980 1 -- 172.21.15.170:0/46417007 shutdown_connections 2024-03-28T16:49:19.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:19.995+0000 7f7de4513980 1 -- 172.21.15.170:0/46417007 wait complete. 2024-03-28T16:49:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de4513980 1 Processor -- start 2024-03-28T16:49:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de4513980 1 -- start start 2024-03-28T16:49:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de4513980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de4513980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d510618620 con 0x55d51067c050 2024-03-28T16:49:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107f8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56702/0 (socket says 172.21.15.170:56702) 2024-03-28T16:49:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de202e640 1 -- 172.21.15.170:0/3761398154 learned_addr learned my addr 172.21.15.170:0/3761398154 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.007+0000 7f7de202e640 1 -- 172.21.15.170:0/3761398154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8005d30 con 0x55d51067c050 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de202e640 1 --2- 172.21.15.170:0/3761398154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107f8910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4006fb0 tx=0x7f7dd4001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ce7b0a0ea76bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7cd0608640 1 -- 172.21.15.170:0/3761398154 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4004530 con 0x55d51067c050 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7cd0608640 1 -- 172.21.15.170:0/3761398154 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4004690 con 0x55d51067c050 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7cd0608640 1 -- 172.21.15.170:0/3761398154 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd40049b0 con 0x55d51067c050 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- 172.21.15.170:0/3761398154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 msgr2=0x55d5107f8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 --2- 172.21.15.170:0/3761398154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107f8910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4006fb0 tx=0x7f7dd4001940 comp rx=0 tx=0).stop 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- 172.21.15.170:0/3761398154 shutdown_connections 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 --2- 172.21.15.170:0/3761398154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107f8910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- 172.21.15.170:0/3761398154 >> 172.21.15.170:0/3761398154 conn(0x55d510702670 msgr2=0x55d5107f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- 172.21.15.170:0/3761398154 shutdown_connections 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- 172.21.15.170:0/3761398154 wait complete. 2024-03-28T16:49:20.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 Processor -- start 2024-03-28T16:49:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- start start 2024-03-28T16:49:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d510613e00 con 0x55d51067c050 2024-03-28T16:49:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107ff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56714/0 (socket says 172.21.15.170:56714) 2024-03-28T16:49:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de202e640 1 -- 172.21.15.170:0/2125384631 learned_addr learned my addr 172.21.15.170:0/2125384631 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de202e640 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5106544c0 con 0x55d51067c050 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de202e640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107ff880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d5107f97a0 tx=0x7f7dd4000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7ccee05640 1 -- 172.21.15.170:0/2125384631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4013760 con 0x55d51067c050 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7ccee05640 1 -- 172.21.15.170:0/2125384631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd40138c0 con 0x55d51067c050 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5107ffdc0 con 0x55d51067c050 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7ccee05640 1 -- 172.21.15.170:0/2125384631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4013b60 con 0x55d51067c050 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.011+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5108001f0 con 0x55d51067c050 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7ccee05640 1 -- 172.21.15.170:0/2125384631 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7dd4028460 con 0x55d51067c050 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7ccee05640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cc80342b0 0x7f7cc8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7ccee05640 1 -- 172.21.15.170:0/2125384631 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7dd4049450 con 0x55d51067c050 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de182d640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cc80342b0 0x7f7cc8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d510804450 0x55d510806840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510806d80 con 0x55d510804450 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de282f640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d510804450 0x55d510806840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de182d640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cc80342b0 0x7f7cc8036770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80ad000 tx=0x7f7dd80a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de282f640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d510804450 0x55d510806840 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de282f640 1 -- 172.21.15.170:0/2125384631 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d510804450 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108070e0 con 0x55d510804450 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de282f640 1 -- 172.21.15.170:0/2125384631 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d510804450 2024-03-28T16:49:20.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510808e00 0x55d51080b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51080b730 con 0x55d510808e00 2024-03-28T16:49:20.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de202e640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510808e00 0x55d51080b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de202e640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510808e00 0x55d51080b1f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de202e640 1 -- 172.21.15.170:0/2125384631 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f7dd4024020 con 0x55d510808e00 2024-03-28T16:49:20.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510807c10 con 0x55d510804450 2024-03-28T16:49:20.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de282f640 1 -- 172.21.15.170:0/2125384631 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d510804450 2024-03-28T16:49:20.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d51080cac0 0x55d51080eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51080f500 con 0x55d51080cac0 2024-03-28T16:49:20.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.015+0000 7f7de182d640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d51080cac0 0x55d51080eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de182d640 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d51080cac0 0x55d51080eeb0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de182d640 1 -- 172.21.15.170:0/2125384631 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d51080cac0 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de4513980 1 Processor -- start 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de4513980 1 -- start start 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de4513980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51081d430 0x55d51081d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de4513980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5105ace30 con 0x55d51081d430 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51081d430 0x55d51081d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51081d430 0x55d51081d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56728/0 (socket says 172.21.15.170:56728) 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de202e640 1 -- 172.21.15.170:0/1349287266 learned_addr learned my addr 172.21.15.170:0/1349287266 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de202e640 1 -- 172.21.15.170:0/1349287266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d51081dd90 con 0x55d51081d430 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de202e640 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51081d430 0x55d51081d850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4047990 tx=0x7f7dd40490b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7cc7fff640 1 -- 172.21.15.170:0/1349287266 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd40115e0 con 0x55d51081d430 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d51081dfe0 con 0x55d51081d430 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7cc7fff640 1 -- 172.21.15.170:0/1349287266 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4012aa0 con 0x55d51081d430 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7cc7fff640 1 -- 172.21.15.170:0/1349287266 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4012d40 con 0x55d51081d430 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d51081eb30 con 0x55d51081d430 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7cc7fff640 1 -- 172.21.15.170:0/1349287266 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7dd4007a70 con 0x55d51081d430 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7cc7fff640 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cbc0342b0 0x7f7cbc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7cc7fff640 1 -- 172.21.15.170:0/1349287266 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7dd404ade0 con 0x55d51081d430 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de182d640 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cbc0342b0 0x7f7cbc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.019+0000 7f7de4513980 1 Processor -- start 2024-03-28T16:49:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- start start 2024-03-28T16:49:20.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de182d640 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cbc0342b0 0x7f7cbc036770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55d510803eb0 tx=0x7f7dd8005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d51082a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5105a0790 con 0x55d51082a230 2024-03-28T16:49:20.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d51082a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d51082a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56732/0 (socket says 172.21.15.170:56732) 2024-03-28T16:49:20.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 -- 172.21.15.170:0/1553698835 learned_addr learned my addr 172.21.15.170:0/1553698835 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 -- 172.21.15.170:0/1553698835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d51082ac20 con 0x55d51082a230 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 --2- 172.21.15.170:0/1553698835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d51082a650 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7dd401d690 tx=0x7f7dd4006fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac1901808b0ae81 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7cc5ffb640 1 -- 172.21.15.170:0/1553698835 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4058070 con 0x55d51082a230 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7cc5ffb640 1 -- 172.21.15.170:0/1553698835 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd40327a0 con 0x55d51082a230 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7cc5ffb640 1 -- 172.21.15.170:0/1553698835 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4014dd0 con 0x55d51082a230 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- 172.21.15.170:0/1553698835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 msgr2=0x55d51082a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 --2- 172.21.15.170:0/1553698835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d51082a650 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7dd401d690 tx=0x7f7dd4006fe0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- 172.21.15.170:0/1553698835 shutdown_connections 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 --2- 172.21.15.170:0/1553698835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d51082a650 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- 172.21.15.170:0/1553698835 >> 172.21.15.170:0/1553698835 conn(0x55d510826570 msgr2=0x55d5108245d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- 172.21.15.170:0/1553698835 shutdown_connections 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- 172.21.15.170:0/1553698835 wait complete. 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 Processor -- start 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- start start 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d5108310f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d510831630 con 0x55d51082a230 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d5108310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d5108310f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56740/0 (socket says 172.21.15.170:56740) 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 learned_addr learned my addr 172.21.15.170:0/650713591 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d51081dd90 con 0x55d51082a230 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de202e640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d5108310f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4032910 tx=0x7f7dd404ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7cb3fff640 1 -- 172.21.15.170:0/650713591 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4046590 con 0x55d51082a230 2024-03-28T16:49:20.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7cb3fff640 1 -- 172.21.15.170:0/650713591 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4011760 con 0x55d51082a230 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d51081dd90 con 0x55d51082a230 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7cb3fff640 1 -- 172.21.15.170:0/650713591 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd4007410 con 0x55d51082a230 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.023+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d51081dd90 con 0x55d51082a230 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7cb3fff640 1 -- 172.21.15.170:0/650713591 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7dd4007570 con 0x55d51082a230 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7cb3fff640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cac0342b0 0x7f7cac036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7cb3fff640 1 -- 172.21.15.170:0/650713591 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f7dd4070ab0 con 0x55d51082a230 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de182d640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cac0342b0 0x7f7cac036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d51083c390 0x55d51083e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083efa0 con 0x55d51083c390 2024-03-28T16:49:20.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de282f640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d51083c390 0x55d51083e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de182d640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cac0342b0 0x7f7cac036770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000c10 tx=0x7f7dd80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de282f640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d51083c390 0x55d51083e850 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 realm 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083f370 con 0x55d51083c390 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083ff50 con 0x55d51083c390 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510840900 0x55d510842cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108432b0 con 0x55d510840900 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de202e640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510840900 0x55d510842cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.027+0000 7f7de202e640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510840900 0x55d510842cf0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de4513980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de4513980 20 searching for the correct realm 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5108454b0 0x55d510847970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510848040 con 0x55d5108454b0 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de182d640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5108454b0 0x55d510847970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de182d640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5108454b0 0x55d510847970 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de182d640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7dd80da040 0x7f7dd80cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510848040 con 0x7f7dd80da040 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7dd80da040 0x7f7dd80cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7dd80da040 0x7f7dd80cc420 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.031+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x7f7dd80da040 2024-03-28T16:49:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x7f7dd80da040 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.035+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x7f7dd80da040 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x7f7dd80da040 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d5108454b0 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.039+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d510840900 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x55d51083c390 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51083fbf0 con 0x7f7dd80da040 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:20.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108438b0 con 0x55d51083c390 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108438b0 con 0x55d51083c390 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510843860 con 0x7f7dd80da040 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 zone default found 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 4 Realm: () 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 zonegroup default 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510844100 con 0x55d510840900 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510849e60 con 0x55d5108454b0 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084a1c0 con 0x55d5108454b0 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 started sync module instance, tier type = 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510849740 con 0x7f7dd80da040 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94373593518752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510849af0 con 0x7f7dd80da040 2024-03-28T16:49:20.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.043+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108496c0 con 0x55d51083c390 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.047+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94373593518752] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.047+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.047+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94373593523488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084ade0 con 0x55d51083c390 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.047+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084b750 con 0x55d5108454b0 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94373593523488] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94373593524544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084ad60 con 0x55d5108454b0 2024-03-28T16:49:20.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084c440 con 0x7f7dd80da040 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94373593530544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084cc90 con 0x7f7dd80da040 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084d600 con 0x55d51083c390 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94373593524544] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94373593530544] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94373593534816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084cbf0 con 0x55d51083c390 2024-03-28T16:49:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.051+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084d580 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.055+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94373593534816] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.055+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.055+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94373593532544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084de80 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.055+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084e680 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.055+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94373593532544] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94373593536992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084ee80 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084f7f0 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94373593536992] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94373593543504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084fc10 con 0x55d510840900 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 20 add_watcher() i=0 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 20 add_watcher() i=1 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 20 add_watcher() i=2 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 20 add_watcher() i=3 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 20 add_watcher() i=4 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 20 add_watcher() i=5 2024-03-28T16:49:20.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.059+0000 7f7de4513980 20 add_watcher() i=6 2024-03-28T16:49:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94373593543504] v29'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 add_watcher() i=7 2024-03-28T16:49:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510852eb0 con 0x55d51083c390 2024-03-28T16:49:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94373593544928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108501a0 con 0x55d51083c390 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94373593544928] v29'119 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7c98fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 5 note: GC not initialized 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7c137fe640 20 reqs_thread_entry: start 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 Filter name: none 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7c117fa640 20 reqs_thread_entry: start 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510874ea0 con 0x55d51083c390 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510876a00 con 0x7f7dd80da040 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:20.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.063+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108781d0 con 0x55d5108454b0 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv2 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-03-28T16:49:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51087c960 con 0x55d510840900 2024-03-28T16:49:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(65 fud.buckets [call] v29'5 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510882760 con 0x55d5108454b0 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'3 uv3 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.067+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 7.9 7:93e5b521:::notify.7:head [notify cookie 94373593712256 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51087cd50 con 0x55d510840900 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== watch-notify(notify (1) cookie 94373593543504 notify 124554051591 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55d51082ac20 con 0x55d510840900 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7cc5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051591 cookie 94373593543504 notifier 4515 bl.length()=725 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7cc5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7cc5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7cc5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7cc5ffb640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f7ca8003ea0 con 0x55d510840900 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(67 notify.7 [notify cookie 94373593712256 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(68 notify.7 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== watch-notify(notify_complete (2) cookie 94373593712256 notify 124554051591 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55d5108001f0 con 0x55d510840900 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de4513980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_sources= new_sources= 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de4513980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential sources added=[] removed=[] 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de4513980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_dests= new_dests= 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de4513980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential dests added=[] removed=[] 2024-03-28T16:49:20.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.071+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510878660 con 0x7f7dd80da040 2024-03-28T16:49:20.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(69 foo.buckets [call] v29'24 uv24 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510878660 con 0x7f7dd80da040 2024-03-28T16:49:20.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v29'25 uv25 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de4513980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-03-28T16:49:20.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de4513980 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:20.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de4513980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:49:20.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94373593712224 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d5108814e0 con 0x55d510840900 2024-03-28T16:49:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== watch-notify(notify (1) cookie 94373593536992 notify 124554051592 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x55d5108001f0 con 0x55d510840900 2024-03-28T16:49:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7cc57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051592 cookie 94373593536992 notifier 4515 bl.length()=314 2024-03-28T16:49:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(71 notify.6 [notify cookie 94373593712224 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7cc57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-03-28T16:49:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7cc57fa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:20.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.075+0000 7f7cc57fa640 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f7c18003400 con 0x55d510840900 2024-03-28T16:49:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== watch-notify(notify_complete (2) cookie 94373593712224 notify 124554051592 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f7dd4061040 con 0x55d510840900 2024-03-28T16:49:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7c98fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94373593544928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d510878660 con 0x55d51083c390 2024-03-28T16:49:20.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94373593544928] v29'120 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94373593518752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084f8e0 con 0x7f7dd80da040 2024-03-28T16:49:20.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94373593518752] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de4513980 20 remove_watcher() i=0 2024-03-28T16:49:20.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de4513980 2 removed watcher, disabling cache 2024-03-28T16:49:20.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.079+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94373593523488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084ee80 con 0x55d51083c390 2024-03-28T16:49:20.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.083+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94373593523488] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.083+0000 7f7de4513980 20 remove_watcher() i=1 2024-03-28T16:49:20.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.083+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94373593524544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084f8e0 con 0x55d5108454b0 2024-03-28T16:49:20.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.083+0000 7f7de182d640 1 -- 172.21.15.170:0/650713591 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94373593524544] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd80abb60 con 0x55d5108454b0 2024-03-28T16:49:20.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.083+0000 7f7de4513980 20 remove_watcher() i=2 2024-03-28T16:49:20.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.083+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94373593530544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084a9c0 con 0x7f7dd80da040 2024-03-28T16:49:20.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.087+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94373593530544] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x7f7dd80da040 2024-03-28T16:49:20.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.087+0000 7f7de4513980 20 remove_watcher() i=3 2024-03-28T16:49:20.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.087+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94373593534816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084a9c0 con 0x55d51083c390 2024-03-28T16:49:20.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.087+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94373593534816] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7ddc0083f0 con 0x55d51083c390 2024-03-28T16:49:20.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.087+0000 7f7de4513980 20 remove_watcher() i=4 2024-03-28T16:49:20.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.087+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94373593532544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084a9c0 con 0x55d510840900 2024-03-28T16:49:20.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.091+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94373593532544] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.091+0000 7f7de4513980 20 remove_watcher() i=5 2024-03-28T16:49:20.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.091+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94373593536992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084a9c0 con 0x55d510840900 2024-03-28T16:49:20.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.091+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94373593536992] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.091+0000 7f7de4513980 20 remove_watcher() i=6 2024-03-28T16:49:20.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.091+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94373593543504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55d51084edc0 con 0x55d510840900 2024-03-28T16:49:20.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de202e640 1 -- 172.21.15.170:0/650713591 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94373593543504] v29'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7dd406c070 con 0x55d510840900 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 20 remove_watcher() i=7 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510840900 msgr2=0x55d510842cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510840900 0x55d510842cf0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d51083c390 msgr2=0x55d51083e850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d51083c390 0x55d51083e850 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7dd80da040 msgr2=0x7f7dd80cc420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f7dd80da040 0x7f7dd80cc420 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5108454b0 msgr2=0x55d510847970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5108454b0 0x55d510847970 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cac0342b0 msgr2=0x7f7cac036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cac0342b0 0x7f7cac036770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000c10 tx=0x7f7dd80a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 msgr2=0x55d5108310f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/650713591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51082a230 0x55d5108310f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4032910 tx=0x7f7dd404ced0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de282f640 1 -- 172.21.15.170:0/650713591 reap_dead start 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 shutdown_connections 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 >> 172.21.15.170:0/650713591 conn(0x55d510826570 msgr2=0x55d510823bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 shutdown_connections 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/650713591 wait complete. 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cbc0342b0 msgr2=0x7f7cbc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cbc0342b0 0x7f7cbc036770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55d510803eb0 tx=0x7f7dd8005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51081d430 msgr2=0x55d51081d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51081d430 0x55d51081d850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4047990 tx=0x7f7dd40490b0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 shutdown_connections 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cbc0342b0 0x7f7cbc036770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/1349287266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51081d430 0x55d51081d850 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 >> 172.21.15.170:0/1349287266 conn(0x55d510812240 msgr2=0x55d510812d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 shutdown_connections 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/1349287266 wait complete. 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510808e00 msgr2=0x55d51080b1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d510808e00 0x55d51080b1f0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d510804450 msgr2=0x55d510806840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d510804450 0x55d510806840 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d51080cac0 msgr2=0x55d51080eeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d51080cac0 0x55d51080eeb0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cc80342b0 msgr2=0x7f7cc8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f7cc80342b0 0x7f7cc8036770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80ad000 tx=0x7f7dd80a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 msgr2=0x55d5107ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 --2- 172.21.15.170:0/2125384631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51067c050 0x55d5107ff880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55d5107f97a0 tx=0x7f7dd4000f50 comp rx=0 tx=0).stop 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de282f640 1 -- 172.21.15.170:0/2125384631 reap_dead start 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 shutdown_connections 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 >> 172.21.15.170:0/2125384631 conn(0x55d510702670 msgr2=0x55d5107f2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 shutdown_connections 2024-03-28T16:49:20.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.095+0000 7f7de4513980 1 -- 172.21.15.170:0/2125384631 wait complete. 2024-03-28T16:49:20.105 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-03-28T16:49:20.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-03-28T16:49:20.105 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-03-28T16:49:20.183 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:20.183 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:20.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1d56e980 1 Processor -- start 2024-03-28T16:49:20.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1d56e980 1 -- start start 2024-03-28T16:49:20.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1d56e980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56163976b350 0x56163976d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1d56e980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5616396b3520 con 0x5616396db8b0 2024-03-28T16:49:20.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1d56e980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56163967a0f0 con 0x56163976b350 2024-03-28T16:49:20.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1a888640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56163976b350 0x56163976d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1b089640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5616396db8b0 0x561639769c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38240/0 (socket says 172.21.15.170:38240) 2024-03-28T16:49:20.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.183+0000 7f4b1b089640 1 -- 172.21.15.170:0/3966163088 learned_addr learned my addr 172.21.15.170:0/3966163088 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:20.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1a888640 1 -- 172.21.15.170:0/3966163088 >> v1:172.21.15.119:6789/0 conn(0x5616396db8b0 legacy=0x561639769c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1a888640 1 -- 172.21.15.170:0/3966163088 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56163976ecc0 con 0x56163976b350 2024-03-28T16:49:20.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1a888640 1 --2- 172.21.15.170:0/3966163088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56163976b350 0x56163976d740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400a420 tx=0x7f4b0401afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8ee67e35453351 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1a087640 1 -- 172.21.15.170:0/3966163088 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b040aa530 con 0x56163976b350 2024-03-28T16:49:20.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1a087640 1 -- 172.21.15.170:0/3966163088 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b040a6050 con 0x56163976b350 2024-03-28T16:49:20.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1a087640 1 -- 172.21.15.170:0/3966163088 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b040aa800 con 0x56163976b350 2024-03-28T16:49:20.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3966163088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56163976b350 msgr2=0x56163976d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/3966163088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56163976b350 0x56163976d740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400a420 tx=0x7f4b0401afd0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3966163088 shutdown_connections 2024-03-28T16:49:20.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/3966163088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56163976b350 0x56163976d740 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3966163088 >> 172.21.15.170:0/3966163088 conn(0x561639761710 msgr2=0x561639763b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3966163088 shutdown_connections 2024-03-28T16:49:20.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.187+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3966163088 wait complete. 2024-03-28T16:49:20.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1d56e980 1 Processor -- start 2024-03-28T16:49:20.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1d56e980 1 -- start start 2024-03-28T16:49:20.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1d56e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639850ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1d56e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561639677620 con 0x5616396db8b0 2024-03-28T16:49:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639850ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639850ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56762/0 (socket says 172.21.15.170:56762) 2024-03-28T16:49:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1b089640 1 -- 172.21.15.170:0/1925439790 learned_addr learned my addr 172.21.15.170:0/1925439790 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1b089640 1 -- 172.21.15.170:0/1925439790 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0400a100 con 0x5616396db8b0 2024-03-28T16:49:20.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4b1b089640 1 --2- 172.21.15.170:0/1925439790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639850ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c001c90 tx=0x7f4b0c00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42801a27466acf15 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4a0960a640 1 -- 172.21.15.170:0/1925439790 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c00e690 con 0x5616396db8b0 2024-03-28T16:49:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4a0960a640 1 -- 172.21.15.170:0/1925439790 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c00e7f0 con 0x5616396db8b0 2024-03-28T16:49:20.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.203+0000 7f4a0960a640 1 -- 172.21.15.170:0/1925439790 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c00eaf0 con 0x5616396db8b0 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- 172.21.15.170:0/1925439790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 msgr2=0x561639850ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/1925439790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639850ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c001c90 tx=0x7f4b0c00b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- 172.21.15.170:0/1925439790 shutdown_connections 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/1925439790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639850ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- 172.21.15.170:0/1925439790 >> 172.21.15.170:0/1925439790 conn(0x561639761710 msgr2=0x56163984ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- 172.21.15.170:0/1925439790 shutdown_connections 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- 172.21.15.170:0/1925439790 wait complete. 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 Processor -- start 2024-03-28T16:49:20.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- start start 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639857cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561639672e00 con 0x5616396db8b0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639857cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639857cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56772/0 (socket says 172.21.15.170:56772) 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1b089640 1 -- 172.21.15.170:0/192498752 learned_addr learned my addr 172.21.15.170:0/192498752 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1b089640 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616396b3520 con 0x5616396db8b0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1b089640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639857cf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c001ee0 tx=0x7f4b0c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4a07e07640 1 -- 172.21.15.170:0/192498752 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c01f070 con 0x5616396db8b0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4a07e07640 1 -- 172.21.15.170:0/192498752 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c018830 con 0x5616396db8b0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561639858230 con 0x5616396db8b0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4a07e07640 1 -- 172.21.15.170:0/192498752 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c018ad0 con 0x5616396db8b0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.207+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561639858660 con 0x5616396db8b0 2024-03-28T16:49:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4a07e07640 1 -- 172.21.15.170:0/192498752 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f4b0c00a040 con 0x5616396db8b0 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4a07e07640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a000342b0 0x7f4a00036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4a07e07640 1 -- 172.21.15.170:0/192498752 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f4b0c049490 con 0x5616396db8b0 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1a888640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a000342b0 0x7f4a00036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56163976b350 0x56163985e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985eeb0 con 0x56163976b350 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b88a640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56163976b350 0x56163985e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1a888640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a000342b0 0x7f4a00036770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b040ac000 tx=0x7f4b040a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b88a640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56163976b350 0x56163985e970 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b88a640 1 -- 172.21.15.170:0/192498752 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x56163976b350 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985f210 con 0x56163976b350 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b88a640 1 -- 172.21.15.170:0/192498752 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4b140083f0 con 0x56163976b350 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561639860ba0 0x561639862f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639863720 con 0x561639860ba0 2024-03-28T16:49:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b089640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561639860ba0 0x561639862f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b089640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561639860ba0 0x561639862f90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b089640 1 -- 172.21.15.170:0/192498752 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f4b0c014070 con 0x561639860ba0 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985fa10 con 0x56163976b350 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1b88a640 1 -- 172.21.15.170:0/192498752 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4b140083f0 con 0x56163976b350 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561639864b40 0x561639866f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398675d0 con 0x561639864b40 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.211+0000 7f4b1a888640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561639864b40 0x561639866f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1a888640 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561639864b40 0x561639866f30 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1a888640 1 -- 172.21.15.170:0/192498752 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f4b040b1040 con 0x561639864b40 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1d56e980 1 Processor -- start 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1d56e980 1 -- start start 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1d56e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398754f0 0x561639875910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1d56e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56163960be30 con 0x5616398754f0 2024-03-28T16:49:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398754f0 0x561639875910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398754f0 0x561639875910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56776/0 (socket says 172.21.15.170:56776) 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1b089640 1 -- 172.21.15.170:0/3804259717 learned_addr learned my addr 172.21.15.170:0/3804259717 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1b089640 1 -- 172.21.15.170:0/3804259717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561639875e50 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1b089640 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398754f0 0x561639875910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c028000 tx=0x7f4b0c02dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4a05602640 1 -- 172.21.15.170:0/3804259717 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c0492c0 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5616398760a0 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561639876bf0 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4a05602640 1 -- 172.21.15.170:0/3804259717 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c0419e0 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4a05602640 1 -- 172.21.15.170:0/3804259717 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c041c80 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4a05602640 1 -- 172.21.15.170:0/3804259717 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f4b0c0460e0 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4a05602640 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49f4034300 0x7f49f40367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4a05602640 1 -- 172.21.15.170:0/3804259717 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f4b0c032040 con 0x5616398754f0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.215+0000 7f4b1a888640 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49f4034300 0x7f49f40367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 Processor -- start 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- start start 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398827c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616395ff790 con 0x5616398823c0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1a888640 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49f4034300 0x7f49f40367c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x56163985fd70 tx=0x7f4b04005d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398827c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56778/0 (socket says 172.21.15.170:56778) 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 -- 172.21.15.170:0/204268721 learned_addr learned my addr 172.21.15.170:0/204268721 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 -- 172.21.15.170:0/204268721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561639882d90 con 0x5616398823c0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 --2- 172.21.15.170:0/204268721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398827c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c041970 tx=0x7f4b0c052cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b10ba52222fd9d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f49feffd640 1 -- 172.21.15.170:0/204268721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c041de0 con 0x5616398823c0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f49feffd640 1 -- 172.21.15.170:0/204268721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c016a50 con 0x5616398823c0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f49feffd640 1 -- 172.21.15.170:0/204268721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c046650 con 0x5616398823c0 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- 172.21.15.170:0/204268721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 msgr2=0x5616398827c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/204268721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398827c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c041970 tx=0x7f4b0c052cc0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- 172.21.15.170:0/204268721 shutdown_connections 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/204268721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398827c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- 172.21.15.170:0/204268721 >> 172.21.15.170:0/204268721 conn(0x56163987e750 msgr2=0x56163987c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- 172.21.15.170:0/204268721 shutdown_connections 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- 172.21.15.170:0/204268721 wait complete. 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 Processor -- start 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- start start 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398890e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1d56e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561639889620 con 0x5616398823c0 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398890e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56782/0 (socket says 172.21.15.170:56782) 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 learned_addr learned my addr 172.21.15.170:0/4235887733 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561639876bf0 con 0x5616398823c0 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.219+0000 7f4b1b089640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398890e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c035010 tx=0x7f4b0c034f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f49fd7fa640 1 -- 172.21.15.170:0/4235887733 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c0163b0 con 0x5616398823c0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f49fd7fa640 1 -- 172.21.15.170:0/4235887733 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c016510 con 0x5616398823c0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561639876bf0 con 0x5616398823c0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f49fd7fa640 1 -- 172.21.15.170:0/4235887733 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b0c0167d0 con 0x5616398823c0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561639876bf0 con 0x5616398823c0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f49fd7fa640 1 -- 172.21.15.170:0/4235887733 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f4b0c03a070 con 0x5616398823c0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f49fd7fa640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49e40342b0 0x7f49e4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f49fd7fa640 1 -- 172.21.15.170:0/4235887733 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f4b0c073b00 con 0x5616398823c0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1a888640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49e40342b0 0x7f49e4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561639894390 0x561639896850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639896fa0 con 0x561639894390 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1b88a640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561639894390 0x561639896850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1a888640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49e40342b0 0x7f49e4036770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b040ad040 tx=0x7f4b040a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1b88a640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561639894390 0x561639896850 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 realm 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897370 con 0x561639894390 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897f20 con 0x561639894390 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616398988d0 0x56163989acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989b310 con 0x5616398988d0 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.223+0000 7f4b1b089640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616398988d0 0x56163989acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b089640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616398988d0 0x56163989acc0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1d56e980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1d56e980 20 searching for the correct realm 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56163989d4b0 0x56163989f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989fff0 con 0x56163989d4b0 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1a888640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56163989d4b0 0x56163989f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1a888640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56163989d4b0 0x56163989f970 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1a888640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4b040b6050 0x7f4b040be420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989fff0 con 0x7f4b040b6050 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b88a640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4b040b6050 0x7f4b040be420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b88a640 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4b040b6050 0x7f4b040be420 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.227+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x7f4b040b6050 2024-03-28T16:49:20.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x7f4b040b6050 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.231+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x7f4b040b6050 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.235+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x7f4b040b6050 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x56163989d4b0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x5616398988d0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x561639894390 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x561639897bc0 con 0x7f4b040b6050 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989b8f0 con 0x561639894390 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989b8f0 con 0x561639894390 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989b8c0 con 0x7f4b040b6050 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 zone default found 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 4 Realm: () 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 zonegroup default 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989c190 con 0x5616398988d0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163989c590 con 0x56163989d4b0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a1fc0 con 0x56163989d4b0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.239+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 20 started sync module instance, tier type = 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a16f0 con 0x7f4b040b6050 2024-03-28T16:49:20.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94653454619216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a1aa0 con 0x7f4b040b6050 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a2d20 con 0x561639894390 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94653454619216] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94653454623712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a2d20 con 0x561639894390 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.243+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a37c0 con 0x56163989d4b0 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94653454627488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a4010 con 0x56163989d4b0 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a49e0 con 0x7f4b040b6050 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94653454623712] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94653454632160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a4d90 con 0x7f4b040b6050 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a5700 con 0x561639894390 2024-03-28T16:49:20.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94653454627488] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.247+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94653454632160] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.251+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.251+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94653454624768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a5260 con 0x561639894390 2024-03-28T16:49:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.251+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a5aa0 con 0x5616398988d0 2024-03-28T16:49:20.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.251+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94653454624768] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.251+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.251+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94653454635616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a5680 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.251+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a6790 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94653454641904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a73b0 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a7d60 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94653454635616] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94653454645376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a8000 con 0x5616398988d0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=0 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=1 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=2 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=3 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=4 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=5 2024-03-28T16:49:20.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94653454641904] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=6 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94653454645376] v29'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 20 add_watcher() i=7 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.255+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a5e40 con 0x561639894390 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94653454642960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a5e40 con 0x561639894390 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94653454642960] v29'121 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f49d1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1d56e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1d56e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b3650 con 0x561639894390 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.259+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b3a90 con 0x5616398988d0 2024-03-28T16:49:20.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.263+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.263+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b3a90 con 0x56163989d4b0 2024-03-28T16:49:20.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.263+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.263+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b4460 con 0x561639894390 2024-03-28T16:49:20.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.263+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'122 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.263+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b16c0 con 0x7f4b040b6050 2024-03-28T16:49:20.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.267+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'35 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.267+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b16c0 con 0x56163989d4b0 2024-03-28T16:49:20.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.267+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'23 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.267+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b4fd0 con 0x561639894390 2024-03-28T16:49:20.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.267+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.267+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b4fd0 con 0x5616398988d0 2024-03-28T16:49:20.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.271+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'23 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.271+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b5500 con 0x7f4b040b6050 2024-03-28T16:49:20.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.271+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.271+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b5500 con 0x56163989d4b0 2024-03-28T16:49:20.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b59a0 con 0x7f4b040b6050 2024-03-28T16:49:20.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'36 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b59a0 con 0x5616398988d0 2024-03-28T16:49:20.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b0580 con 0x7f4b040b6050 2024-03-28T16:49:20.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.275+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b0b90 con 0x5616398988d0 2024-03-28T16:49:20.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.279+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'30 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.279+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b0b90 con 0x561639894390 2024-03-28T16:49:20.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.279+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'123 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.279+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.283+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'29 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.283+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x7f4b040b6050 2024-03-28T16:49:20.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.283+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'37 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.283+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.283+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.283+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x561639894390 2024-03-28T16:49:20.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'23 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x561639894390 2024-03-28T16:49:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'124 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'30 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x7f4b040b6050 2024-03-28T16:49:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'25 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.287+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'24 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.291+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x5616398988d0 2024-03-28T16:49:20.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.295+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.295+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.295+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.295+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x5616398988d0 2024-03-28T16:49:20.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.299+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'24 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.299+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x56163989d4b0 2024-03-28T16:49:20.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.299+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'31 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.299+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x5616398988d0 2024-03-28T16:49:20.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'31 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398b03b0 con 0x7f4b040b6050 2024-03-28T16:49:20.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f49d0fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:20.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1d56e980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1d56e980 20 Filter name: none 2024-03-28T16:49:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f494a7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1d56e980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:49:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1d56e980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.303+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985bc00 con 0x5616398988d0 2024-03-28T16:49:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(94 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:49:20.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-03-28T16:49:20.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:49:20.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:20.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985bbd0 con 0x5616398988d0 2024-03-28T16:49:20.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(95 fud.buckets [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 155+0+15 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-03-28T16:49:20.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:49:20.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:96 7.15 7:a93a5511:::notify.2:head [notify cookie 94653454333744 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398d2c00 con 0x56163989d4b0 2024-03-28T16:49:20.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== watch-notify(notify (1) cookie 94653454627488 notify 124554051593 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x56163976ecc0 con 0x56163989d4b0 2024-03-28T16:49:20.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f49feffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051593 cookie 94653454627488 notifier 4522 bl.length()=170 2024-03-28T16:49:20.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(96 notify.2 [notify cookie 94653454333744 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f49feffd640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f49e0003d60 con 0x56163989d4b0 2024-03-28T16:49:20.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(97 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== watch-notify(notify_complete (2) cookie 94653454333744 notify 124554051593 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f4b040cd700 con 0x56163989d4b0 2024-03-28T16:49:20.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:98 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985b730 con 0x5616398988d0 2024-03-28T16:49:20.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(98 p5YnriCv1nAtykxBrupQ [delete] v29'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 removing email index: 2024-03-28T16:49:20.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 removing user buckets index 2024-03-28T16:49:20.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-03-28T16:49:20.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.307+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:99 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94653454333744 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398d2c00 con 0x5616398988d0 2024-03-28T16:49:20.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== watch-notify(notify (1) cookie 94653454641904 notify 124554051593 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x7f4b0c058050 con 0x5616398988d0 2024-03-28T16:49:20.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f49feffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051593 cookie 94653454641904 notifier 4522 bl.length()=160 2024-03-28T16:49:20.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(99 notify.6 [notify cookie 94653454333744 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f49feffd640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f49e0003d60 con 0x5616398988d0 2024-03-28T16:49:20.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== watch-notify(notify_complete (2) cookie 94653454333744 notify 124554051593 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561639875e50 con 0x5616398988d0 2024-03-28T16:49:20.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(100 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985b730 con 0x5616398988d0 2024-03-28T16:49:20.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(101 fud.buckets [delete] v29'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1d56e980 10 removing user index: fud 2024-03-28T16:49:20.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1d56e980 10 removing default.rgw.meta+users.uid+fud from cache 2024-03-28T16:49:20.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1d56e980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-03-28T16:49:20.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [notify cookie 94653454333744 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398d1bf0 con 0x5616398988d0 2024-03-28T16:49:20.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== watch-notify(notify (1) cookie 94653454635616 notify 124554051594 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x561639875e50 con 0x5616398988d0 2024-03-28T16:49:20.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f49fe7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051594 cookie 94653454635616 notifier 4522 bl.length()=152 2024-03-28T16:49:20.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(102 notify.5 [notify cookie 94653454333744 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.311+0000 7f49fe7fc640 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f49500032c0 con 0x5616398988d0 2024-03-28T16:49:20.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.315+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(103 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.315+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== watch-notify(notify_complete (2) cookie 94653454333744 notify 124554051594 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561639858660 con 0x5616398988d0 2024-03-28T16:49:20.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.315+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:104 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985b730 con 0x5616398988d0 2024-03-28T16:49:20.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.315+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(104 fud [call,call,delete] v29'2 uv2 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.315+0000 7f49d1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.315+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94653454642960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56163985b730 con 0x561639894390 2024-03-28T16:49:20.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94653454642960] v29'125 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94653454619216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a6f10 con 0x7f4b040b6050 2024-03-28T16:49:20.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94653454619216] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1d56e980 20 remove_watcher() i=0 2024-03-28T16:49:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1d56e980 2 removed watcher, disabling cache 2024-03-28T16:49:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94653454623712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a6f10 con 0x561639894390 2024-03-28T16:49:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94653454623712] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1d56e980 20 remove_watcher() i=1 2024-03-28T16:49:20.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.319+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94653454627488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a23e0 con 0x56163989d4b0 2024-03-28T16:49:20.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.323+0000 7f4b1a888640 1 -- 172.21.15.170:0/4235887733 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94653454627488] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b040b1040 con 0x56163989d4b0 2024-03-28T16:49:20.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.323+0000 7f4b1d56e980 20 remove_watcher() i=2 2024-03-28T16:49:20.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.323+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94653454632160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a6f10 con 0x7f4b040b6050 2024-03-28T16:49:20.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.323+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94653454632160] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x7f4b040b6050 2024-03-28T16:49:20.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.323+0000 7f4b1d56e980 20 remove_watcher() i=3 2024-03-28T16:49:20.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.323+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94653454624768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a32a0 con 0x561639894390 2024-03-28T16:49:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.327+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94653454624768] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b140083f0 con 0x561639894390 2024-03-28T16:49:20.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.327+0000 7f4b1d56e980 20 remove_watcher() i=4 2024-03-28T16:49:20.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.327+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94653454635616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a44e0 con 0x5616398988d0 2024-03-28T16:49:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.327+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94653454635616] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.327+0000 7f4b1d56e980 20 remove_watcher() i=5 2024-03-28T16:49:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.327+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94653454641904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a6f10 con 0x5616398988d0 2024-03-28T16:49:20.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94653454641904] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 20 remove_watcher() i=6 2024-03-28T16:49:20.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94653454645376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5616398a32a0 con 0x5616398988d0 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1b089640 1 -- 172.21.15.170:0/4235887733 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94653454645376] v29'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f4b0c035040 con 0x5616398988d0 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 20 remove_watcher() i=7 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616398988d0 msgr2=0x56163989acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5616398988d0 0x56163989acc0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561639894390 msgr2=0x561639896850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561639894390 0x561639896850 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4b040b6050 msgr2=0x7f4b040be420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f4b040b6050 0x7f4b040be420 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56163989d4b0 msgr2=0x56163989f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56163989d4b0 0x56163989f970 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49e40342b0 msgr2=0x7f49e4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49e40342b0 0x7f49e4036770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b040ad040 tx=0x7f4b040a7000 comp rx=0 tx=0).stop 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 msgr2=0x5616398890e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1b88a640 1 -- 172.21.15.170:0/4235887733 reap_dead start 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398890e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c035010 tx=0x7f4b0c034f60 comp rx=0 tx=0).stop 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 shutdown_connections 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/4235887733 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398823c0 0x5616398890e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 >> 172.21.15.170:0/4235887733 conn(0x56163987e750 msgr2=0x56163987c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 shutdown_connections 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/4235887733 wait complete. 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49f4034300 msgr2=0x7f49f40367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49f4034300 0x7f49f40367c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x56163985fd70 tx=0x7f4b04005d90 comp rx=0 tx=0).stop 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398754f0 msgr2=0x561639875910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398754f0 0x561639875910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c028000 tx=0x7f4b0c02dc20 comp rx=0 tx=0).stop 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 shutdown_connections 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f49f4034300 0x7f49f40367c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/3804259717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616398754f0 0x561639875910 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 >> 172.21.15.170:0/3804259717 conn(0x56163986a300 msgr2=0x56163986ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 shutdown_connections 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/3804259717 wait complete. 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561639860ba0 msgr2=0x561639862f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561639860ba0 0x561639862f90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56163976b350 msgr2=0x56163985e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56163976b350 0x56163985e970 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561639864b40 msgr2=0x561639866f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561639864b40 0x561639866f30 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a000342b0 msgr2=0x7f4a00036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a000342b0 0x7f4a00036770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b040ac000 tx=0x7f4b040a5000 comp rx=0 tx=0).stop 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 msgr2=0x561639857cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 --2- 172.21.15.170:0/192498752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5616396db8b0 0x561639857cf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c001ee0 tx=0x7f4b0c000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1b88a640 1 -- 172.21.15.170:0/192498752 reap_dead start 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 shutdown_connections 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 >> 172.21.15.170:0/192498752 conn(0x561639761710 msgr2=0x56163984b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 shutdown_connections 2024-03-28T16:49:20.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.331+0000 7f4b1d56e980 1 -- 172.21.15.170:0/192498752 wait complete. 2024-03-28T16:49:20.342 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-03-28T16:49:20.342 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-03-28T16:49:20.342 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-03-28T16:49:20.429 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:20.429 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:20.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe594fe980 1 Processor -- start 2024-03-28T16:49:20.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe594fe980 1 -- start start 2024-03-28T16:49:20.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe594fe980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556488c7a3f0 0x556488c7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe594fe980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556488bc24c0 con 0x556488c75ee0 2024-03-28T16:49:20.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe594fe980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x556488b890f0 con 0x556488c7a3f0 2024-03-28T16:49:20.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe56818640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556488c7a3f0 0x556488c7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe57019640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x556488c75ee0 0x556488c762c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38256/0 (socket says 172.21.15.170:38256) 2024-03-28T16:49:20.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe56818640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556488c7a3f0 0x556488c7c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56798/0 (socket says 172.21.15.170:56798) 2024-03-28T16:49:20.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe57019640 1 -- 172.21.15.170:0/3261980878 learned_addr learned my addr 172.21.15.170:0/3261980878 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:20.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe56818640 1 -- 172.21.15.170:0/3261980878 >> v1:172.21.15.119:6789/0 conn(0x556488c75ee0 legacy=0x556488c762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.431+0000 7fbe56818640 1 -- 172.21.15.170:0/3261980878 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488c7dd30 con 0x556488c7a3f0 2024-03-28T16:49:20.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe56818640 1 --2- 172.21.15.170:0/3261980878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c7a3f0 0x556488c7c7e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000a420 tx=0x7fbe4001afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222dee18e21e02f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe56017640 1 -- 172.21.15.170:0/3261980878 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe400aa530 con 0x556488c7a3f0 2024-03-28T16:49:20.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe56017640 1 -- 172.21.15.170:0/3261980878 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe400a6050 con 0x556488c7a3f0 2024-03-28T16:49:20.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe56017640 1 -- 172.21.15.170:0/3261980878 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe400aa800 con 0x556488c7a3f0 2024-03-28T16:49:20.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe594fe980 1 -- 172.21.15.170:0/3261980878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c7a3f0 msgr2=0x556488c7c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe594fe980 1 --2- 172.21.15.170:0/3261980878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c7a3f0 0x556488c7c7e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000a420 tx=0x7fbe4001afd0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe594fe980 1 -- 172.21.15.170:0/3261980878 shutdown_connections 2024-03-28T16:49:20.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe594fe980 1 --2- 172.21.15.170:0/3261980878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c7a3f0 0x556488c7c7e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe594fe980 1 -- 172.21.15.170:0/3261980878 >> 172.21.15.170:0/3261980878 conn(0x556488c717b0 msgr2=0x556488c73bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe594fe980 1 -- 172.21.15.170:0/3261980878 shutdown_connections 2024-03-28T16:49:20.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.435+0000 7fbe594fe980 1 -- 172.21.15.170:0/3261980878 wait complete. 2024-03-28T16:49:20.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe594fe980 1 Processor -- start 2024-03-28T16:49:20.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe594fe980 1 -- start start 2024-03-28T16:49:20.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe594fe980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d5fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe594fe980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556488b86620 con 0x556488c75ee0 2024-03-28T16:49:20.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d5fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d5fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56806/0 (socket says 172.21.15.170:56806) 2024-03-28T16:49:20.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe57019640 1 -- 172.21.15.170:0/580646930 learned_addr learned my addr 172.21.15.170:0/580646930 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe57019640 1 -- 172.21.15.170:0/580646930 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4000a100 con 0x556488c75ee0 2024-03-28T16:49:20.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.447+0000 7fbe57019640 1 --2- 172.21.15.170:0/580646930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d5fe60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe50001c00 tx=0x7fbe5000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b530e5e665ce07f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbd4560a640 1 -- 172.21.15.170:0/580646930 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5000e690 con 0x556488c75ee0 2024-03-28T16:49:20.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbd4560a640 1 -- 172.21.15.170:0/580646930 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5000e810 con 0x556488c75ee0 2024-03-28T16:49:20.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbd4560a640 1 -- 172.21.15.170:0/580646930 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5000eb30 con 0x556488c75ee0 2024-03-28T16:49:20.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- 172.21.15.170:0/580646930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 msgr2=0x556488d5fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 --2- 172.21.15.170:0/580646930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d5fe60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbe50001c00 tx=0x7fbe5000b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- 172.21.15.170:0/580646930 shutdown_connections 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 --2- 172.21.15.170:0/580646930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d5fe60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- 172.21.15.170:0/580646930 >> 172.21.15.170:0/580646930 conn(0x556488c717b0 msgr2=0x556488d5b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- 172.21.15.170:0/580646930 shutdown_connections 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- 172.21.15.170:0/580646930 wait complete. 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 Processor -- start 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- start start 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d66cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556488b81e00 con 0x556488c75ee0 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d66cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d66cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56816/0 (socket says 172.21.15.170:56816) 2024-03-28T16:49:20.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe57019640 1 -- 172.21.15.170:0/242657630 learned_addr learned my addr 172.21.15.170:0/242657630 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe57019640 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488bc24c0 con 0x556488c75ee0 2024-03-28T16:49:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe57019640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d66cb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000b710 tx=0x7fbe50000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbd43e07640 1 -- 172.21.15.170:0/242657630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5001f070 con 0x556488c75ee0 2024-03-28T16:49:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbd43e07640 1 -- 172.21.15.170:0/242657630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5001a910 con 0x556488c75ee0 2024-03-28T16:49:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556488d671f0 con 0x556488c75ee0 2024-03-28T16:49:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbd43e07640 1 -- 172.21.15.170:0/242657630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5001abf0 con 0x556488c75ee0 2024-03-28T16:49:20.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.451+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556488d67620 con 0x556488c75ee0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbd43e07640 1 -- 172.21.15.170:0/242657630 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbe5000a040 con 0x556488c75ee0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbd43e07640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd3c0342b0 0x7fbd3c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbd43e07640 1 -- 172.21.15.170:0/242657630 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbe50049d90 con 0x556488c75ee0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe56818640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd3c0342b0 0x7fbd3c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488c7a3f0 0x556488d6d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488d6de70 con 0x556488c7a3f0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe5781a640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488c7a3f0 0x556488d6d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe56818640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd3c0342b0 0x7fbd3c036770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe400ac000 tx=0x7fbe400a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe5781a640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488c7a3f0 0x556488d6d930 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe5781a640 1 -- 172.21.15.170:0/242657630 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488c7a3f0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488d6e1d0 con 0x556488c7a3f0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe5781a640 1 -- 172.21.15.170:0/242657630 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbe480083f0 con 0x556488c7a3f0 2024-03-28T16:49:20.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488d6fef0 0x556488d722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488d72820 con 0x556488d6fef0 2024-03-28T16:49:20.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe57019640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488d6fef0 0x556488d722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe57019640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488d6fef0 0x556488d722e0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe57019640 1 -- 172.21.15.170:0/242657630 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fbe500173e0 con 0x556488d6fef0 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488d6ed60 con 0x556488c7a3f0 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe5781a640 1 -- 172.21.15.170:0/242657630 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbe480083f0 con 0x556488c7a3f0 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556488d73c40 0x556488d76030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488d766d0 con 0x556488d73c40 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.455+0000 7fbe56818640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556488d73c40 0x556488d76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe56818640 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556488d73c40 0x556488d76030 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe56818640 1 -- 172.21.15.170:0/242657630 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fbe400b1040 con 0x556488d73c40 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe594fe980 1 Processor -- start 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe594fe980 1 -- start start 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe594fe980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488d845e0 0x556488d84a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe594fe980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556488b1ae30 con 0x556488d845e0 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488d845e0 0x556488d84a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488d845e0 0x556488d84a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56828/0 (socket says 172.21.15.170:56828) 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe57019640 1 -- 172.21.15.170:0/1930126653 learned_addr learned my addr 172.21.15.170:0/1930126653 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe57019640 1 -- 172.21.15.170:0/1930126653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488d84f40 con 0x556488d845e0 2024-03-28T16:49:20.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe57019640 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488d845e0 0x556488d84a00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe500224e0 tx=0x7fbe50023b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbd41602640 1 -- 172.21.15.170:0/1930126653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe500226d0 con 0x556488d845e0 2024-03-28T16:49:20.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556488d85190 con 0x556488d845e0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556488d85ce0 con 0x556488d845e0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbd41602640 1 -- 172.21.15.170:0/1930126653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50022830 con 0x556488d845e0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbd41602640 1 -- 172.21.15.170:0/1930126653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5004b030 con 0x556488d845e0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbd41602640 1 -- 172.21.15.170:0/1930126653 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbe5004e060 con 0x556488d845e0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbd41602640 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd300342b0 0x7fbd30036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.459+0000 7fbd41602640 1 -- 172.21.15.170:0/1930126653 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbe5004c070 con 0x556488d845e0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe56818640 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd300342b0 0x7fbd30036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 Processor -- start 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- start start 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488bec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556488b0e790 con 0x556488bebcc0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe56818640 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd300342b0 0x7fbd30036770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe40017be0 tx=0x7fbe40005d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488bec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488bec0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56836/0 (socket says 172.21.15.170:56836) 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 -- 172.21.15.170:0/2525665937 learned_addr learned my addr 172.21.15.170:0/2525665937 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 -- 172.21.15.170:0/2525665937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488bea510 con 0x556488bebcc0 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 --2- 172.21.15.170:0/2525665937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488bec0c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbe50045010 tx=0x7fbe5001c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a3ca4df9e236c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbd3affd640 1 -- 172.21.15.170:0/2525665937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5000fc00 con 0x556488bebcc0 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbd3affd640 1 -- 172.21.15.170:0/2525665937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50056040 con 0x556488bebcc0 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbd3affd640 1 -- 172.21.15.170:0/2525665937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5000dac0 con 0x556488bebcc0 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- 172.21.15.170:0/2525665937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 msgr2=0x556488bec0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 --2- 172.21.15.170:0/2525665937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488bec0c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbe50045010 tx=0x7fbe5001c280 comp rx=0 tx=0).stop 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- 172.21.15.170:0/2525665937 shutdown_connections 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 --2- 172.21.15.170:0/2525665937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488bec0c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- 172.21.15.170:0/2525665937 >> 172.21.15.170:0/2525665937 conn(0x556488d8d8f0 msgr2=0x556488d8b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- 172.21.15.170:0/2525665937 shutdown_connections 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- 172.21.15.170:0/2525665937 wait complete. 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 Processor -- start 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- start start 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488d9a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe594fe980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556488d9ab20 con 0x556488bebcc0 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488d9a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488d9a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56840/0 (socket says 172.21.15.170:56840) 2024-03-28T16:49:20.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 learned_addr learned my addr 172.21.15.170:0/1658387696 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556488d85ce0 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.463+0000 7fbe57019640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488d9a5e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbe500087c0 tx=0x7fbe5001c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbd397fa640 1 -- 172.21.15.170:0/1658387696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe50008600 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556488d85ce0 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbd397fa640 1 -- 172.21.15.170:0/1658387696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5003a8d0 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbd397fa640 1 -- 172.21.15.170:0/1658387696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5003abb0 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556488d85ce0 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbd397fa640 1 -- 172.21.15.170:0/1658387696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbe5006e020 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbd397fa640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd200342b0 0x7fbd20036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbd397fa640 1 -- 172.21.15.170:0/1658387696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fbe50072af0 con 0x556488bebcc0 2024-03-28T16:49:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe56818640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd200342b0 0x7fbd20036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488da58c0 0x556488da7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da84d0 con 0x556488da58c0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe5781a640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488da58c0 0x556488da7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe56818640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd200342b0 0x7fbd20036770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe400aafd0 tx=0x7fbe400a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe5781a640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488da58c0 0x556488da7d80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 realm 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da88a0 con 0x556488da58c0 2024-03-28T16:49:20.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da9450 con 0x556488da58c0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488da9e00 0x556488dac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dac840 con 0x556488da9e00 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.467+0000 7fbe57019640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488da9e00 0x556488dac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe57019640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488da9e00 0x556488dac1f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe594fe980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe594fe980 20 searching for the correct realm 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556488daea00 0x556488db0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db1540 con 0x556488daea00 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe56818640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556488daea00 0x556488db0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe56818640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556488daea00 0x556488db0ec0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe56818640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbe400b6050 0x7fbe400be420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db1540 con 0x7fbe400b6050 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbe400b6050 0x7fbe400be420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbe400b6050 0x7fbe400be420 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.471+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x7fbe400b6050 2024-03-28T16:49:20.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x7fbe400b6050 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.475+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x7fbe400b6050 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x7fbe400b6050 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488daea00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.479+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da9e00 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x556488da58c0 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488da90f0 con 0x7fbe400b6050 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:20.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dace20 con 0x556488da58c0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dace20 con 0x556488da58c0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dace20 con 0x7fbe400b6050 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 zone default found 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 4 Realm: () 2024-03-28T16:49:20.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 zonegroup default 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dad670 con 0x556488da9e00 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dadad0 con 0x556488daea00 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db34b0 con 0x556488daea00 2024-03-28T16:49:20.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 started sync module instance, tier type = 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db2c40 con 0x7fbe400b6050 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93890281154464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db2ff0 con 0x7fbe400b6050 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.483+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db4210 con 0x556488da58c0 2024-03-28T16:49:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.487+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93890281154464] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.487+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.487+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93890281158864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db2ff0 con 0x556488da58c0 2024-03-28T16:49:20.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.487+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db4840 con 0x556488daea00 2024-03-28T16:49:20.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.487+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.487+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93890281161568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db5090 con 0x556488daea00 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db5a80 con 0x7fbe400b6050 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93890281158864] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93890281166176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db5e30 con 0x7fbe400b6050 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db67a0 con 0x556488da58c0 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93890281161568] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93890281166176] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93890281159920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db6300 con 0x556488da58c0 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.491+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db6b00 con 0x556488da9e00 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93890281159920] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93890281169664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db6720 con 0x556488da9e00 2024-03-28T16:49:20.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db77f0 con 0x556488da9e00 2024-03-28T16:49:20.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93890281169664] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93890281173136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db7ff0 con 0x556488da9e00 2024-03-28T16:49:20.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.495+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db8850 con 0x556488da9e00 2024-03-28T16:49:20.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93890281173136] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93890281175888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db8a30 con 0x556488da9e00 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=0 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=1 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=2 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=3 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=4 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=5 2024-03-28T16:49:20.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=6 2024-03-28T16:49:20.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93890281175888] v29'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 20 add_watcher() i=7 2024-03-28T16:49:20.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:20.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dbbdf0 con 0x556488da58c0 2024-03-28T16:49:20.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.499+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93890281179424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db9250 con 0x556488da58c0 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93890281179424] v29'126 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbd0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 5 note: GC not initialized 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbd0cfe1640 20 reqs_thread_entry: start 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 Filter name: none 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbc867fc640 20 reqs_thread_entry: start 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488ddded0 con 0x556488da9e00 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 246+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488ddc5d0 con 0x556488da9e00 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-03-28T16:49:20.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:20.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:49:20.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:49:20.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.503+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~385 in=385b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488de0870 con 0x556488da9e00 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~385] v29'5 uv5 ondisk = 0) v8 ==== 288+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 93890281338112 in=588b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488de1170 con 0x7fbe400b6050 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== watch-notify(notify (1) cookie 93890281166176 notify 124554051596 ret 0) v3 ==== 618+0+0 (crc 0 0 0) 0x7fbe4800ac20 con 0x7fbe400b6050 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbd3affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051596 cookie 93890281166176 notifier 4529 bl.length()=576 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbd3affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-03-28T16:49:20.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbd3affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-03-28T16:49:20.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 notify.3 [notify cookie 93890281338112 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbd3affd640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fbd1c003ea0 con 0x7fbe400b6050 2024-03-28T16:49:20.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== watch-notify(notify_complete (2) cookie 93890281338112 notify 124554051596 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fbe48042440 con 0x7fbe400b6050 2024-03-28T16:49:20.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488de1030 con 0x556488da9e00 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v29'6 uv6 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.507+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 93890281343888 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488de0fb0 con 0x556488daea00 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== watch-notify(notify (1) cookie 93890281161568 notify 124554051594 ret 0) v3 ==== 234+0+0 (crc 0 0 0) 0x556488c7dd30 con 0x556488daea00 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbd3a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051594 cookie 93890281161568 notifier 4529 bl.length()=192 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbd3a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbd3a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:49:20.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbd3a7fc640 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fbc8c003400 con 0x556488daea00 2024-03-28T16:49:20.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(68 notify.2 [notify cookie 93890281343888 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== watch-notify(notify_complete (2) cookie 93890281343888 notify 124554051594 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fbe400cd650 con 0x556488daea00 2024-03-28T16:49:20.511 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"testx$tenanteduser","display_name":"tenanted-user","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"testx$tenanteduser","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:20.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbd0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93890281179424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488dbe5a0 con 0x556488da58c0 2024-03-28T16:49:20.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93890281179424] v29'127 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93890281154464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db8920 con 0x7fbe400b6050 2024-03-28T16:49:20.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93890281154464] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe594fe980 20 remove_watcher() i=0 2024-03-28T16:49:20.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe594fe980 2 removed watcher, disabling cache 2024-03-28T16:49:20.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.511+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93890281158864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db8920 con 0x556488da58c0 2024-03-28T16:49:20.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.515+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93890281158864] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.515+0000 7fbe594fe980 20 remove_watcher() i=1 2024-03-28T16:49:20.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.515+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93890281161568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db38d0 con 0x556488daea00 2024-03-28T16:49:20.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.515+0000 7fbe56818640 1 -- 172.21.15.170:0/1658387696 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93890281161568] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe400b1040 con 0x556488daea00 2024-03-28T16:49:20.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.515+0000 7fbe594fe980 20 remove_watcher() i=2 2024-03-28T16:49:20.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.515+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93890281166176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db8920 con 0x7fbe400b6050 2024-03-28T16:49:20.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.519+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93890281166176] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x7fbe400b6050 2024-03-28T16:49:20.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.519+0000 7fbe594fe980 20 remove_watcher() i=3 2024-03-28T16:49:20.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.519+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93890281159920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db38d0 con 0x556488da58c0 2024-03-28T16:49:20.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.519+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93890281159920] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe480083f0 con 0x556488da58c0 2024-03-28T16:49:20.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.519+0000 7fbe594fe980 20 remove_watcher() i=4 2024-03-28T16:49:20.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.519+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93890281169664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db3d70 con 0x556488da9e00 2024-03-28T16:49:20.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.523+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93890281169664] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.523+0000 7fbe594fe980 20 remove_watcher() i=5 2024-03-28T16:49:20.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.523+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93890281173136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db8920 con 0x556488da9e00 2024-03-28T16:49:20.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.523+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93890281173136] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.523+0000 7fbe594fe980 20 remove_watcher() i=6 2024-03-28T16:49:20.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.523+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93890281175888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x556488db6300 con 0x556488da9e00 2024-03-28T16:49:20.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe57019640 1 -- 172.21.15.170:0/1658387696 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93890281175888] v29'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fbe500173e0 con 0x556488da9e00 2024-03-28T16:49:20.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 20 remove_watcher() i=7 2024-03-28T16:49:20.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488da9e00 msgr2=0x556488dac1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488da9e00 0x556488dac1f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488da58c0 msgr2=0x556488da7d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488da58c0 0x556488da7d80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbe400b6050 msgr2=0x7fbe400be420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fbe400b6050 0x7fbe400be420 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556488daea00 msgr2=0x556488db0ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556488daea00 0x556488db0ec0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd200342b0 msgr2=0x7fbd20036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd200342b0 0x7fbd20036770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe400aafd0 tx=0x7fbe400a7000 comp rx=0 tx=0).stop 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 msgr2=0x556488d9a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe5781a640 1 -- 172.21.15.170:0/1658387696 reap_dead start 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488d9a5e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbe500087c0 tx=0x7fbe5001c100 comp rx=0 tx=0).stop 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 shutdown_connections 2024-03-28T16:49:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1658387696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488bebcc0 0x556488d9a5e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 >> 172.21.15.170:0/1658387696 conn(0x556488d8d8f0 msgr2=0x556488d8af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 shutdown_connections 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1658387696 wait complete. 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd300342b0 msgr2=0x7fbd30036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd300342b0 0x7fbd30036770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbe40017be0 tx=0x7fbe40005d90 comp rx=0 tx=0).stop 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488d845e0 msgr2=0x556488d84a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488d845e0 0x556488d84a00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe500224e0 tx=0x7fbe50023b10 comp rx=0 tx=0).stop 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 shutdown_connections 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd300342b0 0x7fbd30036770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/1930126653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488d845e0 0x556488d84a00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 >> 172.21.15.170:0/1930126653 conn(0x556488d793f0 msgr2=0x556488d79f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 shutdown_connections 2024-03-28T16:49:20.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/1930126653 wait complete. 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488d6fef0 msgr2=0x556488d722e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556488d6fef0 0x556488d722e0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488c7a3f0 msgr2=0x556488d6d930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556488c7a3f0 0x556488d6d930 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556488d73c40 msgr2=0x556488d76030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556488d73c40 0x556488d76030 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd3c0342b0 msgr2=0x7fbd3c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbd3c0342b0 0x7fbd3c036770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe400ac000 tx=0x7fbe400a5000 comp rx=0 tx=0).stop 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 msgr2=0x556488d66cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 --2- 172.21.15.170:0/242657630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556488c75ee0 0x556488d66cb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000b710 tx=0x7fbe50000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe5781a640 1 -- 172.21.15.170:0/242657630 reap_dead start 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 shutdown_connections 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 >> 172.21.15.170:0/242657630 conn(0x556488c717b0 msgr2=0x556488d5a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 shutdown_connections 2024-03-28T16:49:20.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.527+0000 7fbe594fe980 1 -- 172.21.15.170:0/242657630 wait complete. 2024-03-28T16:49:20.536 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:20.536 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-03-28T16:49:20.536 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-03-28T16:49:20.537 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-03-28T16:49:20.620 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:20.620 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.619+0000 7efe74d6f980 1 Processor -- start 2024-03-28T16:49:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.619+0000 7efe74d6f980 1 -- start start 2024-03-28T16:49:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.619+0000 7efe74d6f980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56414bef86b0 0x56414befaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.619+0000 7efe74d6f980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56414be3e4c0 con 0x56414be66890 2024-03-28T16:49:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.619+0000 7efe74d6f980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56414be050f0 con 0x56414bef86b0 2024-03-28T16:49:20.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe72089640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56414bef86b0 0x56414befaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe7288a640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x56414be66890 0x56414bef6f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38272/0 (socket says 172.21.15.170:38272) 2024-03-28T16:49:20.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe7288a640 1 -- 172.21.15.170:0/1977928813 learned_addr learned my addr 172.21.15.170:0/1977928813 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:20.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe72089640 1 -- 172.21.15.170:0/1977928813 >> v1:172.21.15.119:6789/0 conn(0x56414be66890 legacy=0x56414bef6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe72089640 1 -- 172.21.15.170:0/1977928813 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414befbff0 con 0x56414bef86b0 2024-03-28T16:49:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe72089640 1 --2- 172.21.15.170:0/1977928813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414bef86b0 0x56414befaaa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00a480 tx=0x7efe5c01b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc843038a2f81f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe71888640 1 -- 172.21.15.170:0/1977928813 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe5c0ab540 con 0x56414bef86b0 2024-03-28T16:49:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe71888640 1 -- 172.21.15.170:0/1977928813 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c0ab6a0 con 0x56414bef86b0 2024-03-28T16:49:20.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe71888640 1 -- 172.21.15.170:0/1977928813 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe5c0ab960 con 0x56414bef86b0 2024-03-28T16:49:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe74d6f980 1 -- 172.21.15.170:0/1977928813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414bef86b0 msgr2=0x56414befaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1977928813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414bef86b0 0x56414befaaa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00a480 tx=0x7efe5c01b280 comp rx=0 tx=0).stop 2024-03-28T16:49:20.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe74d6f980 1 -- 172.21.15.170:0/1977928813 shutdown_connections 2024-03-28T16:49:20.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1977928813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414bef86b0 0x56414befaaa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe74d6f980 1 -- 172.21.15.170:0/1977928813 >> 172.21.15.170:0/1977928813 conn(0x56414beec6b0 msgr2=0x56414beeead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe74d6f980 1 -- 172.21.15.170:0/1977928813 shutdown_connections 2024-03-28T16:49:20.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.623+0000 7efe74d6f980 1 -- 172.21.15.170:0/1977928813 wait complete. 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 Processor -- start 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- start start 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414be02620 con 0x56414be66890 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfde020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56856/0 (socket says 172.21.15.170:56856) 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe7288a640 1 -- 172.21.15.170:0/1485252939 learned_addr learned my addr 172.21.15.170:0/1485252939 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe7288a640 1 -- 172.21.15.170:0/1485252939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c005d30 con 0x56414be66890 2024-03-28T16:49:20.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe7288a640 1 --2- 172.21.15.170:0/1485252939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfde020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efe64006fb0 tx=0x7efe64001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d83b2478c81e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efd60e09640 1 -- 172.21.15.170:0/1485252939 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe64004530 con 0x56414be66890 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efd60e09640 1 -- 172.21.15.170:0/1485252939 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64004690 con 0x56414be66890 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efd60e09640 1 -- 172.21.15.170:0/1485252939 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe640049b0 con 0x56414be66890 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- 172.21.15.170:0/1485252939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 msgr2=0x56414bfde020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1485252939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfde020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efe64006fb0 tx=0x7efe64001940 comp rx=0 tx=0).stop 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- 172.21.15.170:0/1485252939 shutdown_connections 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1485252939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfde020 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- 172.21.15.170:0/1485252939 >> 172.21.15.170:0/1485252939 conn(0x56414beec6b0 msgr2=0x56414bfd9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- 172.21.15.170:0/1485252939 shutdown_connections 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- 172.21.15.170:0/1485252939 wait complete. 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 Processor -- start 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.639+0000 7efe74d6f980 1 -- start start 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe74d6f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfe4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe74d6f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414bdfde00 con 0x56414be66890 2024-03-28T16:49:20.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfe4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfe4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56864/0 (socket says 172.21.15.170:56864) 2024-03-28T16:49:20.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7288a640 1 -- 172.21.15.170:0/2630619753 learned_addr learned my addr 172.21.15.170:0/2630619753 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7288a640 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414be3e4c0 con 0x56414be66890 2024-03-28T16:49:20.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7288a640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfe4f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x56414bfdeeb0 tx=0x7efe64000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efd5f606640 1 -- 172.21.15.170:0/2630619753 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe64013760 con 0x56414be66890 2024-03-28T16:49:20.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efd5f606640 1 -- 172.21.15.170:0/2630619753 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe640138c0 con 0x56414be66890 2024-03-28T16:49:20.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56414bfe54d0 con 0x56414be66890 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efd5f606640 1 -- 172.21.15.170:0/2630619753 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe64013b60 con 0x56414be66890 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56414bfe5900 con 0x56414be66890 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efd5f606640 1 -- 172.21.15.170:0/2630619753 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7efe64028460 con 0x56414be66890 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efd5f606640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd580342b0 0x7efd58036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efd5f606640 1 -- 172.21.15.170:0/2630619753 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7efe64049410 con 0x56414be66890 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe72089640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd580342b0 0x7efd58036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414bef86b0 0x56414bfebb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414bfec0b0 con 0x56414bef86b0 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7308b640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414bef86b0 0x56414bfebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe72089640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd580342b0 0x7efd58036770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0ad000 tx=0x7efe5c0a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7308b640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414bef86b0 0x56414bfebb70 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe7308b640 1 -- 172.21.15.170:0/2630619753 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414bef86b0 2024-03-28T16:49:20.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.643+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414bfec410 con 0x56414bef86b0 2024-03-28T16:49:20.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe7308b640 1 -- 172.21.15.170:0/2630619753 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414bef86b0 2024-03-28T16:49:20.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414bfee130 0x56414bff0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414bff0a60 con 0x56414bfee130 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe7288a640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414bfee130 0x56414bff0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe7288a640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414bfee130 0x56414bff0520 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe7288a640 1 -- 172.21.15.170:0/2630619753 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7efe64024020 con 0x56414bfee130 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414bfecf10 con 0x56414bef86b0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe7308b640 1 -- 172.21.15.170:0/2630619753 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414bef86b0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56414bff1dd0 0x56414bff41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414bff47d0 con 0x56414bff1dd0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe72089640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56414bff1dd0 0x56414bff41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe72089640 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56414bff1dd0 0x56414bff41c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe72089640 1 -- 172.21.15.170:0/2630619753 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414bff1dd0 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 Processor -- start 2024-03-28T16:49:20.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 -- start start 2024-03-28T16:49:20.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c002700 0x56414c002b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.647+0000 7efe74d6f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414bd96e30 con 0x56414c002700 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c002700 0x56414c002b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c002700 0x56414c002b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56870/0 (socket says 172.21.15.170:56870) 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 -- 172.21.15.170:0/1035367376 learned_addr learned my addr 172.21.15.170:0/1035367376 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 -- 172.21.15.170:0/1035367376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414c003060 con 0x56414c002700 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c002700 0x56414c002b20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efe64047990 tx=0x7efe6401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efd5ce01640 1 -- 172.21.15.170:0/1035367376 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe640115e0 con 0x56414c002700 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efd5ce01640 1 -- 172.21.15.170:0/1035367376 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64012aa0 con 0x56414c002700 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56414c0032b0 con 0x56414c002700 2024-03-28T16:49:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efd5ce01640 1 -- 172.21.15.170:0/1035367376 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe64012d80 con 0x56414c002700 2024-03-28T16:49:20.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56414c003e00 con 0x56414c002700 2024-03-28T16:49:20.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efd5ce01640 1 -- 172.21.15.170:0/1035367376 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7efe64007a70 con 0x56414c002700 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efd5ce01640 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd4c034260 0x7efd4c036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efd5ce01640 1 -- 172.21.15.170:0/1035367376 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7efe6404ade0 con 0x56414c002700 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe72089640 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd4c034260 0x7efd4c036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe74d6f980 1 Processor -- start 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe74d6f980 1 -- start start 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe72089640 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd4c034260 0x7efd4c036720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56414bfed270 tx=0x7efe5c005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe74d6f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c00f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe74d6f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414bd8a790 con 0x56414c00f4d0 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c00f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c00f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56876/0 (socket says 172.21.15.170:56876) 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 -- 172.21.15.170:0/1000491477 learned_addr learned my addr 172.21.15.170:0/1000491477 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.651+0000 7efe7288a640 1 -- 172.21.15.170:0/1000491477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414c00fec0 con 0x56414c00f4d0 2024-03-28T16:49:20.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe7288a640 1 --2- 172.21.15.170:0/1000491477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c00f8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efe640461b0 tx=0x7efe640072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff7fa94a2eb1db4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efd567fc640 1 -- 172.21.15.170:0/1000491477 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe640327a0 con 0x56414c00f4d0 2024-03-28T16:49:20.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efd567fc640 1 -- 172.21.15.170:0/1000491477 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64010e40 con 0x56414c00f4d0 2024-03-28T16:49:20.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efd567fc640 1 -- 172.21.15.170:0/1000491477 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe64046460 con 0x56414c00f4d0 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- 172.21.15.170:0/1000491477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 msgr2=0x56414c00f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1000491477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c00f8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efe640461b0 tx=0x7efe640072a0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- 172.21.15.170:0/1000491477 shutdown_connections 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1000491477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c00f8f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- 172.21.15.170:0/1000491477 >> 172.21.15.170:0/1000491477 conn(0x56414c00b860 msgr2=0x56414c0098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- 172.21.15.170:0/1000491477 shutdown_connections 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- 172.21.15.170:0/1000491477 wait complete. 2024-03-28T16:49:20.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 Processor -- start 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- start start 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c0163e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414c016920 con 0x56414c00f4d0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c0163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe7288a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c0163e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56882/0 (socket says 172.21.15.170:56882) 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 learned_addr learned my addr 172.21.15.170:0/3312992651 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56414c003060 con 0x56414c00f4d0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe7288a640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c0163e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efe64010fd0 tx=0x7efe64008f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efd54ff9640 1 -- 172.21.15.170:0/3312992651 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe6406a070 con 0x56414c00f4d0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efd54ff9640 1 -- 172.21.15.170:0/3312992651 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64061070 con 0x56414c00f4d0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56414c003060 con 0x56414c00f4d0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efd54ff9640 1 -- 172.21.15.170:0/3312992651 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe64011760 con 0x56414c00f4d0 2024-03-28T16:49:20.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.655+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56414c003060 con 0x56414c00f4d0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efd54ff9640 1 -- 172.21.15.170:0/3312992651 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7efe64007410 con 0x56414c00f4d0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efd54ff9640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd3c034350 0x7efd3c036810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efd54ff9640 1 -- 172.21.15.170:0/3312992651 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7efe64074810 con 0x56414c00f4d0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe72089640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd3c034350 0x7efd3c036810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414c0215c0 0x56414c023a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0241d0 con 0x56414c0215c0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7308b640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414c0215c0 0x56414c023a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe72089640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd3c034350 0x7efd3c036810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0abfd0 tx=0x7efe5c0bff90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7308b640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414c0215c0 0x56414c023a80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 realm 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0245a0 con 0x56414c0215c0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c025180 con 0x56414c0215c0 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414c025b30 0x56414c027f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0284e0 con 0x56414c025b30 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7288a640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414c025b30 0x56414c027f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.659+0000 7efe7288a640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414c025b30 0x56414c027f20 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe74d6f980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe74d6f980 20 searching for the correct realm 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56414c02a6d0 0x56414c02cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c02d260 con 0x56414c02a6d0 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe72089640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56414c02a6d0 0x56414c02cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe72089640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56414c02a6d0 0x56414c02cb90 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe72089640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe5c0cc050 0x7efe5c0cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c02d260 con 0x7efe5c0cc050 2024-03-28T16:49:20.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe5c0cc050 0x7efe5c0cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe5c0cc050 0x7efe5c0cb3b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.663+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x7efe5c0cc050 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x7efe5c0cc050 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.667+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x7efe5c0cc050 2024-03-28T16:49:20.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x7efe5c0cc050 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c02a6d0 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.671+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c025b30 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x56414c0215c0 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c024e20 con 0x7efe5c0cc050 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c028a90 con 0x56414c0215c0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c028a90 con 0x56414c0215c0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c028a90 con 0x7efe5c0cc050 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 zone default found 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 4 Realm: () 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 zonegroup default 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0292e0 con 0x56414c025b30 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c029750 con 0x56414c02a6d0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c02f160 con 0x56414c02a6d0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 started sync module instance, tier type = 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c02e960 con 0x7efe5c0cc050 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.675+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94838448121024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c02ed10 con 0x7efe5c0cc050 2024-03-28T16:49:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c02fec0 con 0x56414c0215c0 2024-03-28T16:49:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94838448121024] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94838448125312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c02fe80 con 0x56414c0215c0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c030940 con 0x56414c02a6d0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94838448125312] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.679+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94838448129024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c030cf0 con 0x56414c02a6d0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c031660 con 0x7efe5c0cc050 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94838448132816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c031d70 con 0x7efe5c0cc050 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c032840 con 0x56414c0215c0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94838448129024] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94838448132816] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94838448136992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c032800 con 0x56414c0215c0 2024-03-28T16:49:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.683+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c031940 con 0x56414c025b30 2024-03-28T16:49:20.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94838448136992] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94838448126368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c033090 con 0x56414c025b30 2024-03-28T16:49:20.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c033890 con 0x56414c025b30 2024-03-28T16:49:20.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94838448143344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0344b0 con 0x56414c025b30 2024-03-28T16:49:20.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c034ed0 con 0x56414c025b30 2024-03-28T16:49:20.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.687+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94838448126368] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94838448143344] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94838448146864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c035230 con 0x56414c025b30 2024-03-28T16:49:20.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=0 2024-03-28T16:49:20.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=1 2024-03-28T16:49:20.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=2 2024-03-28T16:49:20.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=3 2024-03-28T16:49:20.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=4 2024-03-28T16:49:20.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=5 2024-03-28T16:49:20.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=6 2024-03-28T16:49:20.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94838448146864] v29'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 20 add_watcher() i=7 2024-03-28T16:49:20.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:20.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c038700 con 0x56414c0215c0 2024-03-28T16:49:20.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.691+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94838448149040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c035af0 con 0x56414c0215c0 2024-03-28T16:49:20.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94838448149040] v29'128 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efd297e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:20.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:20.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 5 note: GC not initialized 2024-03-28T16:49:20.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efca3fff640 20 reqs_thread_entry: start 2024-03-28T16:49:20.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:20.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 Filter name: none 2024-03-28T16:49:20.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efca1ffb640 20 reqs_thread_entry: start 2024-03-28T16:49:20.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-03-28T16:49:20.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c05a590 con 0x56414c025b30 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rados_obj.operate() r=0 bl.length=385 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c05c0d0 con 0x7efe5c0cc050 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv25 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c05d780 con 0x56414c02a6d0 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:20.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-03-28T16:49:20.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.695+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c061f60 con 0x7efe5c0cc050 2024-03-28T16:49:20.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.699+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 foo.buckets [call] v29'26 uv26 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.699+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:66 8.7 8:e39c74b6:root::.bucket.meta.testx%3amyfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [create,call version.inc in=38b,writefull 0~371 in=371b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0655e0 con 0x56414c025b30 2024-03-28T16:49:20.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [create,call,writefull 0~371,setxattr (203)] v29'2 uv2 ondisk = 0) v8 ==== 338+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x7 2024-03-28T16:49:20.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:20.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-28T16:49:20.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:20.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94838448344816 in=818b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c063400 con 0x56414c025b30 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== watch-notify(notify (1) cookie 94838448143344 notify 124554051595 ret 0) v3 ==== 848+0+0 (crc 0 0 0) 0x56414c00fec0 con 0x56414c025b30 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efd567fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051595 cookie 94838448143344 notifier 4536 bl.length()=806 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efd567fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x7 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efd567fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efd567fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efd567fc640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7efd38003ea0 con 0x56414c025b30 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(67 notify.6 [notify cookie 94838448344816 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(68 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== watch-notify(notify_complete (2) cookie 94838448344816 notify 124554051595 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56414bfe5900 con 0x56414c025b30 2024-03-28T16:49:20.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 20 handle_bi_update(): bucket=testx:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_sources= new_sources= 2024-03-28T16:49:20.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 20 handle_bi_update(): bucket=testx:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential sources added=[] removed=[] 2024-03-28T16:49:20.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 20 handle_bi_update(): bucket=testx:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_dests= new_dests= 2024-03-28T16:49:20.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 20 handle_bi_update(): bucket=testx:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential dests added=[] removed=[] 2024-03-28T16:49:20.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0632c0 con 0x56414c0215c0 2024-03-28T16:49:20.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v29'1 uv1 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0632c0 con 0x56414c0215c0 2024-03-28T16:49:20.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.703+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v29'1 uv1 ondisk = 0) v8 ==== 281+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-03-28T16:49:20.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-03-28T16:49:20.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-03-28T16:49:20.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 94838448336416 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c0215c0 2024-03-28T16:49:20.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94838448136992 notify 124554051598 ret 0) v3 ==== 390+0+0 (crc 0 0 0) 0x7efe6c046040 con 0x56414c0215c0 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(71 notify.4 [notify cookie 94838448336416 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efd55ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051598 cookie 94838448136992 notifier 4536 bl.length()=348 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efd55ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efd55ffb640 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efd55ffb640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7efca8003400 con 0x56414c0215c0 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== watch-notify(notify_complete (2) cookie 94838448336416 notify 124554051598 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7efe6c00aa00 con 0x56414c0215c0 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 10 removing default.rgw.meta+root+myfoo from cache 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:49:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94838448336416 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c025b30 2024-03-28T16:49:20.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== watch-notify(notify (1) cookie 94838448143344 notify 124554051596 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x56414bfe5900 con 0x56414c025b30 2024-03-28T16:49:20.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efd55ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051596 cookie 94838448143344 notifier 4536 bl.length()=149 2024-03-28T16:49:20.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efd55ffb640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7efca8003400 con 0x56414c025b30 2024-03-28T16:49:20.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(73 notify.6 [notify cookie 94838448336416 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(74 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== watch-notify(notify_complete (2) cookie 94838448336416 notify 124554051596 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7efe6402d070 con 0x56414c025b30 2024-03-28T16:49:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.707+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:75 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x7efe5c0cc050 2024-03-28T16:49:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(75 myfoo [call,delete] v29'27 uv27 ondisk = 0) v8 ==== 191+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe74d6f980 10 removing default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 from cache 2024-03-28T16:49:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe74d6f980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.9 7:93e5b521:::notify.7:head [notify cookie 94838448315440 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c025b30 2024-03-28T16:49:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== watch-notify(notify (1) cookie 94838448146864 notify 124554051597 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x7efe6402d070 con 0x56414c025b30 2024-03-28T16:49:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efd567fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051597 cookie 94838448146864 notifier 4536 bl.length()=206 2024-03-28T16:49:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efd567fc640 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7efd38003ea0 con 0x56414c025b30 2024-03-28T16:49:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(76 notify.7 [notify cookie 94838448315440 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== watch-notify(notify_complete (2) cookie 94838448315440 notify 124554051597 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56414bfe54d0 con 0x56414c025b30 2024-03-28T16:49:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:78 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c02a6d0 2024-03-28T16:49:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(77 notify.7 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(78 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call,delete] v29'4 uv4 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efd297e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.711+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94838448149040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c0215c0 2024-03-28T16:49:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94838448149040] v29'129 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94838448121024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x7efe5c0cc050 2024-03-28T16:49:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94838448121024] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe74d6f980 20 remove_watcher() i=0 2024-03-28T16:49:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe74d6f980 2 removed watcher, disabling cache 2024-03-28T16:49:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94838448125312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c0215c0 2024-03-28T16:49:20.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94838448125312] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe74d6f980 20 remove_watcher() i=1 2024-03-28T16:49:20.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.715+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94838448129024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c02a6d0 2024-03-28T16:49:20.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.719+0000 7efe72089640 1 -- 172.21.15.170:0/3312992651 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94838448129024] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe5c0abc30 con 0x56414c02a6d0 2024-03-28T16:49:20.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.719+0000 7efe74d6f980 20 remove_watcher() i=2 2024-03-28T16:49:20.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.719+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94838448132816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x7efe5c0cc050 2024-03-28T16:49:20.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.719+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94838448132816] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x7efe5c0cc050 2024-03-28T16:49:20.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.719+0000 7efe74d6f980 20 remove_watcher() i=3 2024-03-28T16:49:20.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.719+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94838448136992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c0215c0 2024-03-28T16:49:20.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.723+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94838448136992] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6c0083f0 con 0x56414c0215c0 2024-03-28T16:49:20.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.723+0000 7efe74d6f980 20 remove_watcher() i=4 2024-03-28T16:49:20.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.723+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94838448126368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c025b30 2024-03-28T16:49:20.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.723+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94838448126368] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.723+0000 7efe74d6f980 20 remove_watcher() i=5 2024-03-28T16:49:20.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.723+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94838448143344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c025b30 2024-03-28T16:49:20.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94838448143344] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 20 remove_watcher() i=6 2024-03-28T16:49:20.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94838448146864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x56414c0639a0 con 0x56414c025b30 2024-03-28T16:49:20.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe7288a640 1 -- 172.21.15.170:0/3312992651 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94838448146864] v29'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7efe6406c070 con 0x56414c025b30 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 20 remove_watcher() i=7 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414c025b30 msgr2=0x56414c027f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414c025b30 0x56414c027f20 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414c0215c0 msgr2=0x56414c023a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414c0215c0 0x56414c023a80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe5c0cc050 msgr2=0x7efe5c0cb3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7efe5c0cc050 0x7efe5c0cb3b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56414c02a6d0 msgr2=0x56414c02cb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x56414c02a6d0 0x56414c02cb90 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd3c034350 msgr2=0x7efd3c036810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd3c034350 0x7efd3c036810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0abfd0 tx=0x7efe5c0bff90 comp rx=0 tx=0).stop 2024-03-28T16:49:20.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 msgr2=0x56414c0163e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe7308b640 1 -- 172.21.15.170:0/3312992651 reap_dead start 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c0163e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efe64010fd0 tx=0x7efe64008f70 comp rx=0 tx=0).stop 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 shutdown_connections 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/3312992651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c00f4d0 0x56414c0163e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 >> 172.21.15.170:0/3312992651 conn(0x56414c00b860 msgr2=0x56414c008ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 shutdown_connections 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/3312992651 wait complete. 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd4c034260 msgr2=0x7efd4c036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd4c034260 0x7efd4c036720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56414bfed270 tx=0x7efe5c005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c002700 msgr2=0x56414c002b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c002700 0x56414c002b20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efe64047990 tx=0x7efe6401d380 comp rx=0 tx=0).stop 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 shutdown_connections 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd4c034260 0x7efd4c036720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/1035367376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414c002700 0x56414c002b20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 >> 172.21.15.170:0/1035367376 conn(0x56414bff7510 msgr2=0x56414bff8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 shutdown_connections 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/1035367376 wait complete. 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414bfee130 msgr2=0x56414bff0520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56414bfee130 0x56414bff0520 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414bef86b0 msgr2=0x56414bfebb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56414bef86b0 0x56414bfebb70 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56414bff1dd0 msgr2=0x56414bff41c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56414bff1dd0 0x56414bff41c0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd580342b0 msgr2=0x7efd58036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7efd580342b0 0x7efd58036770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0ad000 tx=0x7efe5c0a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 msgr2=0x56414bfe4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 --2- 172.21.15.170:0/2630619753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56414be66890 0x56414bfe4f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x56414bfdeeb0 tx=0x7efe64000f50 comp rx=0 tx=0).stop 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe7308b640 1 -- 172.21.15.170:0/2630619753 reap_dead start 2024-03-28T16:49:20.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 shutdown_connections 2024-03-28T16:49:20.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 >> 172.21.15.170:0/2630619753 conn(0x56414beec6b0 msgr2=0x56414bfd86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 shutdown_connections 2024-03-28T16:49:20.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.727+0000 7efe74d6f980 1 -- 172.21.15.170:0/2630619753 wait complete. 2024-03-28T16:49:20.738 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-03-28T16:49:20.738 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-03-28T16:49:20.738 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-03-28T16:49:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.807+0000 7f077fc25980 1 Processor -- start 2024-03-28T16:49:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077fc25980 1 -- start start 2024-03-28T16:49:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077fc25980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55fa278356d0 0x55fa27837ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077fc25980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55fa2777b520 con 0x55fa277a30b0 2024-03-28T16:49:20.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077fc25980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55fa277420f0 con 0x55fa278356d0 2024-03-28T16:49:20.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077cf3f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55fa278356d0 0x55fa27837ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077cf3f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55fa278356d0 0x55fa27837ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56890/0 (socket says 172.21.15.170:56890) 2024-03-28T16:49:20.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077cf3f640 1 -- 172.21.15.170:0/3122313127 learned_addr learned my addr 172.21.15.170:0/3122313127 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077d740640 1 --1- 172.21.15.170:0/3122313127 >> v1:172.21.15.119:6789/0 conn(0x55fa277a30b0 0x55fa27833fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38288/0 (socket says 172.21.15.170:38288) 2024-03-28T16:49:20.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077cf3f640 1 -- 172.21.15.170:0/3122313127 >> v1:172.21.15.119:6789/0 conn(0x55fa277a30b0 legacy=0x55fa27833fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077cf3f640 1 -- 172.21.15.170:0/3122313127 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa27839010 con 0x55fa278356d0 2024-03-28T16:49:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f077cf3f640 1 --2- 172.21.15.170:0/3122313127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa278356d0 0x55fa27837ac0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f076800a5c0 tx=0x7f076801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c7bf6e80b4952a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f0777fff640 1 -- 172.21.15.170:0/3122313127 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07680aa470 con 0x55fa278356d0 2024-03-28T16:49:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f0777fff640 1 -- 172.21.15.170:0/3122313127 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07680aa5d0 con 0x55fa278356d0 2024-03-28T16:49:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.811+0000 7f0777fff640 1 -- 172.21.15.170:0/3122313127 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07680aa890 con 0x55fa278356d0 2024-03-28T16:49:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.815+0000 7f077fc25980 1 -- 172.21.15.170:0/3122313127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa278356d0 msgr2=0x55fa27837ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.815+0000 7f077fc25980 1 --2- 172.21.15.170:0/3122313127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa278356d0 0x55fa27837ac0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f076800a5c0 tx=0x7f076801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.815+0000 7f077fc25980 1 -- 172.21.15.170:0/3122313127 shutdown_connections 2024-03-28T16:49:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.815+0000 7f077fc25980 1 --2- 172.21.15.170:0/3122313127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa278356d0 0x55fa27837ac0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.815+0000 7f077fc25980 1 -- 172.21.15.170:0/3122313127 >> 172.21.15.170:0/3122313127 conn(0x55fa278296d0 msgr2=0x55fa2782baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.815+0000 7f077fc25980 1 -- 172.21.15.170:0/3122313127 shutdown_connections 2024-03-28T16:49:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.815+0000 7f077fc25980 1 -- 172.21.15.170:0/3122313127 wait complete. 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077fc25980 1 Processor -- start 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077fc25980 1 -- start start 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077fc25980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa2791b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077fc25980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2773f620 con 0x55fa277a30b0 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa2791b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa2791b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56906/0 (socket says 172.21.15.170:56906) 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077d740640 1 -- 172.21.15.170:0/67386857 learned_addr learned my addr 172.21.15.170:0/67386857 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077d740640 1 -- 172.21.15.170:0/67386857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768005df0 con 0x55fa277a30b0 2024-03-28T16:49:20.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f077d740640 1 --2- 172.21.15.170:0/67386857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa2791b170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0770006fb0 tx=0x7f0770001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872acce0f938b23b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f066b606640 1 -- 172.21.15.170:0/67386857 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0770013420 con 0x55fa277a30b0 2024-03-28T16:49:20.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f066b606640 1 -- 172.21.15.170:0/67386857 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770013580 con 0x55fa277a30b0 2024-03-28T16:49:20.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.827+0000 7f066b606640 1 -- 172.21.15.170:0/67386857 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0770013860 con 0x55fa277a30b0 2024-03-28T16:49:20.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/67386857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 msgr2=0x55fa2791b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 --2- 172.21.15.170:0/67386857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa2791b170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0770006fb0 tx=0x7f0770001940 comp rx=0 tx=0).stop 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/67386857 shutdown_connections 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 --2- 172.21.15.170:0/67386857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa2791b170 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/67386857 >> 172.21.15.170:0/67386857 conn(0x55fa278296d0 msgr2=0x55fa27916d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/67386857 shutdown_connections 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/67386857 wait complete. 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 Processor -- start 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- start start 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa27921fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2773ae00 con 0x55fa277a30b0 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa27921fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa27921fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56918/0 (socket says 172.21.15.170:56918) 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077d740640 1 -- 172.21.15.170:0/2612888710 learned_addr learned my addr 172.21.15.170:0/2612888710 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077d740640 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa2777b520 con 0x55fa277a30b0 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077d740640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa27921fc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55fa2791c000 tx=0x7f0770000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f0669e03640 1 -- 172.21.15.170:0/2612888710 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07700037a0 con 0x55fa277a30b0 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fa27922500 con 0x55fa277a30b0 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f0669e03640 1 -- 172.21.15.170:0/2612888710 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770003900 con 0x55fa277a30b0 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f0669e03640 1 -- 172.21.15.170:0/2612888710 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0770003ba0 con 0x55fa277a30b0 2024-03-28T16:49:20.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa27922930 con 0x55fa277a30b0 2024-03-28T16:49:20.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f0669e03640 1 -- 172.21.15.170:0/2612888710 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0770028460 con 0x55fa277a30b0 2024-03-28T16:49:20.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f0669e03640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06640342b0 0x7f0664036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f0669e03640 1 -- 172.21.15.170:0/2612888710 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0770049330 con 0x55fa277a30b0 2024-03-28T16:49:20.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06640342b0 0x7f0664036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa278356d0 0x55fa27928c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27929180 con 0x55fa278356d0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.831+0000 7f077df41640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa278356d0 0x55fa27928c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06640342b0 0x7f0664036770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07680ae000 tx=0x7f07680a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077df41640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa278356d0 0x55fa27928c40 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077df41640 1 -- 172.21.15.170:0/2612888710 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa278356d0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279294e0 con 0x55fa278356d0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077df41640 1 -- 172.21.15.170:0/2612888710 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f07780083f0 con 0x55fa278356d0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa2792b200 0x55fa2792d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2792db30 con 0x55fa2792b200 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077d740640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa2792b200 0x55fa2792d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077d740640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa2792b200 0x55fa2792d5f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077d740640 1 -- 172.21.15.170:0/2612888710 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f0770024020 con 0x55fa2792b200 2024-03-28T16:49:20.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2792a010 con 0x55fa278356d0 2024-03-28T16:49:20.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077df41640 1 -- 172.21.15.170:0/2612888710 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f07780083f0 con 0x55fa278356d0 2024-03-28T16:49:20.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fa2792eec0 0x55fa279312b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27931900 con 0x55fa2792eec0 2024-03-28T16:49:20.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fa2792eec0 0x55fa279312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fa2792eec0 0x55fa279312b0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.835+0000 7f077cf3f640 1 -- 172.21.15.170:0/2612888710 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f07680b0020 con 0x55fa2792eec0 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 Processor -- start 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 -- start start 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2793f830 0x55fa2793fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa276d3e30 con 0x55fa2793f830 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2793f830 0x55fa2793fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2793f830 0x55fa2793fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56926/0 (socket says 172.21.15.170:56926) 2024-03-28T16:49:20.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077d740640 1 -- 172.21.15.170:0/201530313 learned_addr learned my addr 172.21.15.170:0/201530313 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077d740640 1 -- 172.21.15.170:0/201530313 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa27940190 con 0x55fa2793f830 2024-03-28T16:49:20.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077d740640 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2793f830 0x55fa2793fc50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0770047900 tx=0x7f077001df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f0662ffd640 1 -- 172.21.15.170:0/201530313 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f077003a070 con 0x55fa2793f830 2024-03-28T16:49:20.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f0662ffd640 1 -- 172.21.15.170:0/201530313 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07700125d0 con 0x55fa2793f830 2024-03-28T16:49:20.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f0662ffd640 1 -- 172.21.15.170:0/201530313 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0770006fe0 con 0x55fa2793f830 2024-03-28T16:49:20.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fa279403e0 con 0x55fa2793f830 2024-03-28T16:49:20.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa27940f30 con 0x55fa2793f830 2024-03-28T16:49:20.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f0662ffd640 1 -- 172.21.15.170:0/201530313 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0770007140 con 0x55fa2793f830 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f0662ffd640 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06540342b0 0x7f0654036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f0662ffd640 1 -- 172.21.15.170:0/201530313 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0770040ac0 con 0x55fa2793f830 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077cf3f640 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06540342b0 0x7f0654036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 Processor -- start 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 -- start start 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077cf3f640 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06540342b0 0x7f0654036770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55fa27926690 tx=0x7f0768005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa2794c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.839+0000 7f077fc25980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa276c7790 con 0x55fa2794c5a0 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa2794c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa2794c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56928/0 (socket says 172.21.15.170:56928) 2024-03-28T16:49:20.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 -- 172.21.15.170:0/614143807 learned_addr learned my addr 172.21.15.170:0/614143807 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 -- 172.21.15.170:0/614143807 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa2794cf90 con 0x55fa2794c5a0 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 --2- 172.21.15.170:0/614143807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa2794c9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f077000a010 tx=0x7f0770027f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f822a5e774ed3c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f0660ff9640 1 -- 172.21.15.170:0/614143807 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0770047d90 con 0x55fa2794c5a0 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f0660ff9640 1 -- 172.21.15.170:0/614143807 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770020da0 con 0x55fa2794c5a0 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f0660ff9640 1 -- 172.21.15.170:0/614143807 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07700496d0 con 0x55fa2794c5a0 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- 172.21.15.170:0/614143807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 msgr2=0x55fa2794c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 --2- 172.21.15.170:0/614143807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa2794c9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f077000a010 tx=0x7f0770027f80 comp rx=0 tx=0).stop 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- 172.21.15.170:0/614143807 shutdown_connections 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 --2- 172.21.15.170:0/614143807 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa2794c9c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- 172.21.15.170:0/614143807 >> 172.21.15.170:0/614143807 conn(0x55fa27948930 msgr2=0x55fa27946990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- 172.21.15.170:0/614143807 shutdown_connections 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- 172.21.15.170:0/614143807 wait complete. 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 Processor -- start 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- start start 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa279534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa27953a10 con 0x55fa2794c5a0 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa279534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa279534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56940/0 (socket says 172.21.15.170:56940) 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 learned_addr learned my addr 172.21.15.170:0/2972745921 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa27940190 con 0x55fa2794c5a0 2024-03-28T16:49:20.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077d740640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa279534d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0770046700 tx=0x7f0770028bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f0652ffd640 1 -- 172.21.15.170:0/2972745921 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0770065040 con 0x55fa2794c5a0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fa27940190 con 0x55fa2794c5a0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f0652ffd640 1 -- 172.21.15.170:0/2972745921 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0770027730 con 0x55fa2794c5a0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.843+0000 7f0652ffd640 1 -- 172.21.15.170:0/2972745921 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0770041da0 con 0x55fa2794c5a0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa27940190 con 0x55fa2794c5a0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f0652ffd640 1 -- 172.21.15.170:0/2972745921 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0770012a40 con 0x55fa2794c5a0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f0652ffd640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0648034260 0x7f0648036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f0652ffd640 1 -- 172.21.15.170:0/2972745921 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f0770075af0 con 0x55fa2794c5a0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0648034260 0x7f0648036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa2795e690 0x55fa27960b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961280 con 0x55fa2795e690 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077df41640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa2795e690 0x55fa27960b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0648034260 0x7f0648036720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f076801b380 tx=0x7f07680c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077df41640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa2795e690 0x55fa27960b30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 realm 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961650 con 0x55fa2795e690 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27962200 con 0x55fa2795e690 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa27962bb0 0x55fa27964fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27965520 con 0x55fa27962bb0 2024-03-28T16:49:20.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.847+0000 7f077d740640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa27962bb0 0x55fa27964fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077d740640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa27962bb0 0x55fa27964fa0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077fc25980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077fc25980 20 searching for the correct realm 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fa27967420 0x55fa27969810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27969ec0 con 0x55fa27967420 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fa27967420 0x55fa27969810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fa27967420 0x55fa27969810 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077cf3f640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f07680e0040 0x7f07680d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27969ec0 con 0x7f07680e0040 2024-03-28T16:49:20.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f07680e0040 0x7f07680d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f07680e0040 0x7f07680d3400 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.851+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x7f07680e0040 2024-03-28T16:49:20.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x7f07680e0040 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.855+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x7f07680e0040 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.859+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x7f07680e0040 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27967420 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa27962bb0 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x55fa2795e690 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27961ea0 con 0x7f07680e0040 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27965d90 con 0x55fa2795e690 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27965d90 con 0x55fa2795e690 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27965d60 con 0x7f07680e0040 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:20.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 zone default found 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 4 Realm: () 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 zonegroup default 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279665e0 con 0x55fa27962bb0 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279669e0 con 0x55fa27967420 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796be00 con 0x55fa27967420 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 started sync module instance, tier type = 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.863+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796b5c0 con 0x7f07680e0040 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.867+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.867+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94532894372128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796b970 con 0x7f07680e0040 2024-03-28T16:49:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.867+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796cb60 con 0x55fa2795e690 2024-03-28T16:49:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.867+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94532894372128] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.867+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.867+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94532894376480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796cb40 con 0x55fa2795e690 2024-03-28T16:49:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.867+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796d5d0 con 0x55fa27967420 2024-03-28T16:49:20.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94532894376480] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94532894377536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796d980 con 0x55fa27967420 2024-03-28T16:49:20.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796e2f0 con 0x7f07680e0040 2024-03-28T16:49:20.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94532894383968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796ea00 con 0x7f07680e0040 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796f4d0 con 0x55fa2795e690 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94532894377536] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.871+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94532894383968] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.875+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.875+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94532894388144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796f490 con 0x55fa2795e690 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.875+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796f880 con 0x55fa27962bb0 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.875+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94532894388144] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.875+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.875+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94532894385248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796fc30 con 0x55fa27962bb0 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.875+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279705a0 con 0x55fa27962bb0 2024-03-28T16:49:20.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94532894385248] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94532894380176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27970da0 con 0x55fa27962bb0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27971710 con 0x55fa27962bb0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94532894380176] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94532894394896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27971710 con 0x55fa27962bb0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 20 add_watcher() i=0 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 20 add_watcher() i=1 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 20 add_watcher() i=2 2024-03-28T16:49:20.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 20 add_watcher() i=3 2024-03-28T16:49:20.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 20 add_watcher() i=4 2024-03-28T16:49:20.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 20 add_watcher() i=5 2024-03-28T16:49:20.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.879+0000 7f077fc25980 20 add_watcher() i=6 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94532894394896] v29'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 20 add_watcher() i=7 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27974a60 con 0x55fa2795e690 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94532894397408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27971ca0 con 0x55fa2795e690 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94532894397408] v29'130 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f06337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 5 note: GC not initialized 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f06327e4640 20 reqs_thread_entry: start 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 20 Filter name: none 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f05a3fff640 20 reqs_thread_entry: start 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 20 rados->read ofs=0 len=0 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27996350 con 0x55fa2795e690 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 281+0+332 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.883+0000 7f077fc25980 20 rados_obj.operate() r=0 bl.length=193 2024-03-28T16:49:20.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f06337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:20.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94532894397408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa27958340 con 0x55fa2795e690 2024-03-28T16:49:20.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94532894397408] v29'131 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94532894372128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279713e0 con 0x7f07680e0040 2024-03-28T16:49:20.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94532894372128] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f077fc25980 20 remove_watcher() i=0 2024-03-28T16:49:20.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f077fc25980 2 removed watcher, disabling cache 2024-03-28T16:49:20.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.887+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94532894376480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279713e0 con 0x55fa2795e690 2024-03-28T16:49:20.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94532894376480] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077fc25980 20 remove_watcher() i=1 2024-03-28T16:49:20.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94532894377536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796c220 con 0x55fa27967420 2024-03-28T16:49:20.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077cf3f640 1 -- 172.21.15.170:0/2972745921 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94532894377536] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07680b0020 con 0x55fa27967420 2024-03-28T16:49:20.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077fc25980 20 remove_watcher() i=2 2024-03-28T16:49:20.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94532894383968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279713e0 con 0x7f07680e0040 2024-03-28T16:49:20.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94532894383968] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x7f07680e0040 2024-03-28T16:49:20.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077fc25980 20 remove_watcher() i=3 2024-03-28T16:49:20.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.891+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94532894388144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279713e0 con 0x55fa2795e690 2024-03-28T16:49:20.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.895+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94532894388144] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f07780083f0 con 0x55fa2795e690 2024-03-28T16:49:20.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.895+0000 7f077fc25980 20 remove_watcher() i=4 2024-03-28T16:49:20.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.895+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94532894385248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796efb0 con 0x55fa27962bb0 2024-03-28T16:49:20.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.895+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94532894385248] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.895+0000 7f077fc25980 20 remove_watcher() i=5 2024-03-28T16:49:20.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.895+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94532894380176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa279713e0 con 0x55fa27962bb0 2024-03-28T16:49:20.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94532894380176] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 20 remove_watcher() i=6 2024-03-28T16:49:20.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94532894394896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55fa2796d090 con 0x55fa27962bb0 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077d740640 1 -- 172.21.15.170:0/2972745921 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94532894394896] v29'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0770024020 con 0x55fa27962bb0 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 20 remove_watcher() i=7 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa27962bb0 msgr2=0x55fa27964fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa27962bb0 0x55fa27964fa0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa2795e690 msgr2=0x55fa27960b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa2795e690 0x55fa27960b30 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f07680e0040 msgr2=0x7f07680d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f07680e0040 0x7f07680d3400 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fa27967420 msgr2=0x55fa27969810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55fa27967420 0x55fa27969810 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0648034260 msgr2=0x7f0648036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0648034260 0x7f0648036720 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f076801b380 tx=0x7f07680c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 msgr2=0x55fa279534d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077df41640 1 -- 172.21.15.170:0/2972745921 reap_dead start 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa279534d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0770046700 tx=0x7f0770028bf0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 shutdown_connections 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/2972745921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2794c5a0 0x55fa279534d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 >> 172.21.15.170:0/2972745921 conn(0x55fa27948930 msgr2=0x55fa27945f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 shutdown_connections 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/2972745921 wait complete. 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06540342b0 msgr2=0x7f0654036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06540342b0 0x7f0654036770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55fa27926690 tx=0x7f0768005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2793f830 msgr2=0x55fa2793fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2793f830 0x55fa2793fc50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0770047900 tx=0x7f077001df70 comp rx=0 tx=0).stop 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 shutdown_connections 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06540342b0 0x7f0654036770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 --2- 172.21.15.170:0/201530313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa2793f830 0x55fa2793fc50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 >> 172.21.15.170:0/201530313 conn(0x55fa27934640 msgr2=0x55fa27935150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 shutdown_connections 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.899+0000 7f077fc25980 1 -- 172.21.15.170:0/201530313 wait complete. 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa2792b200 msgr2=0x55fa2792d5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55fa2792b200 0x55fa2792d5f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa278356d0 msgr2=0x55fa27928c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55fa278356d0 0x55fa27928c40 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fa2792eec0 msgr2=0x55fa279312b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55fa2792eec0 0x55fa279312b0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06640342b0 msgr2=0x7f0664036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f06640342b0 0x7f0664036770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f07680ae000 tx=0x7f07680a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 msgr2=0x55fa27921fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 --2- 172.21.15.170:0/2612888710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fa277a30b0 0x55fa27921fc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55fa2791c000 tx=0x7f0770000f50 comp rx=0 tx=0).stop 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077df41640 1 -- 172.21.15.170:0/2612888710 reap_dead start 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 shutdown_connections 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 >> 172.21.15.170:0/2612888710 conn(0x55fa278296d0 msgr2=0x55fa27915810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 shutdown_connections 2024-03-28T16:49:20.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.903+0000 7f077fc25980 1 -- 172.21.15.170:0/2612888710 wait complete. 2024-03-28T16:49:20.910 INFO:teuthology.orchestra.run.smithi170.stdout:{"key":"bucket:testx/myfoo","ver":{"tag":"8iVDLmY07GTYXjzWCvYaQu7p","ver":2},"mtime":"2024-03-28T16:49:20.707562Z","data":{"bucket":{"name":"myfoo","marker":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","bucket_id":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","tenant":"testx","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"testx$tenanteduser","creation_time":"2024-03-28T16:49:17.311675Z","linked":true,"has_bucket_info":false}} 2024-03-28T16:49:20.911 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': '8iVDLmY07GTYXjzWCvYaQu7p', 'ver': 2}, 'mtime': '2024-03-28T16:49:20.707562Z', 'data': {'bucket': {'name': 'myfoo', 'marker': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'bucket_id': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-03-28T16:49:17.311675Z', 'linked': True, 'has_bucket_info': False}} 2024-03-28T16:49:20.911 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-03-28T16:49:20.911 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-03-28T16:49:20.911 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-03-28T16:49:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:20.984 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:20.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.983+0000 7fcd08439980 1 Processor -- start 2024-03-28T16:49:20.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.983+0000 7fcd08439980 1 -- start start 2024-03-28T16:49:20.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.983+0000 7fcd08439980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5619c126ef50 0x5619c126f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:20.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.983+0000 7fcd08439980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5619c11bb4c0 con 0x5619c126f870 2024-03-28T16:49:20.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.983+0000 7fcd08439980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5619c11820f0 con 0x5619c126ef50 2024-03-28T16:49:20.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd05f54640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5619c126ef50 0x5619c126f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:20.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd05753640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5619c126f870 0x5619c1270230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38300/0 (socket says 172.21.15.170:38300) 2024-03-28T16:49:20.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd05753640 1 -- 172.21.15.170:0/139427016 learned_addr learned my addr 172.21.15.170:0/139427016 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:20.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd05f54640 1 -- 172.21.15.170:0/139427016 >> v1:172.21.15.119:6789/0 conn(0x5619c126f870 legacy=0x5619c1270230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd05f54640 1 -- 172.21.15.170:0/139427016 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c1270a60 con 0x5619c126ef50 2024-03-28T16:49:20.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd05f54640 1 --2- 172.21.15.170:0/139427016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c126f330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fccf800a480 tx=0x7fccf801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d0ba5f67b82af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:20.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd04f52640 1 -- 172.21.15.170:0/139427016 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80ab540 con 0x5619c126ef50 2024-03-28T16:49:20.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd04f52640 1 -- 172.21.15.170:0/139427016 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf80ab6a0 con 0x5619c126ef50 2024-03-28T16:49:20.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd04f52640 1 -- 172.21.15.170:0/139427016 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80ab960 con 0x5619c126ef50 2024-03-28T16:49:20.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd08439980 1 -- 172.21.15.170:0/139427016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 msgr2=0x5619c126f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:20.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd08439980 1 --2- 172.21.15.170:0/139427016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c126f330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fccf800a480 tx=0x7fccf801b280 comp rx=0 tx=0).stop 2024-03-28T16:49:20.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd08439980 1 -- 172.21.15.170:0/139427016 shutdown_connections 2024-03-28T16:49:20.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd08439980 1 --2- 172.21.15.170:0/139427016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c126f330 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:20.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd08439980 1 -- 172.21.15.170:0/139427016 >> 172.21.15.170:0/139427016 conn(0x5619c126a820 msgr2=0x5619c126cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:20.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd08439980 1 -- 172.21.15.170:0/139427016 shutdown_connections 2024-03-28T16:49:20.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:20.987+0000 7fcd08439980 1 -- 172.21.15.170:0/139427016 wait complete. 2024-03-28T16:49:21.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 Processor -- start 2024-03-28T16:49:21.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- start start 2024-03-28T16:49:21.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c13593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c117f620 con 0x5619c126ef50 2024-03-28T16:49:21.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c13593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c13593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56958/0 (socket says 172.21.15.170:56958) 2024-03-28T16:49:21.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd05f54640 1 -- 172.21.15.170:0/3798678561 learned_addr learned my addr 172.21.15.170:0/3798678561 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd05f54640 1 -- 172.21.15.170:0/3798678561 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8005d30 con 0x5619c126ef50 2024-03-28T16:49:21.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd05f54640 1 --2- 172.21.15.170:0/3798678561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c13593f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fccf80ae040 tx=0x7fccf8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485cf2297aca8d52 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcbf3e07640 1 -- 172.21.15.170:0/3798678561 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80ab810 con 0x5619c126ef50 2024-03-28T16:49:21.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcbf3e07640 1 -- 172.21.15.170:0/3798678561 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf80ab970 con 0x5619c126ef50 2024-03-28T16:49:21.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcbf3e07640 1 -- 172.21.15.170:0/3798678561 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80abc10 con 0x5619c126ef50 2024-03-28T16:49:21.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- 172.21.15.170:0/3798678561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 msgr2=0x5619c13593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 --2- 172.21.15.170:0/3798678561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c13593f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fccf80ae040 tx=0x7fccf8003030 comp rx=0 tx=0).stop 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- 172.21.15.170:0/3798678561 shutdown_connections 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 --2- 172.21.15.170:0/3798678561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c13593f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- 172.21.15.170:0/3798678561 >> 172.21.15.170:0/3798678561 conn(0x5619c126a820 msgr2=0x5619c1354f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- 172.21.15.170:0/3798678561 shutdown_connections 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- 172.21.15.170:0/3798678561 wait complete. 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 Processor -- start 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.003+0000 7fcd08439980 1 -- start start 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd08439980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c1360120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd08439980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c117ae00 con 0x5619c126ef50 2024-03-28T16:49:21.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c1360120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c1360120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56974/0 (socket says 172.21.15.170:56974) 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd05f54640 1 -- 172.21.15.170:0/1954392721 learned_addr learned my addr 172.21.15.170:0/1954392721 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd05f54640 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c1270a60 con 0x5619c126ef50 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd05f54640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c1360120 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fccf801bb20 tx=0x7fccf8018020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcbf2604640 1 -- 172.21.15.170:0/1954392721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80c2070 con 0x5619c126ef50 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619c1360660 con 0x5619c126ef50 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcbf2604640 1 -- 172.21.15.170:0/1954392721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf80b59d0 con 0x5619c126ef50 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcbf2604640 1 -- 172.21.15.170:0/1954392721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80b5c70 con 0x5619c126ef50 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c1360af0 con 0x5619c126ef50 2024-03-28T16:49:21.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcbf2604640 1 -- 172.21.15.170:0/1954392721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fccf80b2070 con 0x5619c126ef50 2024-03-28T16:49:21.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcbf2604640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbec0342b0 0x7fcbec036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcbf2604640 1 -- 172.21.15.170:0/1954392721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fccf80e36a0 con 0x5619c126ef50 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd05753640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbec0342b0 0x7fcbec036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c126f870 0x5619c1366d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c1367240 con 0x5619c126f870 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd06755640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c126f870 0x5619c1366d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd05753640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbec0342b0 0x7fcbec036770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000f70 tx=0x7fccfc0012c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd06755640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c126f870 0x5619c1366d00 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd06755640 1 -- 172.21.15.170:0/1954392721 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c126f870 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.007+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13675a0 con 0x5619c126f870 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd06755640 1 -- 172.21.15.170:0/1954392721 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c126f870 2024-03-28T16:49:21.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13692c0 0x5619c136b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c136bbf0 con 0x5619c13692c0 2024-03-28T16:49:21.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05f54640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13692c0 0x5619c136b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05f54640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13692c0 0x5619c136b6b0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05f54640 1 -- 172.21.15.170:0/1954392721 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fccf80c5370 con 0x5619c13692c0 2024-03-28T16:49:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c1368130 con 0x5619c126f870 2024-03-28T16:49:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd06755640 1 -- 172.21.15.170:0/1954392721 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c126f870 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5619c136cf90 0x5619c136f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c136fa20 con 0x5619c136cf90 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05753640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5619c136cf90 0x5619c136f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05753640 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5619c136cf90 0x5619c136f380 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05753640 1 -- 172.21.15.170:0/1954392721 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c136cf90 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 Processor -- start 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 -- start start 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c137d950 0x5619c137dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd08439980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c1113e30 con 0x5619c137d950 2024-03-28T16:49:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c137d950 0x5619c137dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c137d950 0x5619c137dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56980/0 (socket says 172.21.15.170:56980) 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.011+0000 7fcd05f54640 1 -- 172.21.15.170:0/2867264628 learned_addr learned my addr 172.21.15.170:0/2867264628 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05f54640 1 -- 172.21.15.170:0/2867264628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c137e2b0 con 0x5619c137d950 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05f54640 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c137d950 0x5619c137dd70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fccf80e3010 tx=0x7fccf80d0810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcbeb7fe640 1 -- 172.21.15.170:0/2867264628 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80e33a0 con 0x5619c137d950 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcbeb7fe640 1 -- 172.21.15.170:0/2867264628 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf80e3500 con 0x5619c137d950 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619c137e500 con 0x5619c137d950 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcbeb7fe640 1 -- 172.21.15.170:0/2867264628 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80e3b70 con 0x5619c137d950 2024-03-28T16:49:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c137f050 con 0x5619c137d950 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcbeb7fe640 1 -- 172.21.15.170:0/2867264628 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fccf80e3cd0 con 0x5619c137d950 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcbeb7fe640 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbe0034300 0x7fcbe00367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcbeb7fe640 1 -- 172.21.15.170:0/2867264628 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fccf80e4070 con 0x5619c137d950 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05753640 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbe0034300 0x7fcbe00367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd08439980 1 Processor -- start 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd08439980 1 -- start start 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05753640 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbe0034300 0x7fcbe00367c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00bf60 tx=0x7fccfc00f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd08439980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c11e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd08439980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c1107790 con 0x5619c11e4cc0 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c11e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c11e50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56988/0 (socket says 172.21.15.170:56988) 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05f54640 1 -- 172.21.15.170:0/3701524339 learned_addr learned my addr 172.21.15.170:0/3701524339 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.015+0000 7fcd05f54640 1 -- 172.21.15.170:0/3701524339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c11e3580 con 0x5619c11e4cc0 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd05f54640 1 --2- 172.21.15.170:0/3701524339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c11e50e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fccf80b5660 tx=0x7fccf80bdf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401373317992a77c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcbe97fa640 1 -- 172.21.15.170:0/3701524339 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80dd620 con 0x5619c11e4cc0 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcbe97fa640 1 -- 172.21.15.170:0/3701524339 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf80dd440 con 0x5619c11e4cc0 2024-03-28T16:49:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcbe97fa640 1 -- 172.21.15.170:0/3701524339 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80d20c0 con 0x5619c11e4cc0 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- 172.21.15.170:0/3701524339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 msgr2=0x5619c11e50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 --2- 172.21.15.170:0/3701524339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c11e50e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fccf80b5660 tx=0x7fccf80bdf90 comp rx=0 tx=0).stop 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- 172.21.15.170:0/3701524339 shutdown_connections 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 --2- 172.21.15.170:0/3701524339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c11e50e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- 172.21.15.170:0/3701524339 >> 172.21.15.170:0/3701524339 conn(0x5619c1386a70 msgr2=0x5619c1384ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- 172.21.15.170:0/3701524339 shutdown_connections 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- 172.21.15.170:0/3701524339 wait complete. 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 Processor -- start 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- start start 2024-03-28T16:49:21.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c1393780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c1393cc0 con 0x5619c11e4cc0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c1393780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd05f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c1393780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:56994/0 (socket says 172.21.15.170:56994) 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 learned_addr learned my addr 172.21.15.170:0/3660673719 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf80f1070 con 0x5619c11e4cc0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd05f54640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c1393780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5619c1385820 tx=0x7fccf80f6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcbd77fe640 1 -- 172.21.15.170:0/3660673719 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80ec610 con 0x5619c11e4cc0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcbd77fe640 1 -- 172.21.15.170:0/3660673719 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf80dd620 con 0x5619c11e4cc0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619c1393ec0 con 0x5619c11e4cc0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcbd77fe640 1 -- 172.21.15.170:0/3660673719 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccf80b4c90 con 0x5619c11e4cc0 2024-03-28T16:49:21.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.019+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c1393ec0 con 0x5619c11e4cc0 2024-03-28T16:49:21.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcbd77fe640 1 -- 172.21.15.170:0/3660673719 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fccf80d4980 con 0x5619c11e4cc0 2024-03-28T16:49:21.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcbd77fe640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbd0034260 0x7fcbd0036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcbd77fe640 1 -- 172.21.15.170:0/3660673719 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fccf8113aa0 con 0x5619c11e4cc0 2024-03-28T16:49:21.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd05753640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbd0034260 0x7fcbd0036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c139eb80 0x5619c13a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a1740 con 0x5619c139eb80 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd06755640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c139eb80 0x5619c13a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd05753640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbd0034260 0x7fcbd0036720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5619c1364790 tx=0x7fccfc010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd06755640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c139eb80 0x5619c13a0ff0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 realm 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a1b10 con 0x5619c139eb80 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:21.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a26f0 con 0x5619c139eb80 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13a30a0 0x5619c13a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a5aa0 con 0x5619c13a30a0 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd05f54640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13a30a0 0x5619c13a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd05f54640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13a30a0 0x5619c13a5490 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fccf8105070 con 0x5619c13a30a0 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.023+0000 7fcd08439980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd08439980 20 searching for the correct realm 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5619c13a7c80 0x5619c13aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13aa810 con 0x5619c13a7c80 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05753640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5619c13a7c80 0x5619c13aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05753640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5619c13a7c80 0x5619c13aa140 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05753640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fccfc037040 0x7fccfc036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13aa810 con 0x7fccfc037040 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fccfc037040 0x7fccfc036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fccfc037040 0x7fccfc036410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf8105070 con 0x5619c13a30a0 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.027+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x7fccfc037040 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x7fccfc037040 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.031+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x7fccfc037040 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x7fccfc037040 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a7c80 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.035+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c13a30a0 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x5619c139eb80 2024-03-28T16:49:21.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a2390 con 0x7fccfc037040 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a6080 con 0x5619c139eb80 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a6080 con 0x5619c139eb80 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a6050 con 0x7fccfc037040 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 zone default found 2024-03-28T16:49:21.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 4 Realm: () 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 zonegroup default 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a68d0 con 0x5619c13a30a0 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13a6cd0 con 0x5619c13a7c80 2024-03-28T16:49:21.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13ac780 con 0x5619c13a7c80 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 started sync module instance, tier type = 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.039+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13abf10 con 0x7fccfc037040 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.043+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.043+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94668616022640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13ac2c0 con 0x7fccfc037040 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.043+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13ad4e0 con 0x5619c139eb80 2024-03-28T16:49:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.043+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94668616022640] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.043+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.043+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94668616027040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13ad4c0 con 0x5619c139eb80 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.043+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13adf80 con 0x5619c13a7c80 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94668616027040] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94668616030784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13ae330 con 0x5619c13a7c80 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13aeca0 con 0x7fccfc037040 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94668616034576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13af390 con 0x7fccfc037040 2024-03-28T16:49:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13afe60 con 0x5619c139eb80 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94668616034576] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.047+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94668616030784] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.051+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.051+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94668616028096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13afe20 con 0x5619c139eb80 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.051+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b0210 con 0x5619c13a30a0 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.051+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94668616028096] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.051+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.051+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94668616038720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b05c0 con 0x5619c13a30a0 2024-03-28T16:49:21.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.051+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b0f30 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94668616045200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b1b50 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b2500 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94668616038720] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94668616045200] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94668616046256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b2020 con 0x5619c13a30a0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 20 add_watcher() i=0 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 20 add_watcher() i=1 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 20 add_watcher() i=2 2024-03-28T16:49:21.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 20 add_watcher() i=3 2024-03-28T16:49:21.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 20 add_watcher() i=4 2024-03-28T16:49:21.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 20 add_watcher() i=5 2024-03-28T16:49:21.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.055+0000 7fcd08439980 20 add_watcher() i=6 2024-03-28T16:49:21.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94668616046256] v29'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 add_watcher() i=7 2024-03-28T16:49:21.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:21.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b56f0 con 0x5619c139eb80 2024-03-28T16:49:21.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94668616035824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13b2a70 con 0x5619c139eb80 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94668616035824] v29'132 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcbb77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 5 note: GC not initialized 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcbb67fc640 20 reqs_thread_entry: start 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 Filter name: none 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcb27fff640 20 reqs_thread_entry: start 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:21.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13d7690 con 0x5619c139eb80 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13d91f0 con 0x5619c139eb80 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+241 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 rados_obj.operate() r=0 bl.length=193 2024-03-28T16:49:21.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.059+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.7 8:e39c74b6:root::.bucket.meta.testx%3amyfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13daa50 con 0x5619c13a30a0 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~371 out=371b,getxattrs out=298b] v0'0 uv2 ondisk = 0) v8 ==== 338+0+733 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 20 rados_obj.operate() r=0 bl.length=371 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:21.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-03-28T16:49:21.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:21.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-03-28T16:49:21.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13dee40 con 0x5619c139eb80 2024-03-28T16:49:21.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v29'2 uv2 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [create,call version.inc in=38b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e2700 con 0x5619c13a7c80 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [create,call,writefull 0~352,setxattr (123)] v29'5 uv5 ondisk = 0) v8 ==== 332+0+0 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x7 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.063+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 7.9 7:93e5b521:::notify.7:head [notify cookie 94668616238256 in=713b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e2220 con 0x5619c13a30a0 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== watch-notify(notify (1) cookie 94668616046256 notify 124554051598 ret 0) v3 ==== 743+0+0 (crc 0 0 0) 0x5619c11e3580 con 0x5619c13a30a0 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcbe97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051598 cookie 94668616046256 notifier 4550 bl.length()=701 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcbe97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x7 2024-03-28T16:49:21.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcbe97fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcbe97fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(67 notify.7 [notify cookie 94668616238256 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcbe97fa640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fcbc8003ea0 con 0x5619c13a30a0 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify_complete (2) cookie 94668616238256 notify 124554051598 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5619c137e2b0 con 0x5619c13a30a0 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(68 notify.7 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd08439980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_sources= new_sources= 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd08439980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential sources added=[] removed=[] 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd08439980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_dests= new_dests= 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd08439980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential dests added=[] removed=[] 2024-03-28T16:49:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e04b0 con 0x7fccfc037040 2024-03-28T16:49:21.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(69 foo.buckets [call] v29'28 uv28 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.067+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e14c0 con 0x7fccfc037040 2024-03-28T16:49:21.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v29'29 uv29 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd08439980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-03-28T16:49:21.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd08439980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:21.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd08439980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:49:21.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94668616245984 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e14c0 con 0x5619c13a30a0 2024-03-28T16:49:21.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== watch-notify(notify (1) cookie 94668616045200 notify 124554051599 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x5619c137e2b0 con 0x5619c13a30a0 2024-03-28T16:49:21.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcbe97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051599 cookie 94668616045200 notifier 4550 bl.length()=314 2024-03-28T16:49:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcbe97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-03-28T16:49:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcbe97fa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcbe97fa640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fcbc8003ea0 con 0x5619c13a30a0 2024-03-28T16:49:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(71 notify.6 [notify cookie 94668616245984 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== watch-notify(notify_complete (2) cookie 94668616245984 notify 124554051599 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fccf80d6040 con 0x5619c13a30a0 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd08439980 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd08439980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 7.d 7:b4812045:::notify.4:head [notify cookie 94668616245984 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13df940 con 0x5619c139eb80 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== watch-notify(notify (1) cookie 94668616028096 notify 124554051599 ret 0) v3 ==== 197+0+0 (crc 0 0 0) 0x7fcd0004a040 con 0x5619c139eb80 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcbe8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051599 cookie 94668616028096 notifier 4550 bl.length()=155 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(73 notify.4 [notify cookie 94668616245984 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.071+0000 7fcbe8ff9640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fcb300032c0 con 0x5619c139eb80 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94668616245984 notify 124554051599 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fcd000189f0 con 0x5619c139eb80 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(74 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x5619c139eb80 2024-03-28T16:49:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(75 testx/myfoo [call,delete] v29'2 uv2 ondisk = 0) v8 ==== 197+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd08439980 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 from cache 2024-03-28T16:49:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd08439980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94668616245984 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13d91f0 con 0x5619c13a30a0 2024-03-28T16:49:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== watch-notify(notify (1) cookie 94668616045200 notify 124554051600 ret 0) v3 ==== 254+0+0 (crc 0 0 0) 0x7fccf80d6040 con 0x5619c13a30a0 2024-03-28T16:49:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcbe8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051600 cookie 94668616045200 notifier 4550 bl.length()=212 2024-03-28T16:49:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(76 notify.6 [notify cookie 94668616245984 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcbe8ff9640 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fcb30003330 con 0x5619c13a30a0 2024-03-28T16:49:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== watch-notify(notify_complete (2) cookie 94668616245984 notify 124554051600 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5619c1360af0 con 0x5619c13a30a0 2024-03-28T16:49:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(77 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.075+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 8.7 8:e39c74b6:root::.bucket.meta.testx%3amyfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x5619c13a30a0 2024-03-28T16:49:21.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(78 .bucket.meta.testx:myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call,delete] v29'3 uv3 ondisk = 0) v8 ==== 254+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcbb77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94668616035824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x5619c139eb80 2024-03-28T16:49:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94668616035824] v29'133 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94668616022640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x7fccfc037040 2024-03-28T16:49:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94668616022640] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd08439980 20 remove_watcher() i=0 2024-03-28T16:49:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd08439980 2 removed watcher, disabling cache 2024-03-28T16:49:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.079+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94668616027040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x5619c139eb80 2024-03-28T16:49:21.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.083+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94668616027040] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.083+0000 7fcd08439980 20 remove_watcher() i=1 2024-03-28T16:49:21.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.083+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94668616030784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13acba0 con 0x5619c13a7c80 2024-03-28T16:49:21.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.083+0000 7fcd05753640 1 -- 172.21.15.170:0/3660673719 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94668616030784] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccfc00e620 con 0x5619c13a7c80 2024-03-28T16:49:21.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.083+0000 7fcd08439980 20 remove_watcher() i=2 2024-03-28T16:49:21.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.083+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94668616034576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x7fccfc037040 2024-03-28T16:49:21.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.087+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94668616034576] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x7fccfc037040 2024-03-28T16:49:21.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.087+0000 7fcd08439980 20 remove_watcher() i=3 2024-03-28T16:49:21.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.087+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94668616028096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13ae910 con 0x5619c139eb80 2024-03-28T16:49:21.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.087+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94668616028096] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fcd0000a4d0 con 0x5619c139eb80 2024-03-28T16:49:21.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.087+0000 7fcd08439980 20 remove_watcher() i=4 2024-03-28T16:49:21.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.087+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94668616038720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x5619c13a30a0 2024-03-28T16:49:21.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.091+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94668616038720] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.091+0000 7fcd08439980 20 remove_watcher() i=5 2024-03-28T16:49:21.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.091+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94668616045200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13ae910 con 0x5619c13a30a0 2024-03-28T16:49:21.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.091+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94668616045200] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.091+0000 7fcd08439980 20 remove_watcher() i=6 2024-03-28T16:49:21.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.091+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94668616046256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5619c13e22e0 con 0x5619c13a30a0 2024-03-28T16:49:21.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd05f54640 1 -- 172.21.15.170:0/3660673719 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94668616046256] v29'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fccf811c040 con 0x5619c13a30a0 2024-03-28T16:49:21.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 20 remove_watcher() i=7 2024-03-28T16:49:21.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13a30a0 msgr2=0x5619c13a5490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13a30a0 0x5619c13a5490 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c139eb80 msgr2=0x5619c13a0ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c139eb80 0x5619c13a0ff0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fccfc037040 msgr2=0x7fccfc036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fccfc037040 0x7fccfc036410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5619c13a7c80 msgr2=0x5619c13aa140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5619c13a7c80 0x5619c13aa140 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbd0034260 msgr2=0x7fcbd0036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbd0034260 0x7fcbd0036720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5619c1364790 tx=0x7fccfc010000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 msgr2=0x5619c1393780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd06755640 1 -- 172.21.15.170:0/3660673719 reap_dead start 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c1393780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5619c1385820 tx=0x7fccf80f6000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 shutdown_connections 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/3660673719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c11e4cc0 0x5619c1393780 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 >> 172.21.15.170:0/3660673719 conn(0x5619c1386a70 msgr2=0x5619c13840d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 shutdown_connections 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/3660673719 wait complete. 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbe0034300 msgr2=0x7fcbe00367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbe0034300 0x7fcbe00367c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00bf60 tx=0x7fccfc00f000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c137d950 msgr2=0x5619c137dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c137d950 0x5619c137dd70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fccf80e3010 tx=0x7fccf80d0810 comp rx=0 tx=0).stop 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 shutdown_connections 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbe0034300 0x7fcbe00367c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/2867264628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c137d950 0x5619c137dd70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 >> 172.21.15.170:0/2867264628 conn(0x5619c1372760 msgr2=0x5619c1373270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 shutdown_connections 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/2867264628 wait complete. 2024-03-28T16:49:21.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13692c0 msgr2=0x5619c136b6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5619c13692c0 0x5619c136b6b0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c126f870 msgr2=0x5619c1366d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5619c126f870 0x5619c1366d00 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5619c136cf90 msgr2=0x5619c136f380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5619c136cf90 0x5619c136f380 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbec0342b0 msgr2=0x7fcbec036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbec0342b0 0x7fcbec036770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000f70 tx=0x7fccfc0012c0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 msgr2=0x5619c1360120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 --2- 172.21.15.170:0/1954392721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619c126ef50 0x5619c1360120 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fccf801bb20 tx=0x7fccf8018020 comp rx=0 tx=0).stop 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd06755640 1 -- 172.21.15.170:0/1954392721 reap_dead start 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 shutdown_connections 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 >> 172.21.15.170:0/1954392721 conn(0x5619c126a820 msgr2=0x5619c1353a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 shutdown_connections 2024-03-28T16:49:21.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.095+0000 7fcd08439980 1 -- 172.21.15.170:0/1954392721 wait complete. 2024-03-28T16:49:21.104 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-03-28T16:49:21.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-03-28T16:49:21.105 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-03-28T16:49:21.174 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:21.175 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c3d82980 1 Processor -- start 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c3d82980 1 -- start start 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c3d82980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55acff9f4900 0x55acffa85000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c3d82980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55acff9cc4c0 con 0x55acffa85540 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c3d82980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55acff9930f0 con 0x55acff9f4900 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c189d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55acff9f4900 0x55acffa85000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c109c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55acffa85540 0x55acffa87930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38316/0 (socket says 172.21.15.170:38316) 2024-03-28T16:49:21.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c109c640 1 -- 172.21.15.170:0/3494715628 learned_addr learned my addr 172.21.15.170:0/3494715628 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:21.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c189d640 1 -- 172.21.15.170:0/3494715628 >> v1:172.21.15.119:6789/0 conn(0x55acffa85540 legacy=0x55acffa87930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.175+0000 7f98c189d640 1 -- 172.21.15.170:0/3494715628 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acffa8a060 con 0x55acff9f4900 2024-03-28T16:49:21.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c189d640 1 --2- 172.21.15.170:0/3494715628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffa85000 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00a420 tx=0x7f98bc01afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b189e342e129bdab server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c089b640 1 -- 172.21.15.170:0/3494715628 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0aa530 con 0x55acff9f4900 2024-03-28T16:49:21.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c089b640 1 -- 172.21.15.170:0/3494715628 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0a6050 con 0x55acff9f4900 2024-03-28T16:49:21.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c089b640 1 -- 172.21.15.170:0/3494715628 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0aa800 con 0x55acff9f4900 2024-03-28T16:49:21.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c3d82980 1 -- 172.21.15.170:0/3494715628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 msgr2=0x55acffa85000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3494715628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffa85000 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00a420 tx=0x7f98bc01afd0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c3d82980 1 -- 172.21.15.170:0/3494715628 shutdown_connections 2024-03-28T16:49:21.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3494715628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffa85000 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c3d82980 1 -- 172.21.15.170:0/3494715628 >> 172.21.15.170:0/3494715628 conn(0x55acffa7a720 msgr2=0x55acffa7cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c3d82980 1 -- 172.21.15.170:0/3494715628 shutdown_connections 2024-03-28T16:49:21.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.179+0000 7f98c3d82980 1 -- 172.21.15.170:0/3494715628 wait complete. 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c3d82980 1 Processor -- start 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c3d82980 1 -- start start 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c3d82980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb6c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c3d82980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff990620 con 0x55acff9f4900 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb6c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb6c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57024/0 (socket says 172.21.15.170:57024) 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c189d640 1 -- 172.21.15.170:0/3309972279 learned_addr learned my addr 172.21.15.170:0/3309972279 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.191+0000 7f98c189d640 1 -- 172.21.15.170:0/3309972279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc00a100 con 0x55acff9f4900 2024-03-28T16:49:21.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c189d640 1 --2- 172.21.15.170:0/3309972279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb6c020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0ad040 tx=0x7f98bc0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d652ac2a45a27b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f97afe07640 1 -- 172.21.15.170:0/3309972279 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0aa6c0 con 0x55acff9f4900 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f97afe07640 1 -- 172.21.15.170:0/3309972279 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0aa820 con 0x55acff9f4900 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f97afe07640 1 -- 172.21.15.170:0/3309972279 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0aaac0 con 0x55acff9f4900 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- 172.21.15.170:0/3309972279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 msgr2=0x55acffb6c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3309972279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb6c020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0ad040 tx=0x7f98bc0033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- 172.21.15.170:0/3309972279 shutdown_connections 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3309972279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb6c020 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- 172.21.15.170:0/3309972279 >> 172.21.15.170:0/3309972279 conn(0x55acffa7a720 msgr2=0x55acffb67bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- 172.21.15.170:0/3309972279 shutdown_connections 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- 172.21.15.170:0/3309972279 wait complete. 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 Processor -- start 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- start start 2024-03-28T16:49:21.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb72d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff98be00 con 0x55acff9f4900 2024-03-28T16:49:21.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb72d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb72d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57030/0 (socket says 172.21.15.170:57030) 2024-03-28T16:49:21.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c189d640 1 -- 172.21.15.170:0/88446615 learned_addr learned my addr 172.21.15.170:0/88446615 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c189d640 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acffa8a060 con 0x55acff9f4900 2024-03-28T16:49:21.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c189d640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb72d50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f98bc01b4b0 tx=0x7f98bc019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f97ae604640 1 -- 172.21.15.170:0/88446615 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0c5070 con 0x55acff9f4900 2024-03-28T16:49:21.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55acffb73290 con 0x55acff9f4900 2024-03-28T16:49:21.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f97ae604640 1 -- 172.21.15.170:0/88446615 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0b6bb0 con 0x55acff9f4900 2024-03-28T16:49:21.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f97ae604640 1 -- 172.21.15.170:0/88446615 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0b6e70 con 0x55acff9f4900 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.195+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acffb73720 con 0x55acff9f4900 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f97ae604640 1 -- 172.21.15.170:0/88446615 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f98bc0ca410 con 0x55acff9f4900 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f97ae604640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f97a80342b0 0x7f97a8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f97ae604640 1 -- 172.21.15.170:0/88446615 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f98bc0e3840 con 0x55acff9f4900 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c109c640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f97a80342b0 0x7f97a8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffa85540 0x55acffb79930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffb79e70 con 0x55acffa85540 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c209e640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffa85540 0x55acffb79930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c109c640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f97a80342b0 0x7f97a8036770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f98ac001da0 tx=0x7f98ac001300 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c209e640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffa85540 0x55acffb79930 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c209e640 1 -- 172.21.15.170:0/88446615 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffa85540 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffb7a1d0 con 0x55acffa85540 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c209e640 1 -- 172.21.15.170:0/88446615 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffa85540 2024-03-28T16:49:21.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffb7bef0 0x55acffb7e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffb7e820 con 0x55acffb7bef0 2024-03-28T16:49:21.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c189d640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffb7bef0 0x55acffb7e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c189d640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffb7bef0 0x55acffb7e2e0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c189d640 1 -- 172.21.15.170:0/88446615 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f98bc0b1040 con 0x55acffb7bef0 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffb7ad00 con 0x55acffa85540 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c209e640 1 -- 172.21.15.170:0/88446615 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffa85540 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55acffb7fb90 0x55acffb81f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffb825d0 con 0x55acffb7fb90 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.199+0000 7f98c109c640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55acffb7fb90 0x55acffb81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c109c640 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55acffb7fb90 0x55acffb81f80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c109c640 1 -- 172.21.15.170:0/88446615 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffb7fb90 2024-03-28T16:49:21.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c3d82980 1 Processor -- start 2024-03-28T16:49:21.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c3d82980 1 -- start start 2024-03-28T16:49:21.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c3d82980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb90500 0x55acffb90920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c3d82980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff924e30 con 0x55acffb90500 2024-03-28T16:49:21.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb90500 0x55acffb90920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb90500 0x55acffb90920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57042/0 (socket says 172.21.15.170:57042) 2024-03-28T16:49:21.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c189d640 1 -- 172.21.15.170:0/3104532393 learned_addr learned my addr 172.21.15.170:0/3104532393 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c189d640 1 -- 172.21.15.170:0/3104532393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acffb90e60 con 0x55acffb90500 2024-03-28T16:49:21.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c189d640 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb90500 0x55acffb90920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0a9b50 tx=0x7f98bc0b3640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f97a77fe640 1 -- 172.21.15.170:0/3104532393 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0db070 con 0x55acffb90500 2024-03-28T16:49:21.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55acffb91050 con 0x55acffb90500 2024-03-28T16:49:21.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f97a77fe640 1 -- 172.21.15.170:0/3104532393 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0e3d10 con 0x55acffb90500 2024-03-28T16:49:21.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f97a77fe640 1 -- 172.21.15.170:0/3104532393 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0ca840 con 0x55acffb90500 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acffb91ba0 con 0x55acffb90500 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f97a77fe640 1 -- 172.21.15.170:0/3104532393 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f98bc0ca9a0 con 0x55acffb90500 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f97a77fe640 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f979c0342b0 0x7f979c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f97a77fe640 1 -- 172.21.15.170:0/3104532393 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f98bc0edab0 con 0x55acffb90500 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.203+0000 7f98c109c640 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f979c0342b0 0x7f979c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 Processor -- start 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- start start 2024-03-28T16:49:21.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c109c640 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f979c0342b0 0x7f979c036770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55acffb7b060 tx=0x7f98ac00f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffb9d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acff918790 con 0x55acffb9d400 2024-03-28T16:49:21.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffb9d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffb9d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57046/0 (socket says 172.21.15.170:57046) 2024-03-28T16:49:21.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 -- 172.21.15.170:0/1601658121 learned_addr learned my addr 172.21.15.170:0/1601658121 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 -- 172.21.15.170:0/1601658121 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acffb9ddf0 con 0x55acffb9d400 2024-03-28T16:49:21.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 --2- 172.21.15.170:0/1601658121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffb9d820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0e8000 tx=0x7f98bc0d6ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84dcb2675752d0d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f97a57fa640 1 -- 172.21.15.170:0/1601658121 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0edd50 con 0x55acffb9d400 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f97a57fa640 1 -- 172.21.15.170:0/1601658121 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0edeb0 con 0x55acffb9d400 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f97a57fa640 1 -- 172.21.15.170:0/1601658121 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0e5cf0 con 0x55acffb9d400 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- 172.21.15.170:0/1601658121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 msgr2=0x55acffb9d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 --2- 172.21.15.170:0/1601658121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffb9d820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0e8000 tx=0x7f98bc0d6ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- 172.21.15.170:0/1601658121 shutdown_connections 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 --2- 172.21.15.170:0/1601658121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffb9d820 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- 172.21.15.170:0/1601658121 >> 172.21.15.170:0/1601658121 conn(0x55acffb99740 msgr2=0x55acffb977a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- 172.21.15.170:0/1601658121 shutdown_connections 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- 172.21.15.170:0/1601658121 wait complete. 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 Processor -- start 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- start start 2024-03-28T16:49:21.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffba4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c3d82980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acffba4840 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffba4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffba4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57054/0 (socket says 172.21.15.170:57054) 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 learned_addr learned my addr 172.21.15.170:0/2642299706 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acffb91050 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.207+0000 7f98c189d640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffba4300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0c6ff0 tx=0x7f98bc0e8f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f97937fe640 1 -- 172.21.15.170:0/2642299706 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc104070 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f97937fe640 1 -- 172.21.15.170:0/2642299706 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0d6430 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55acffb91050 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f97937fe640 1 -- 172.21.15.170:0/2642299706 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98bc0dfb10 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acffb91050 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f97937fe640 1 -- 172.21.15.170:0/2642299706 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f98bc108070 con 0x55acffb9d400 2024-03-28T16:49:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f97937fe640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f978c0342b0 0x7f978c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f97937fe640 1 -- 172.21.15.170:0/2642299706 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f98bc115af0 con 0x55acffb9d400 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c109c640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f978c0342b0 0x7f978c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffbaf4f0 0x55acffbb19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2100 con 0x55acffbaf4f0 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c209e640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffbaf4f0 0x55acffbb19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c109c640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f978c0342b0 0x7f978c036770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55acffb82990 tx=0x7f98ac010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c209e640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffbaf4f0 0x55acffbb19b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 realm 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb24d0 con 0x55acffbaf4f0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb30b0 con 0x55acffbaf4f0 2024-03-28T16:49:21.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffbb3a60 0x55acffbb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb6460 con 0x55acffbb3a60 2024-03-28T16:49:21.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.211+0000 7f98c189d640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffbb3a60 0x55acffbb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c189d640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffbb3a60 0x55acffbb5e50 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c3d82980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c3d82980 20 searching for the correct realm 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55acffbb8660 0x55acffbbab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbb1f0 con 0x55acffbb8660 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c109c640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55acffbb8660 0x55acffbbab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c109c640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55acffbb8660 0x55acffbbab20 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c109c640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f98ac037040 0x7f98ac036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbb1f0 con 0x7f98ac037040 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f98ac037040 0x7f98ac036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f98ac037040 0x7f98ac036410 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.215+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x7f98ac037040 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x7f98ac037040 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.219+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x7f98ac037040 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x7f98ac037040 2024-03-28T16:49:21.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb8660 2024-03-28T16:49:21.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.223+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbb3a60 2024-03-28T16:49:21.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x55acffbaf4f0 2024-03-28T16:49:21.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb2d50 con 0x7f98ac037040 2024-03-28T16:49:21.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:21.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb6a60 con 0x55acffbaf4f0 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb6a60 con 0x55acffbaf4f0 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb6a10 con 0x7f98ac037040 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 zone default found 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 4 Realm: () 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 zonegroup default 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb72b0 con 0x55acffbb3a60 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbb76f0 con 0x55acffbb8660 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbd150 con 0x55acffbb8660 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 started sync module instance, tier type = 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.227+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbc8f0 con 0x7f98ac037040 2024-03-28T16:49:21.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.231+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.231+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94201513231440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbc870 con 0x7f98ac037040 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.231+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbdeb0 con 0x55acffbaf4f0 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.231+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94201513231440] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.231+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.231+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94201513235824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbde30 con 0x55acffbaf4f0 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.231+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbe630 con 0x55acffbb8660 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94201513235824] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94201513237152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbe990 con 0x55acffbb8660 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbf300 con 0x7f98ac037040 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94201513242480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbf9f0 con 0x7f98ac037040 2024-03-28T16:49:21.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc04c0 con 0x55acffbaf4f0 2024-03-28T16:49:21.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94201513237152] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94201513242480] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94201513246624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc0480 con 0x55acffbaf4f0 2024-03-28T16:49:21.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.235+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbf5c0 con 0x55acffbb3a60 2024-03-28T16:49:21.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94201513246624] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94201513238272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc0d10 con 0x55acffbb3a60 2024-03-28T16:49:21.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc1510 con 0x55acffbb3a60 2024-03-28T16:49:21.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94201513252976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc2130 con 0x55acffbb3a60 2024-03-28T16:49:21.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc2b40 con 0x55acffbb3a60 2024-03-28T16:49:21.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.239+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94201513238272] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94201513248880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc3340 con 0x55acffbb3a60 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=0 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=1 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=2 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=3 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=4 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=5 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94201513252976] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=6 2024-03-28T16:49:21.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94201513248880] v29'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 20 add_watcher() i=7 2024-03-28T16:49:21.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:21.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc6370 con 0x55acffbaf4f0 2024-03-28T16:49:21.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.243+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94201513258656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc3760 con 0x55acffbaf4f0 2024-03-28T16:49:21.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.247+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94201513258656] v29'134 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.247+0000 7f97737fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.247+0000 7f98c3d82980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:21.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.247+0000 7f98c3d82980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:21.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.247+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbce720 con 0x55acffbaf4f0 2024-03-28T16:49:21.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.247+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.247+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbce7c0 con 0x55acffbb3a60 2024-03-28T16:49:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcec20 con 0x55acffbb8660 2024-03-28T16:49:21.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcf4c0 con 0x55acffbaf4f0 2024-03-28T16:49:21.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'135 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcf9f0 con 0x7f98ac037040 2024-03-28T16:49:21.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'38 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcf3a0 con 0x55acffbb8660 2024-03-28T16:49:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.251+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'25 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.255+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd0500 con 0x55acffbaf4f0 2024-03-28T16:49:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.255+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.255+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd0500 con 0x55acffbb3a60 2024-03-28T16:49:21.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.255+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'25 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.255+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd0960 con 0x7f98ac037040 2024-03-28T16:49:21.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.259+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.259+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd0960 con 0x55acffbb8660 2024-03-28T16:49:21.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.259+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.259+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd0e90 con 0x7f98ac037040 2024-03-28T16:49:21.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.259+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'39 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.259+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd0e90 con 0x55acffbb3a60 2024-03-28T16:49:21.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.263+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.263+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd13c0 con 0x7f98ac037040 2024-03-28T16:49:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.263+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.263+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd13c0 con 0x55acffbb3a60 2024-03-28T16:49:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'32 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbaf4f0 2024-03-28T16:49:21.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'136 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd0e90 con 0x55acffbb8660 2024-03-28T16:49:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'32 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x7f98ac037040 2024-03-28T16:49:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'40 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.267+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.271+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'16 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.271+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbaf4f0 2024-03-28T16:49:21.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.271+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'24 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.271+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbaf4f0 2024-03-28T16:49:21.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.271+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'137 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.271+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.275+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'33 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.275+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x7f98ac037040 2024-03-28T16:49:21.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.275+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'26 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.275+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.275+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.275+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.279+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.279+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.279+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.279+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.279+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'26 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.279+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb3a60 2024-03-28T16:49:21.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.283+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.283+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.283+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.283+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb3a60 2024-03-28T16:49:21.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.283+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'26 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.283+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb8660 2024-03-28T16:49:21.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.287+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'34 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.287+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x55acffbb3a60 2024-03-28T16:49:21.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.287+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'33 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.287+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbd1860 con 0x7f98ac037040 2024-03-28T16:49:21.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f97727fc640 20 reqs_thread_entry: start 2024-03-28T16:49:21.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:21.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 20 Filter name: none 2024-03-28T16:49:21.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f96e3fff640 20 reqs_thread_entry: start 2024-03-28T16:49:21.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-03-28T16:49:21.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:94 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcbe70 con 0x55acffbb3a60 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(94 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 20 rados_obj.operate() r=0 bl.length=385 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcc370 con 0x55acffbaf4f0 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(95 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 170+0+15 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:49:21.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:96 7.15 7:a93a5511:::notify.2:head [notify cookie 94201513293888 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcc370 con 0x55acffbb8660 2024-03-28T16:49:21.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== watch-notify(notify (1) cookie 94201513237152 notify 124554051595 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55acff9cc4c0 con 0x55acffbb8660 2024-03-28T16:49:21.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(96 notify.2 [notify cookie 94201513293888 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f97a57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051595 cookie 94201513237152 notifier 4557 bl.length()=170 2024-03-28T16:49:21.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.291+0000 7f97a57fa640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f9784003d60 con 0x55acffbb8660 2024-03-28T16:49:21.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== watch-notify(notify_complete (2) cookie 94201513293888 notify 124554051595 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f98ac036a20 con 0x55acffbb8660 2024-03-28T16:49:21.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(97 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:98 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcc370 con 0x55acffbb3a60 2024-03-28T16:49:21.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(98 p5YnriCv1nAtykxBrupQ [delete] v29'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c3d82980 10 removing email index: 2024-03-28T16:49:21.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c3d82980 10 removing user buckets index 2024-03-28T16:49:21.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c3d82980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-03-28T16:49:21.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [notify cookie 94201513293888 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcc350 con 0x55acffbaf4f0 2024-03-28T16:49:21.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== watch-notify(notify (1) cookie 94201513246624 notify 124554051600 ret 0) v3 ==== 217+0+0 (crc 0 0 0) 0x7f98b400aab0 con 0x55acffbaf4f0 2024-03-28T16:49:21.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f97a57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051600 cookie 94201513246624 notifier 4557 bl.length()=175 2024-03-28T16:49:21.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f97a57fa640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f9784003d60 con 0x55acffbaf4f0 2024-03-28T16:49:21.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(99 notify.4 [notify cookie 94201513293888 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(100 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== watch-notify(notify_complete (2) cookie 94201513293888 notify 124554051600 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f98b400ad10 con 0x55acffbaf4f0 2024-03-28T16:49:21.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.295+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:101 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcc350 con 0x55acffbaf4f0 2024-03-28T16:49:21.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(101 testx$tenanteduser.buckets [delete] v29'3 uv3 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c3d82980 10 removing user index: testx$tenanteduser 2024-03-28T16:49:21.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c3d82980 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-03-28T16:49:21.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c3d82980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-03-28T16:49:21.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:102 7.e 7:7759931f:::notify.3:head [notify cookie 94201513293792 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbccc40 con 0x7f98ac037040 2024-03-28T16:49:21.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== watch-notify(notify (1) cookie 94201513242480 notify 124554051597 ret 0) v3 ==== 209+0+0 (crc 0 0 0) 0x7f98b400ad10 con 0x7f98ac037040 2024-03-28T16:49:21.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f97a4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051597 cookie 94201513242480 notifier 4557 bl.length()=167 2024-03-28T16:49:21.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(102 notify.3 [notify cookie 94201513293792 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f97a4ff9640 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:103 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7f96ec0032c0 con 0x7f98ac037040 2024-03-28T16:49:21.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(103 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== watch-notify(notify_complete (2) cookie 94201513293792 notify 124554051597 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f98b4042650 con 0x7f98ac037040 2024-03-28T16:49:21.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:104 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbcb7e0 con 0x55acffbb3a60 2024-03-28T16:49:21.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.299+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(104 testx$tenanteduser [call,call,delete] v29'8 uv8 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f97737fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94201513258656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbccc40 con 0x55acffbaf4f0 2024-03-28T16:49:21.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94201513258656] v29'138 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94201513231440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc2ea0 con 0x7f98ac037040 2024-03-28T16:49:21.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94201513231440] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c3d82980 20 remove_watcher() i=0 2024-03-28T16:49:21.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c3d82980 2 removed watcher, disabling cache 2024-03-28T16:49:21.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94201513235824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc2ea0 con 0x55acffbaf4f0 2024-03-28T16:49:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.303+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94201513235824] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c3d82980 20 remove_watcher() i=1 2024-03-28T16:49:21.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94201513237152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbd570 con 0x55acffbb8660 2024-03-28T16:49:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c109c640 1 -- 172.21.15.170:0/2642299706 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94201513237152] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98ac00e6c0 con 0x55acffbb8660 2024-03-28T16:49:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c3d82980 20 remove_watcher() i=2 2024-03-28T16:49:21.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94201513242480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc2ea0 con 0x7f98ac037040 2024-03-28T16:49:21.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94201513242480] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x7f98ac037040 2024-03-28T16:49:21.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c3d82980 20 remove_watcher() i=3 2024-03-28T16:49:21.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.307+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94201513246624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbd570 con 0x55acffbaf4f0 2024-03-28T16:49:21.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.311+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94201513246624] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98b40083f0 con 0x55acffbaf4f0 2024-03-28T16:49:21.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.311+0000 7f98c3d82980 20 remove_watcher() i=4 2024-03-28T16:49:21.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.311+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94201513238272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbef70 con 0x55acffbb3a60 2024-03-28T16:49:21.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.311+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94201513238272] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.311+0000 7f98c3d82980 20 remove_watcher() i=5 2024-03-28T16:49:21.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.311+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94201513252976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbc2ea0 con 0x55acffbb3a60 2024-03-28T16:49:21.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94201513252976] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 20 remove_watcher() i=6 2024-03-28T16:49:21.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94201513248880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55acffbbd570 con 0x55acffbb3a60 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c189d640 1 -- 172.21.15.170:0/2642299706 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94201513248880] v29'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f98bc0fc070 con 0x55acffbb3a60 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 20 remove_watcher() i=7 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffbb3a60 msgr2=0x55acffbb5e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffbb3a60 0x55acffbb5e50 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffbaf4f0 msgr2=0x55acffbb19b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffbaf4f0 0x55acffbb19b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f98ac037040 msgr2=0x7f98ac036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f98ac037040 0x7f98ac036410 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55acffbb8660 msgr2=0x55acffbbab20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55acffbb8660 0x55acffbbab20 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f978c0342b0 msgr2=0x7f978c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f978c0342b0 0x7f978c036770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55acffb82990 tx=0x7f98ac010000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 msgr2=0x55acffba4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/2642299706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb9d400 0x55acffba4300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0c6ff0 tx=0x7f98bc0e8f90 comp rx=0 tx=0).stop 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c209e640 1 -- 172.21.15.170:0/2642299706 reap_dead start 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 shutdown_connections 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 >> 172.21.15.170:0/2642299706 conn(0x55acffb99740 msgr2=0x55acffb97770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 shutdown_connections 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/2642299706 wait complete. 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f979c0342b0 msgr2=0x7f979c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f979c0342b0 0x7f979c036770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55acffb7b060 tx=0x7f98ac00f000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb90500 msgr2=0x55acffb90920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb90500 0x55acffb90920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0a9b50 tx=0x7f98bc0b3640 comp rx=0 tx=0).stop 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 shutdown_connections 2024-03-28T16:49:21.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f979c0342b0 0x7f979c036770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/3104532393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acffb90500 0x55acffb90920 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 >> 172.21.15.170:0/3104532393 conn(0x55acffb85310 msgr2=0x55acffb85e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 shutdown_connections 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/3104532393 wait complete. 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffb7bef0 msgr2=0x55acffb7e2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55acffb7bef0 0x55acffb7e2e0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffa85540 msgr2=0x55acffb79930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55acffa85540 0x55acffb79930 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55acffb7fb90 msgr2=0x55acffb81f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55acffb7fb90 0x55acffb81f80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f97a80342b0 msgr2=0x7f97a8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f97a80342b0 0x7f97a8036770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f98ac001da0 tx=0x7f98ac001300 comp rx=0 tx=0).stop 2024-03-28T16:49:21.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 msgr2=0x55acffb72d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 --2- 172.21.15.170:0/88446615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acff9f4900 0x55acffb72d50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f98bc01b4b0 tx=0x7f98bc019810 comp rx=0 tx=0).stop 2024-03-28T16:49:21.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c209e640 1 -- 172.21.15.170:0/88446615 reap_dead start 2024-03-28T16:49:21.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 shutdown_connections 2024-03-28T16:49:21.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 >> 172.21.15.170:0/88446615 conn(0x55acffa7a720 msgr2=0x55acffb666c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 shutdown_connections 2024-03-28T16:49:21.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.315+0000 7f98c3d82980 1 -- 172.21.15.170:0/88446615 wait complete. 2024-03-28T16:49:21.333 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-03-28T16:49:21.336 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-03-28T16:49:21.383 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:21.383 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-03-28T16:49:21.383 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-03-28T16:49:21.384 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-03-28T16:49:21.419 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:21.419 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:21.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8fd4d980 1 Processor -- start 2024-03-28T16:49:21.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8fd4d980 1 -- start start 2024-03-28T16:49:21.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8fd4d980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55f5f31a3e50 0x55f5f31a4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8fd4d980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f5f30f14c0 con 0x55f5f31a4770 2024-03-28T16:49:21.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8fd4d980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f5f30b80f0 con 0x55f5f31a3e50 2024-03-28T16:49:21.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8d868640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55f5f31a3e50 0x55f5f31a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8d067640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55f5f31a4770 0x55f5f31a5130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38324/0 (socket says 172.21.15.170:38324) 2024-03-28T16:49:21.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8d067640 1 -- 172.21.15.170:0/398104457 learned_addr learned my addr 172.21.15.170:0/398104457 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:21.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.419+0000 7f3c8d868640 1 -- 172.21.15.170:0/398104457 >> v1:172.21.15.119:6789/0 conn(0x55f5f31a4770 legacy=0x55f5f31a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8d868640 1 -- 172.21.15.170:0/398104457 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f31a5960 con 0x55f5f31a3e50 2024-03-28T16:49:21.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8d868640 1 --2- 172.21.15.170:0/398104457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f31a4230 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a420 tx=0x7f3c8001afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f52a79479a728dc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8c866640 1 -- 172.21.15.170:0/398104457 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800aa530 con 0x55f5f31a3e50 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8c866640 1 -- 172.21.15.170:0/398104457 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800a6050 con 0x55f5f31a3e50 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8c866640 1 -- 172.21.15.170:0/398104457 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800aa800 con 0x55f5f31a3e50 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/398104457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 msgr2=0x55f5f31a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/398104457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f31a4230 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a420 tx=0x7f3c8001afd0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/398104457 shutdown_connections 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/398104457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f31a4230 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/398104457 >> 172.21.15.170:0/398104457 conn(0x55f5f319f720 msgr2=0x55f5f31a1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/398104457 shutdown_connections 2024-03-28T16:49:21.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.423+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/398104457 wait complete. 2024-03-28T16:49:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 Processor -- start 2024-03-28T16:49:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- start start 2024-03-28T16:49:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f328f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f30b5620 con 0x55f5f31a3e50 2024-03-28T16:49:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f328f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f328f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57066/0 (socket says 172.21.15.170:57066) 2024-03-28T16:49:21.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 -- 172.21.15.170:0/2024455452 learned_addr learned my addr 172.21.15.170:0/2024455452 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 -- 172.21.15.170:0/2024455452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8000a100 con 0x55f5f31a3e50 2024-03-28T16:49:21.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 --2- 172.21.15.170:0/2024455452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f328f400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c800ad040 tx=0x7f3c800033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25338ec6a2f78256 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3b7be07640 1 -- 172.21.15.170:0/2024455452 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800aa6c0 con 0x55f5f31a3e50 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3b7be07640 1 -- 172.21.15.170:0/2024455452 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800aa820 con 0x55f5f31a3e50 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3b7be07640 1 -- 172.21.15.170:0/2024455452 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800aaac0 con 0x55f5f31a3e50 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/2024455452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 msgr2=0x55f5f328f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/2024455452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f328f400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c800ad040 tx=0x7f3c800033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/2024455452 shutdown_connections 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/2024455452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f328f400 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/2024455452 >> 172.21.15.170:0/2024455452 conn(0x55f5f319f720 msgr2=0x55f5f3119260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/2024455452 shutdown_connections 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/2024455452 wait complete. 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 Processor -- start 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- start start 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f32960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8fd4d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f30b0e00 con 0x55f5f31a3e50 2024-03-28T16:49:21.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f32960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f32960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57070/0 (socket says 172.21.15.170:57070) 2024-03-28T16:49:21.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 -- 172.21.15.170:0/3841234863 learned_addr learned my addr 172.21.15.170:0/3841234863 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.439+0000 7f3c8d868640 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f31a5960 con 0x55f5f31a3e50 2024-03-28T16:49:21.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8d868640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f32960f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3c80003430 tx=0x7f3c80019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3b7a604640 1 -- 172.21.15.170:0/3841234863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800b6950 con 0x55f5f31a3e50 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5f3296630 con 0x55f5f31a3e50 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3b7a604640 1 -- 172.21.15.170:0/3841234863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800b6ab0 con 0x55f5f31a3e50 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3b7a604640 1 -- 172.21.15.170:0/3841234863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800b6d50 con 0x55f5f31a3e50 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5f3296ac0 con 0x55f5f31a3e50 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3b7a604640 1 -- 172.21.15.170:0/3841234863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3c800c0450 con 0x55f5f31a3e50 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3b7a604640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b740342b0 0x7f3b74036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3b7a604640 1 -- 172.21.15.170:0/3841234863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f3c800e3d50 con 0x55f5f31a3e50 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8d067640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b740342b0 0x7f3b74036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f31a4770 0x55f5f329cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f329d2b0 con 0x55f5f31a4770 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8e069640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f31a4770 0x55f5f329cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8d067640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b740342b0 0x7f3b74036770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c78000f70 tx=0x7f3c780012c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8e069640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f31a4770 0x55f5f329cd70 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8e069640 1 -- 172.21.15.170:0/3841234863 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f31a4770 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f329d610 con 0x55f5f31a4770 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8e069640 1 -- 172.21.15.170:0/3841234863 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f31a4770 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f329f330 0x55f5f32a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32a1c60 con 0x55f5f329f330 2024-03-28T16:49:21.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8d868640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f329f330 0x55f5f32a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.443+0000 7f3c8d868640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f329f330 0x55f5f32a1720 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d868640 1 -- 172.21.15.170:0/3841234863 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f329f330 2024-03-28T16:49:21.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f329e170 con 0x55f5f31a4770 2024-03-28T16:49:21.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8e069640 1 -- 172.21.15.170:0/3841234863 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f31a4770 2024-03-28T16:49:21.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f5f32a2fd0 0x55f5f32a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32a5980 con 0x55f5f32a2fd0 2024-03-28T16:49:21.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d067640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f5f32a2fd0 0x55f5f32a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d067640 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f5f32a2fd0 0x55f5f32a53c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d067640 1 -- 172.21.15.170:0/3841234863 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32a2fd0 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 Processor -- start 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 -- start start 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f32b38b0 0x55f5f32b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f3049e30 con 0x55f5f32b38b0 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f32b38b0 0x55f5f32b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f32b38b0 0x55f5f32b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57086/0 (socket says 172.21.15.170:57086) 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d868640 1 -- 172.21.15.170:0/4283525642 learned_addr learned my addr 172.21.15.170:0/4283525642 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d868640 1 -- 172.21.15.170:0/4283525642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f32b4210 con 0x55f5f32b38b0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8d868640 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f32b38b0 0x55f5f32b3cd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3c8001ae00 tx=0x7f3c800a9890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3b737fe640 1 -- 172.21.15.170:0/4283525642 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800d9070 con 0x55f5f32b38b0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3b737fe640 1 -- 172.21.15.170:0/4283525642 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800c85c0 con 0x55f5f32b38b0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5f32b4460 con 0x55f5f32b38b0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3b737fe640 1 -- 172.21.15.170:0/4283525642 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800c8860 con 0x55f5f32b38b0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.447+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5f32b4fb0 con 0x55f5f32b38b0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3b737fe640 1 -- 172.21.15.170:0/4283525642 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3c800070a0 con 0x55f5f32b38b0 2024-03-28T16:49:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3b737fe640 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b680342b0 0x7f3b68036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3b737fe640 1 -- 172.21.15.170:0/4283525642 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f3c800e6020 con 0x55f5f32b38b0 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8d067640 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b680342b0 0x7f3b68036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 Processor -- start 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- start start 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8d067640 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b680342b0 0x7f3b68036770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800bf60 tx=0x7f3c7800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f328b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f303d790 con 0x55f5f328b0c0 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f328b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f328b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57088/0 (socket says 172.21.15.170:57088) 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8d868640 1 -- 172.21.15.170:0/1213026111 learned_addr learned my addr 172.21.15.170:0/1213026111 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8d868640 1 -- 172.21.15.170:0/1213026111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f3289940 con 0x55f5f328b0c0 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8d868640 1 --2- 172.21.15.170:0/1213026111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f328b4e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3c800c8d00 tx=0x7f3c800c96a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1bf67ae3378746 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3b717fa640 1 -- 172.21.15.170:0/1213026111 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800f2070 con 0x55f5f328b0c0 2024-03-28T16:49:21.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3b717fa640 1 -- 172.21.15.170:0/1213026111 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800b8070 con 0x55f5f328b0c0 2024-03-28T16:49:21.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3b717fa640 1 -- 172.21.15.170:0/1213026111 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800d2b20 con 0x55f5f328b0c0 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1213026111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 msgr2=0x55f5f328b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1213026111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f328b4e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3c800c8d00 tx=0x7f3c800c96a0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1213026111 shutdown_connections 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1213026111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f328b4e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1213026111 >> 172.21.15.170:0/1213026111 conn(0x55f5f32bcbf0 msgr2=0x55f5f32bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1213026111 shutdown_connections 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1213026111 wait complete. 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 Processor -- start 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 -- start start 2024-03-28T16:49:21.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.451+0000 7f3c8fd4d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f32c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f32c9ea0 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f32c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8d868640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f32c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57092/0 (socket says 172.21.15.170:57092) 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 learned_addr learned my addr 172.21.15.170:0/1406852032 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f32b4fb0 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8d868640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f32c9960 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3c800c9960 tx=0x7f3c800b8350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3b5f7fe640 1 -- 172.21.15.170:0/1406852032 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800fe070 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f5f32ca0a0 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3b5f7fe640 1 -- 172.21.15.170:0/1406852032 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800d5dd0 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3b5f7fe640 1 -- 172.21.15.170:0/1406852032 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c800d26f0 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5f32ca4d0 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3b5f7fe640 1 -- 172.21.15.170:0/1406852032 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3c800c6720 con 0x55f5f328b0c0 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3b5f7fe640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b580342b0 0x7f3b58036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3b5f7fe640 1 -- 172.21.15.170:0/1406852032 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f3c80103ac0 con 0x55f5f328b0c0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8d067640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b580342b0 0x7f3b58036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f32d4d20 0x55f5f32d71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d7910 con 0x55f5f32d4d20 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8e069640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f32d4d20 0x55f5f32d71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8d067640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b580342b0 0x7f3b58036770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55f5f329e4d0 tx=0x7f3c78010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8e069640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f32d4d20 0x55f5f32d71c0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 20 realm 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d7ce0 con 0x55f5f32d4d20 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.455+0000 7f3c8fd4d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8530 con 0x55f5f32d4d20 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f32d95a0 0x55f5f32db990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32dc030 con 0x55f5f32d95a0 2024-03-28T16:49:21.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d868640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f32d95a0 0x55f5f32db990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d868640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f32d95a0 0x55f5f32db990 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 20 searching for the correct realm 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f5f32de1f0 0x55f5f32e06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e0d30 con 0x55f5f32de1f0 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d067640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f5f32de1f0 0x55f5f32e06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d067640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f5f32de1f0 0x55f5f32e06b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d067640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3c78037040 0x7f3c78036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e0d30 con 0x7f3c78037040 2024-03-28T16:49:21.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8e069640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3c78037040 0x7f3c78036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8e069640 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3c78037040 0x7f3c78036410 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.459+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x7f3c78037040 2024-03-28T16:49:21.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.463+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x7f3c78037040 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x7f3c78037040 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.467+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x7f3c78037040 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32de1f0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d95a0 2024-03-28T16:49:21.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x55f5f32d4d20 2024-03-28T16:49:21.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32d8890 con 0x7f3c78037040 2024-03-28T16:49:21.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:21.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32dc5f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.471+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32dc5f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32dc5a0 con 0x7f3c78037040 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 zone default found 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 4 Realm: () 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 zonegroup default 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32dce40 con 0x55f5f32d95a0 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32dd270 con 0x55f5f32de1f0 2024-03-28T16:49:21.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e2c70 con 0x55f5f32de1f0 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 started sync module instance, tier type = 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e2430 con 0x7f3c78037040 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94514835234704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e23b0 con 0x7f3c78037040 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e39d0 con 0x55f5f32d4d20 2024-03-28T16:49:21.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.475+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94514835234704] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.479+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.479+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94514835239056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e3950 con 0x55f5f32d4d20 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.479+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e41b0 con 0x55f5f32de1f0 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.479+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.479+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94514835241600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e4a00 con 0x55f5f32de1f0 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.479+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e53b0 con 0x7f3c78037040 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.479+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94514835239056] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94514835240112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e5760 con 0x7f3c78037040 2024-03-28T16:49:21.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e60d0 con 0x55f5f32d4d20 2024-03-28T16:49:21.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94514835241600] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94514835240112] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94514835246800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e5c30 con 0x55f5f32d4d20 2024-03-28T16:49:21.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.483+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e6430 con 0x55f5f32d95a0 2024-03-28T16:49:21.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94514835246800] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94514835250224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e6050 con 0x55f5f32d95a0 2024-03-28T16:49:21.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e7120 con 0x55f5f32d95a0 2024-03-28T16:49:21.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94514835250224] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94514835253696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e7920 con 0x55f5f32d95a0 2024-03-28T16:49:21.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.487+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e8290 con 0x55f5f32d95a0 2024-03-28T16:49:21.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94514835253696] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94514835256720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e8290 con 0x55f5f32d95a0 2024-03-28T16:49:21.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=0 2024-03-28T16:49:21.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=1 2024-03-28T16:49:21.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=2 2024-03-28T16:49:21.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=3 2024-03-28T16:49:21.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=4 2024-03-28T16:49:21.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=5 2024-03-28T16:49:21.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=6 2024-03-28T16:49:21.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94514835256720] v29'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 20 add_watcher() i=7 2024-03-28T16:49:21.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:21.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32eb650 con 0x55f5f32d4d20 2024-03-28T16:49:21.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.491+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94514835259328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32e8bd0 con 0x55f5f32d4d20 2024-03-28T16:49:21.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.495+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94514835259328] v29'139 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.495+0000 7f3b3ffff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.495+0000 7f3c8fd4d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:21.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.495+0000 7f3c8fd4d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:21.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.495+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f3c90 con 0x55f5f32d4d20 2024-03-28T16:49:21.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.495+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.495+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f4150 con 0x55f5f32d95a0 2024-03-28T16:49:21.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f4150 con 0x55f5f32de1f0 2024-03-28T16:49:21.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f4b20 con 0x55f5f32d4d20 2024-03-28T16:49:21.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'140 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f1dd0 con 0x7f3c78037040 2024-03-28T16:49:21.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'41 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.499+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f1dd0 con 0x55f5f32de1f0 2024-03-28T16:49:21.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.503+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.503+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5d00 con 0x55f5f32d4d20 2024-03-28T16:49:21.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.503+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.503+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5d00 con 0x55f5f32d95a0 2024-03-28T16:49:21.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.507+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'27 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.507+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f6230 con 0x7f3c78037040 2024-03-28T16:49:21.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.507+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.507+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f6230 con 0x55f5f32de1f0 2024-03-28T16:49:21.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.507+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.507+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f6760 con 0x7f3c78037040 2024-03-28T16:49:21.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.511+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'42 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.511+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f6760 con 0x55f5f32d95a0 2024-03-28T16:49:21.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.511+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.511+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f6c90 con 0x7f3c78037040 2024-03-28T16:49:21.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.511+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.511+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f6c90 con 0x55f5f32d95a0 2024-03-28T16:49:21.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.515+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'34 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.515+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5c00 con 0x55f5f32d4d20 2024-03-28T16:49:21.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.515+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'141 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.515+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5c00 con 0x55f5f32de1f0 2024-03-28T16:49:21.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.519+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'35 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.519+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5c00 con 0x7f3c78037040 2024-03-28T16:49:21.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.519+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'43 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.519+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5c00 con 0x55f5f32de1f0 2024-03-28T16:49:21.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.519+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.519+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5c90 con 0x55f5f32d4d20 2024-03-28T16:49:21.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.523+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'25 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.523+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5c90 con 0x55f5f32d4d20 2024-03-28T16:49:21.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.523+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'142 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.523+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f5c90 con 0x55f5f32de1f0 2024-03-28T16:49:21.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.523+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'36 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.523+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x7f3c78037040 2024-03-28T16:49:21.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32de1f0 2024-03-28T16:49:21.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32de1f0 2024-03-28T16:49:21.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32de1f0 2024-03-28T16:49:21.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.527+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32de1f0 2024-03-28T16:49:21.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.531+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'28 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.531+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32d95a0 2024-03-28T16:49:21.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.531+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.531+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32de1f0 2024-03-28T16:49:21.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.531+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.531+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32d95a0 2024-03-28T16:49:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.535+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'28 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.535+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32de1f0 2024-03-28T16:49:21.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.539+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'37 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.539+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x55f5f32d95a0 2024-03-28T16:49:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.539+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'35 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.539+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f7130 con 0x7f3c78037040 2024-03-28T16:49:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.539+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3b3effd640 20 reqs_thread_entry: start 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 Filter name: none 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3b3cff9640 20 reqs_thread_entry: start 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3299f30 con 0x7f3c78037040 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv29 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:21.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:95 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f32f0b70 con 0x55f5f32de1f0 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(95 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 get_obj_state: octx=0x7f3c88008c30 obj=myfoo:four state=0x55f5f3313498 s->prefetch_data=0 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:96 3.a 3:50c1b840:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f33144d0 con 0x55f5f32de1f0 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(96 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1_four [getxattrs out=849b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 234+0+865 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 10 manifest: total_size = 4 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.2134864676430339948 2024-03-28T16:49:21.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 get_obj_state: octx=0x7f3c88008c30 obj=myfoo:four state=0x55f5f3313498 s->prefetch_data=0 2024-03-28T16:49:21.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 prepare_atomic_modification: state is not atomic. state=0x55f5f3313498 2024-03-28T16:49:21.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 2024-03-28T16:49:21.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 4.9 4:916ca974:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3318f90 con 0x55f5f32d4d20 2024-03-28T16:49:21.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(97 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 [stat,call,call] v29'9 uv9 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.543+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:98 3.a 3:50c1b840:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3318f90 con 0x55f5f32de1f0 2024-03-28T16:49:21.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(98 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1_four [call] v29'2 uv2 ondisk = 0) v8 ==== 192+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 4.9 4:916ca974:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32d4d20 2024-03-28T16:49:21.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3b3ffff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94514835259328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32d4d20 2024-03-28T16:49:21.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(100 data_loggenerations_metadata [watch unwatch cookie 94514835259328] v29'143 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94514835234704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x7f3c78037040 2024-03-28T16:49:21.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(99 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 [stat,call,call] v29'10 uv10 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3b717fa640 20 handle_completion(): completion ok for obj=four 2024-03-28T16:49:21.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(101 notify.0 [watch unwatch cookie 94514835234704] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8fd4d980 20 remove_watcher() i=0 2024-03-28T16:49:21.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8fd4d980 2 removed watcher, disabling cache 2024-03-28T16:49:21.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.547+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:102 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94514835239056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32d4d20 2024-03-28T16:49:21.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.551+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(102 notify.1 [watch unwatch cookie 94514835239056] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.551+0000 7f3c8fd4d980 20 remove_watcher() i=1 2024-03-28T16:49:21.551 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.551+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:103 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94514835241600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32de1f0 2024-03-28T16:49:21.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.551+0000 7f3c8d067640 1 -- 172.21.15.170:0/1406852032 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(103 notify.2 [watch unwatch cookie 94514835241600] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c7800e620 con 0x55f5f32de1f0 2024-03-28T16:49:21.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.551+0000 7f3c8fd4d980 20 remove_watcher() i=2 2024-03-28T16:49:21.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.551+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94514835240112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x7f3c78037040 2024-03-28T16:49:21.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.555+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(104 notify.3 [watch unwatch cookie 94514835240112] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x7f3c78037040 2024-03-28T16:49:21.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.555+0000 7f3c8fd4d980 20 remove_watcher() i=3 2024-03-28T16:49:21.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.555+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94514835246800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32d4d20 2024-03-28T16:49:21.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.555+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(105 notify.4 [watch unwatch cookie 94514835246800] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c880083f0 con 0x55f5f32d4d20 2024-03-28T16:49:21.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.555+0000 7f3c8fd4d980 20 remove_watcher() i=4 2024-03-28T16:49:21.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.555+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94514835250224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32d95a0 2024-03-28T16:49:21.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.559+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(106 notify.5 [watch unwatch cookie 94514835250224] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.559+0000 7f3c8fd4d980 20 remove_watcher() i=5 2024-03-28T16:49:21.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.559+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:107 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94514835253696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32d95a0 2024-03-28T16:49:21.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.559+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(107 notify.6 [watch unwatch cookie 94514835253696] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.559+0000 7f3c8fd4d980 20 remove_watcher() i=6 2024-03-28T16:49:21.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.559+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:108 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94514835256720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55f5f3319530 con 0x55f5f32d95a0 2024-03-28T16:49:21.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8d868640 1 -- 172.21.15.170:0/1406852032 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(108 notify.7 [watch unwatch cookie 94514835256720] v29'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3c800c6070 con 0x55f5f32d95a0 2024-03-28T16:49:21.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 20 remove_watcher() i=7 2024-03-28T16:49:21.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f32d95a0 msgr2=0x55f5f32db990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f32d95a0 0x55f5f32db990 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f32d4d20 msgr2=0x55f5f32d71c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f32d4d20 0x55f5f32d71c0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3c78037040 msgr2=0x7f3c78036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f3c78037040 0x7f3c78036410 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f5f32de1f0 msgr2=0x55f5f32e06b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f5f32de1f0 0x55f5f32e06b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b580342b0 msgr2=0x7f3b58036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b580342b0 0x7f3b58036770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55f5f329e4d0 tx=0x7f3c78010000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 msgr2=0x55f5f32c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/1406852032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f328b0c0 0x55f5f32c9960 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3c800c9960 tx=0x7f3c800b8350 comp rx=0 tx=0).stop 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8e069640 1 -- 172.21.15.170:0/1406852032 reap_dead start 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 shutdown_connections 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 >> 172.21.15.170:0/1406852032 conn(0x55f5f32bcbf0 msgr2=0x55f5f32ba250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 shutdown_connections 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/1406852032 wait complete. 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b680342b0 msgr2=0x7f3b68036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b680342b0 0x7f3b68036770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800bf60 tx=0x7f3c7800f000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f32b38b0 msgr2=0x55f5f32b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f32b38b0 0x55f5f32b3cd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3c8001ae00 tx=0x7f3c800a9890 comp rx=0 tx=0).stop 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 shutdown_connections 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b680342b0 0x7f3b68036770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/4283525642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f32b38b0 0x55f5f32b3cd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 >> 172.21.15.170:0/4283525642 conn(0x55f5f32a86c0 msgr2=0x55f5f32a91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 shutdown_connections 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/4283525642 wait complete. 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f329f330 msgr2=0x55f5f32a1720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f5f329f330 0x55f5f32a1720 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f31a4770 msgr2=0x55f5f329cd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f5f31a4770 0x55f5f329cd70 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f5f32a2fd0 msgr2=0x55f5f32a53c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f5f32a2fd0 0x55f5f32a53c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b740342b0 msgr2=0x7f3b74036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3b740342b0 0x7f3b74036770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c78000f70 tx=0x7f3c780012c0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 msgr2=0x55f5f32960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 --2- 172.21.15.170:0/3841234863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f5f31a3e50 0x55f5f32960f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3c80003430 tx=0x7f3c80019810 comp rx=0 tx=0).stop 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8e069640 1 -- 172.21.15.170:0/3841234863 reap_dead start 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 shutdown_connections 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 >> 172.21.15.170:0/3841234863 conn(0x55f5f319f720 msgr2=0x55f5f32889b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 shutdown_connections 2024-03-28T16:49:21.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.563+0000 7f3c8fd4d980 1 -- 172.21.15.170:0/3841234863 wait complete. 2024-03-28T16:49:21.573 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-03-28T16:49:21.573 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-03-28T16:49:21.573 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-03-28T16:49:21.652 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:21.652 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:21.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.651+0000 7ffbbb2f2980 1 Processor -- start 2024-03-28T16:49:21.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.651+0000 7ffbbb2f2980 1 -- start start 2024-03-28T16:49:21.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.651+0000 7ffbbb2f2980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x558553e56310 0x558553e58700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.651+0000 7ffbbb2f2980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558553d9e520 con 0x558553dc6070 2024-03-28T16:49:21.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.651+0000 7ffbbb2f2980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x558553d650f0 con 0x558553e56310 2024-03-28T16:49:21.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x558553e56310 0x558553e58700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb8e0d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x558553dc6070 0x558553e54bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38330/0 (socket says 172.21.15.170:38330) 2024-03-28T16:49:21.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/754463346 learned_addr learned my addr 172.21.15.170:0/754463346 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:21.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb3fff640 1 -- 172.21.15.170:0/754463346 >> v1:172.21.15.119:6789/0 conn(0x558553dc6070 legacy=0x558553e54bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb3fff640 1 -- 172.21.15.170:0/754463346 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558553e59ce0 con 0x558553e56310 2024-03-28T16:49:21.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/754463346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553e56310 0x558553e58700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffba400a480 tx=0x7ffba401b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4597326e7c846d6c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb37fe640 1 -- 172.21.15.170:0/754463346 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffba40ab540 con 0x558553e56310 2024-03-28T16:49:21.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb37fe640 1 -- 172.21.15.170:0/754463346 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba40ab6a0 con 0x558553e56310 2024-03-28T16:49:21.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.655+0000 7ffbb37fe640 1 -- 172.21.15.170:0/754463346 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffba40ab960 con 0x558553e56310 2024-03-28T16:49:21.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.659+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/754463346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553e56310 msgr2=0x558553e58700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.659+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/754463346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553e56310 0x558553e58700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffba400a480 tx=0x7ffba401b280 comp rx=0 tx=0).stop 2024-03-28T16:49:21.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.659+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/754463346 shutdown_connections 2024-03-28T16:49:21.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.659+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/754463346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553e56310 0x558553e58700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.659+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/754463346 >> 172.21.15.170:0/754463346 conn(0x558553e4c6d0 msgr2=0x558553e4eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.659+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/754463346 shutdown_connections 2024-03-28T16:49:21.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.659+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/754463346 wait complete. 2024-03-28T16:49:21.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbbb2f2980 1 Processor -- start 2024-03-28T16:49:21.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbbb2f2980 1 -- start start 2024-03-28T16:49:21.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbbb2f2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f3bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbbb2f2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558553d62620 con 0x558553dc6070 2024-03-28T16:49:21.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f3bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f3bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57098/0 (socket says 172.21.15.170:57098) 2024-03-28T16:49:21.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2421206616 learned_addr learned my addr 172.21.15.170:0/2421206616 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2421206616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4005d30 con 0x558553dc6070 2024-03-28T16:49:21.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.671+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/2421206616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f3bea0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffbac001c00 tx=0x7ffbac00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2859eb53ada2966 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffaa6e05640 1 -- 172.21.15.170:0/2421206616 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac00e690 con 0x558553dc6070 2024-03-28T16:49:21.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffaa6e05640 1 -- 172.21.15.170:0/2421206616 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac00e810 con 0x558553dc6070 2024-03-28T16:49:21.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffaa6e05640 1 -- 172.21.15.170:0/2421206616 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac00eb30 con 0x558553dc6070 2024-03-28T16:49:21.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2421206616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 msgr2=0x558553f3bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2421206616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f3bea0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ffbac001c00 tx=0x7ffbac00b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2421206616 shutdown_connections 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2421206616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f3bea0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2421206616 >> 172.21.15.170:0/2421206616 conn(0x558553e4c6d0 msgr2=0x558553f37a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2421206616 shutdown_connections 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2421206616 wait complete. 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 Processor -- start 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- start start 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f42cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558553d5de00 con 0x558553dc6070 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f42cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f42cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57104/0 (socket says 172.21.15.170:57104) 2024-03-28T16:49:21.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/884946066 learned_addr learned my addr 172.21.15.170:0/884946066 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558553d9e520 con 0x558553dc6070 2024-03-28T16:49:21.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f42cf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00b710 tx=0x7ffbac000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffaa5602640 1 -- 172.21.15.170:0/884946066 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac01f070 con 0x558553dc6070 2024-03-28T16:49:21.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558553f43230 con 0x558553dc6070 2024-03-28T16:49:21.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffaa5602640 1 -- 172.21.15.170:0/884946066 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac01a910 con 0x558553dc6070 2024-03-28T16:49:21.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffaa5602640 1 -- 172.21.15.170:0/884946066 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac01abf0 con 0x558553dc6070 2024-03-28T16:49:21.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.675+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558553f43660 con 0x558553dc6070 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffaa5602640 1 -- 172.21.15.170:0/884946066 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ffbac00a040 con 0x558553dc6070 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffaa5602640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffaa00342b0 0x7ffaa0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffaa5602640 1 -- 172.21.15.170:0/884946066 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7ffbac049e60 con 0x558553dc6070 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffaa00342b0 0x7ffaa0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553e56310 0x558553f49970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f49eb0 con 0x558553e56310 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb960e640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553e56310 0x558553f49970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffaa00342b0 0x7ffaa0036770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffba40ad000 tx=0x7ffba40a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb960e640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553e56310 0x558553f49970 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb960e640 1 -- 172.21.15.170:0/884946066 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553e56310 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f4a210 con 0x558553e56310 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb960e640 1 -- 172.21.15.170:0/884946066 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553e56310 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f4bba0 0x558553f4df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f4e720 con 0x558553f4bba0 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f4bba0 0x558553f4df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f4bba0 0x558553f4df90 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/884946066 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7ffbac024cd0 con 0x558553f4bba0 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.679+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f4aa10 con 0x558553e56310 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb960e640 1 -- 172.21.15.170:0/884946066 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553e56310 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x558553f4fb40 0x558553f51f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f525f0 con 0x558553f4fb40 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x558553f4fb40 0x558553f51f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x558553f4fb40 0x558553f51f30 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb3fff640 1 -- 172.21.15.170:0/884946066 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f4fb40 2024-03-28T16:49:21.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 Processor -- start 2024-03-28T16:49:21.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 -- start start 2024-03-28T16:49:21.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f60520 0x558553f60940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558553cf6e30 con 0x558553f60520 2024-03-28T16:49:21.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f60520 0x558553f60940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f60520 0x558553f60940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57118/0 (socket says 172.21.15.170:57118) 2024-03-28T16:49:21.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/3814044131 learned_addr learned my addr 172.21.15.170:0/3814044131 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/3814044131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558553f60e80 con 0x558553f60520 2024-03-28T16:49:21.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f60520 0x558553f60940 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00efb0 tx=0x7ffbac017ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffa9e7fc640 1 -- 172.21.15.170:0/3814044131 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac0263b0 con 0x558553f60520 2024-03-28T16:49:21.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffa9e7fc640 1 -- 172.21.15.170:0/3814044131 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac026510 con 0x558553f60520 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558553f610d0 con 0x558553f60520 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffa9e7fc640 1 -- 172.21.15.170:0/3814044131 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac0267f0 con 0x558553f60520 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558553f61c20 con 0x558553f60520 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.683+0000 7ffa9e7fc640 1 -- 172.21.15.170:0/3814044131 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ffbac02d5c0 con 0x558553f60520 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffa9e7fc640 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa900342b0 0x7ffa90036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffa9e7fc640 1 -- 172.21.15.170:0/3814044131 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7ffbac04c030 con 0x558553f60520 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa900342b0 0x7ffa90036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 Processor -- start 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- start start 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa900342b0 0x7ffa90036770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x558553f4eab0 tx=0x7ffba4005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558553cea790 con 0x558553f6d340 2024-03-28T16:49:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f6d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57132/0 (socket says 172.21.15.170:57132) 2024-03-28T16:49:21.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/388894829 learned_addr learned my addr 172.21.15.170:0/388894829 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/388894829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558553f6dd30 con 0x558553f6d340 2024-03-28T16:49:21.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/388894829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f6d760 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffbac02cff0 tx=0x7ffbac01df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b4e37a64b2ca4b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffa8ffff640 1 -- 172.21.15.170:0/388894829 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac062070 con 0x558553f6d340 2024-03-28T16:49:21.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffa8ffff640 1 -- 172.21.15.170:0/388894829 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac04b020 con 0x558553f6d340 2024-03-28T16:49:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffa8ffff640 1 -- 172.21.15.170:0/388894829 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac008480 con 0x558553f6d340 2024-03-28T16:49:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/388894829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 msgr2=0x558553f6d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/388894829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f6d760 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffbac02cff0 tx=0x7ffbac01df90 comp rx=0 tx=0).stop 2024-03-28T16:49:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/388894829 shutdown_connections 2024-03-28T16:49:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/388894829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f6d760 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/388894829 >> 172.21.15.170:0/388894829 conn(0x558553f696d0 msgr2=0x558553f67730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/388894829 shutdown_connections 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/388894829 wait complete. 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 Processor -- start 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- start start 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbbb2f2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558553f74710 con 0x558553f6d340 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57138/0 (socket says 172.21.15.170:57138) 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.687+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 learned_addr learned my addr 172.21.15.170:0/2169976672 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558553f60e80 con 0x558553f6d340 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f741d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffbac03a9a0 tx=0x7ffbac04af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffa8e7fc640 1 -- 172.21.15.170:0/2169976672 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac062070 con 0x558553f6d340 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558553f60e80 con 0x558553f6d340 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffa8e7fc640 1 -- 172.21.15.170:0/2169976672 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac06e070 con 0x558553f6d340 2024-03-28T16:49:21.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffa8e7fc640 1 -- 172.21.15.170:0/2169976672 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbac04b190 con 0x558553f6d340 2024-03-28T16:49:21.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558553f60e80 con 0x558553f6d340 2024-03-28T16:49:21.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffa8e7fc640 1 -- 172.21.15.170:0/2169976672 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ffbac03ac40 con 0x558553f6d340 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffa8e7fc640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa840342b0 0x7ffa84036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffa8e7fc640 1 -- 172.21.15.170:0/2169976672 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7ffbac075ae0 con 0x558553f6d340 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa840342b0 0x7ffa84036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553f7f440 0x558553f81900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82050 con 0x558553f7f440 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb960e640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553f7f440 0x558553f81900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa840342b0 0x7ffa84036770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffba40abfd0 tx=0x7ffba40bff90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb960e640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553f7f440 0x558553f81900 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 realm 2024-03-28T16:49:21.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82420 con 0x558553f7f440 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f7f440 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82fd0 con 0x558553f7f440 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.691+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f83980 0x558553f85d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f862f0 con 0x558553f83980 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f83980 0x558553f85d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb8e0d640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f83980 0x558553f85d70 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbbb2f2980 20 searching for the correct realm 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558553f881f0 0x558553f8a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8acf0 con 0x558553f881f0 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558553f881f0 0x558553f8a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558553f881f0 0x558553f8a620 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb3fff640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ffba40cc050 0x7ffba40cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8acf0 con 0x7ffba40cc050 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb960e640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ffba40cc050 0x7ffba40cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb960e640 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ffba40cc050 0x7ffba40cb3b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f83980 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.695+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f7f440 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f7f440 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f83980 2024-03-28T16:49:21.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f83980 2024-03-28T16:49:21.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x7ffba40cc050 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f881f0 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f7f440 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f82c70 con 0x558553f83980 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f7f440 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x7ffba40cc050 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f881f0 2024-03-28T16:49:21.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f881f0 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.699+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f83980 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f881f0 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f881f0 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f7f440 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f7f440 2024-03-28T16:49:21.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f83980 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f881f0 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x7ffba40cc050 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f7f440 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f83980 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f881f0 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.703+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f83980 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x7ffba40cc050 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f881f0 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f83980 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x558553f7f440 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x7ffbac08b040 con 0x7ffba40cc050 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:21.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f86b60 con 0x558553f7f440 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f86b60 con 0x558553f7f440 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f86b60 con 0x7ffba40cc050 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 zone default found 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 4 Realm: () 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 zonegroup default 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f873b0 con 0x558553f83980 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7ffbac064070 con 0x558553f83980 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f877c0 con 0x558553f881f0 2024-03-28T16:49:21.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8cb90 con 0x558553f881f0 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 started sync module instance, tier type = 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.707+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8c3f0 con 0x7ffba40cc050 2024-03-28T16:49:21.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.711+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.711+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94031127822160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8c7a0 con 0x7ffba40cc050 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.711+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8d8f0 con 0x558553f7f440 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.711+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94031127822160] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.711+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.711+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94031127826352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8d930 con 0x558553f7f440 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.711+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8e3d0 con 0x558553f881f0 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94031127826352] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94031127830192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8e730 con 0x558553f881f0 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8f0e0 con 0x7ffba40cc050 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94031127833600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8f930 con 0x7ffba40cc050 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f902e0 con 0x558553f7f440 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94031127830192] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.715+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94031127833600] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.719+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.719+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94031127827408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8f440 con 0x558553f7f440 2024-03-28T16:49:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.719+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f90b30 con 0x558553f83980 2024-03-28T16:49:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.719+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94031127827408] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.719+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.719+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94031127840400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f8f440 con 0x558553f83980 2024-03-28T16:49:21.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.719+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f91330 con 0x558553f83980 2024-03-28T16:49:21.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94031127842448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f91330 con 0x558553f83980 2024-03-28T16:49:21.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f922f0 con 0x558553f83980 2024-03-28T16:49:21.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94031127840400] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94031127842448] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94031127846352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f91510 con 0x558553f83980 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 20 add_watcher() i=0 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 20 add_watcher() i=1 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 20 add_watcher() i=2 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 20 add_watcher() i=3 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 20 add_watcher() i=4 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 20 add_watcher() i=5 2024-03-28T16:49:21.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.723+0000 7ffbbb2f2980 20 add_watcher() i=6 2024-03-28T16:49:21.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94031127846352] v29'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 add_watcher() i=7 2024-03-28T16:49:21.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:21.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f91400 con 0x558553f7f440 2024-03-28T16:49:21.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94031127838208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553f91400 con 0x558553f7f440 2024-03-28T16:49:21.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94031127838208] v29'144 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffa6efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 5 note: GC not initialized 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffa6dfe3640 20 reqs_thread_entry: start 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 Filter name: none 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ff9df7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fb71f0 con 0x7ffba40cc050 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv29 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:63 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fb7760 con 0x558553f881f0 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(63 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.727+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fb6870 con 0x7ffba40cc050 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv29 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fb90c0 con 0x558553f881f0 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(65 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:21.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 4.28 4:1795f074:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fb8b10 con 0x558553f7f440 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 4.22 4:4580d573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fbdad0 con 0x558553f7f440 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 4.2 4:400bbb2d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fb8b10 con 0x558553f7f440 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 4.14 4:2bd00f8d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fbdad0 con 0x558553f881f0 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 4.9 4:93976103:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fc5230 con 0x558553f7f440 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 4.1 4:81996a80:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fc7d40 con 0x7ffba40cc050 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 4.9 4:916ca974:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fcaad0 con 0x558553f7f440 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 4.f 4:f072f27b:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fcd940 con 0x558553f7f440 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:74 4.16 4:69af5b3c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fd0760 con 0x558553f881f0 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 4.29 4:9412bb76:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fd3580 con 0x558553f881f0 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:76 4.15 4:a8ab0eea:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fd63f0 con 0x558553f881f0 2024-03-28T16:49:21.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(66 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(67 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(69 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(70 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(73 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(74 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(71 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.5 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(76 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(75 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(72 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.1.6 [call out=124b] v0'0 uv10 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:77 6.14 6:2bc8c1a7:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fd3580 con 0x7ffba40cc050 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(77 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffa6efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94031127838208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x558553f7f440 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94031127838208] v29'145 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.731+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94031127822160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x7ffba40cc050 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.735+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94031127822160] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.735+0000 7ffbbb2f2980 20 remove_watcher() i=0 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.735+0000 7ffbbb2f2980 2 removed watcher, disabling cache 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.735+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94031127826352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x558553f7f440 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.735+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94031127826352] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.735+0000 7ffbbb2f2980 20 remove_watcher() i=1 2024-03-28T16:49:21.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.735+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94031127830192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x558553f881f0 2024-03-28T16:49:21.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbb3fff640 1 -- 172.21.15.170:0/2169976672 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94031127830192] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffba40abc30 con 0x558553f881f0 2024-03-28T16:49:21.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbbb2f2980 20 remove_watcher() i=2 2024-03-28T16:49:21.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94031127833600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x7ffba40cc050 2024-03-28T16:49:21.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94031127833600] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x7ffba40cc050 2024-03-28T16:49:21.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbbb2f2980 20 remove_watcher() i=3 2024-03-28T16:49:21.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94031127827408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x558553f7f440 2024-03-28T16:49:21.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94031127827408] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbb400a4d0 con 0x558553f7f440 2024-03-28T16:49:21.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbbb2f2980 20 remove_watcher() i=4 2024-03-28T16:49:21.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.739+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94031127840400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x558553f83980 2024-03-28T16:49:21.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.743+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94031127840400] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.743+0000 7ffbbb2f2980 20 remove_watcher() i=5 2024-03-28T16:49:21.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.743+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94031127842448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x558553f83980 2024-03-28T16:49:21.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.743+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94031127842448] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.743+0000 7ffbbb2f2980 20 remove_watcher() i=6 2024-03-28T16:49:21.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.743+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94031127846352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x558553fa8f40 con 0x558553f83980 2024-03-28T16:49:21.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbb8e0d640 1 -- 172.21.15.170:0/2169976672 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94031127846352] v29'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ffbac097040 con 0x558553f83980 2024-03-28T16:49:21.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 20 remove_watcher() i=7 2024-03-28T16:49:21.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f83980 msgr2=0x558553f85d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f83980 0x558553f85d70 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553f7f440 msgr2=0x558553f81900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553f7f440 0x558553f81900 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ffba40cc050 msgr2=0x7ffba40cb3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ffba40cc050 0x7ffba40cb3b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558553f881f0 msgr2=0x558553f8a620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558553f881f0 0x558553f8a620 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa840342b0 msgr2=0x7ffa84036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa840342b0 0x7ffa84036770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffba40abfd0 tx=0x7ffba40bff90 comp rx=0 tx=0).stop 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbb960e640 1 -- 172.21.15.170:0/2169976672 reap_dead start 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 msgr2=0x558553f741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f741d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffbac03a9a0 tx=0x7ffbac04af40 comp rx=0 tx=0).stop 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 shutdown_connections 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/2169976672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f6d340 0x558553f741d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 >> 172.21.15.170:0/2169976672 conn(0x558553f696d0 msgr2=0x558553f66d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 shutdown_connections 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/2169976672 wait complete. 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa900342b0 msgr2=0x7ffa90036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa900342b0 0x7ffa90036770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x558553f4eab0 tx=0x7ffba4005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f60520 msgr2=0x558553f60940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f60520 0x558553f60940 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00efb0 tx=0x7ffbac017ba0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 shutdown_connections 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffa900342b0 0x7ffa90036770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/3814044131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553f60520 0x558553f60940 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 >> 172.21.15.170:0/3814044131 conn(0x558553f55330 msgr2=0x558553f55e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 shutdown_connections 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/3814044131 wait complete. 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f4bba0 msgr2=0x558553f4df90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558553f4bba0 0x558553f4df90 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553e56310 msgr2=0x558553f49970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558553e56310 0x558553f49970 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x558553f4fb40 msgr2=0x558553f51f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x558553f4fb40 0x558553f51f30 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffaa00342b0 msgr2=0x7ffaa0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ffaa00342b0 0x7ffaa0036770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffba40ad000 tx=0x7ffba40a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 msgr2=0x558553f42cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 --2- 172.21.15.170:0/884946066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558553dc6070 0x558553f42cf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00b710 tx=0x7ffbac000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbb960e640 1 -- 172.21.15.170:0/884946066 reap_dead start 2024-03-28T16:49:21.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 shutdown_connections 2024-03-28T16:49:21.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 >> 172.21.15.170:0/884946066 conn(0x558553e4c6d0 msgr2=0x558553f36540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 shutdown_connections 2024-03-28T16:49:21.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.747+0000 7ffbbb2f2980 1 -- 172.21.15.170:0/884946066 wait complete. 2024-03-28T16:49:21.757 INFO:teuthology.orchestra.run.smithi170.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"c7513231-bef6-43ae-ade6-ce57a95c9845","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","marker":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-03-28T16:49:21.066626Z","creation_time":"2024-03-28T16:49:17.311675Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":0,"size_actual":0,"size_utilized":0,"size_kb":0,"size_kb_actual":0,"size_kb_utilized":0,"num_objects":0}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"notifications":[]} 2024-03-28T16:49:21.757 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'c7513231-bef6-43ae-ade6-ce57a95c9845', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'marker': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-03-28T16:49:21.066626Z', 'creation_time': '2024-03-28T16:49:17.311675Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'notifications': []} 2024-03-28T16:49:21.757 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-03-28T16:49:21.757 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-03-28T16:49:21.757 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-03-28T16:49:21.831 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:21.831 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:21.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f1840de4980 1 Processor -- start 2024-03-28T16:49:21.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f1840de4980 1 -- start start 2024-03-28T16:49:21.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f1840de4980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5622c9d0c620 0x5622c9d0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f1840de4980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5622c9c52510 con 0x5622c9d08120 2024-03-28T16:49:21.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f1840de4980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5622c9c190f0 con 0x5622c9d0c620 2024-03-28T16:49:21.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f183e0fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5622c9d0c620 0x5622c9d0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f183e8ff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5622c9d08120 0x5622c9d08500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38340/0 (socket says 172.21.15.170:38340) 2024-03-28T16:49:21.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.831+0000 7f183e8ff640 1 -- 172.21.15.170:0/3867632065 learned_addr learned my addr 172.21.15.170:0/3867632065 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:21.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.835+0000 7f183e0fe640 1 -- 172.21.15.170:0/3867632065 >> v1:172.21.15.119:6789/0 conn(0x5622c9d08120 legacy=0x5622c9d08500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.835+0000 7f183e0fe640 1 -- 172.21.15.170:0/3867632065 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c9d0ff60 con 0x5622c9d0c620 2024-03-28T16:49:21.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f183e0fe640 1 --2- 172.21.15.170:0/3867632065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d0c620 0x5622c9d0ea10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f183400a480 tx=0x7f183401b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c526d9d75eb757dd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f183d8fd640 1 -- 172.21.15.170:0/3867632065 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18340ab540 con 0x5622c9d0c620 2024-03-28T16:49:21.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f183d8fd640 1 -- 172.21.15.170:0/3867632065 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18340ab6a0 con 0x5622c9d0c620 2024-03-28T16:49:21.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f183d8fd640 1 -- 172.21.15.170:0/3867632065 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18340ab960 con 0x5622c9d0c620 2024-03-28T16:49:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f1840de4980 1 -- 172.21.15.170:0/3867632065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d0c620 msgr2=0x5622c9d0ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f1840de4980 1 --2- 172.21.15.170:0/3867632065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d0c620 0x5622c9d0ea10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f183400a480 tx=0x7f183401b280 comp rx=0 tx=0).stop 2024-03-28T16:49:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f1840de4980 1 -- 172.21.15.170:0/3867632065 shutdown_connections 2024-03-28T16:49:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f1840de4980 1 --2- 172.21.15.170:0/3867632065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d0c620 0x5622c9d0ea10 secure :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f183400a480 tx=0x7f183401b280 comp rx=0 tx=0).stop 2024-03-28T16:49:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f1840de4980 1 -- 172.21.15.170:0/3867632065 >> 172.21.15.170:0/3867632065 conn(0x5622c9c7b840 msgr2=0x5622c9c7bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f1840de4980 1 -- 172.21.15.170:0/3867632065 shutdown_connections 2024-03-28T16:49:21.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.839+0000 7f1840de4980 1 -- 172.21.15.170:0/3867632065 wait complete. 2024-03-28T16:49:21.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.851+0000 7f1840de4980 1 Processor -- start 2024-03-28T16:49:21.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.851+0000 7f1840de4980 1 -- start start 2024-03-28T16:49:21.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.851+0000 7f1840de4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.851+0000 7f1840de4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c9c16620 con 0x5622c9d08120 2024-03-28T16:49:21.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.851+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.851+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57166/0 (socket says 172.21.15.170:57166) 2024-03-28T16:49:21.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.851+0000 7f183e8ff640 1 -- 172.21.15.170:0/4010216631 learned_addr learned my addr 172.21.15.170:0/4010216631 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f183e8ff640 1 -- 172.21.15.170:0/4010216631 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834005d30 con 0x5622c9d08120 2024-03-28T16:49:21.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f183e8ff640 1 --2- 172.21.15.170:0/4010216631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df21c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1830001c90 tx=0x7f183000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eab1315884fcf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f172ce09640 1 -- 172.21.15.170:0/4010216631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183000e690 con 0x5622c9d08120 2024-03-28T16:49:21.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f172ce09640 1 -- 172.21.15.170:0/4010216631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183000e7f0 con 0x5622c9d08120 2024-03-28T16:49:21.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f172ce09640 1 -- 172.21.15.170:0/4010216631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183000eaf0 con 0x5622c9d08120 2024-03-28T16:49:21.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- 172.21.15.170:0/4010216631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 msgr2=0x5622c9df21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 --2- 172.21.15.170:0/4010216631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df21c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1830001c90 tx=0x7f183000b6d0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- 172.21.15.170:0/4010216631 shutdown_connections 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 --2- 172.21.15.170:0/4010216631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df21c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- 172.21.15.170:0/4010216631 >> 172.21.15.170:0/4010216631 conn(0x5622c9c7b840 msgr2=0x5622c9dedcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- 172.21.15.170:0/4010216631 shutdown_connections 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- 172.21.15.170:0/4010216631 wait complete. 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 Processor -- start 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- start start 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c9c11e00 con 0x5622c9d08120 2024-03-28T16:49:21.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57178/0 (socket says 172.21.15.170:57178) 2024-03-28T16:49:21.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f183e8ff640 1 -- 172.21.15.170:0/10745456 learned_addr learned my addr 172.21.15.170:0/10745456 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f183e8ff640 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c9c52510 con 0x5622c9d08120 2024-03-28T16:49:21.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f183e8ff640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df9010 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1830001ee0 tx=0x7f1830000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f172b606640 1 -- 172.21.15.170:0/10745456 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183001f070 con 0x5622c9d08120 2024-03-28T16:49:21.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5622c9df9550 con 0x5622c9d08120 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f172b606640 1 -- 172.21.15.170:0/10745456 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183001a4d0 con 0x5622c9d08120 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f172b606640 1 -- 172.21.15.170:0/10745456 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183001a770 con 0x5622c9d08120 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.855+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c9df9980 con 0x5622c9d08120 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f172b606640 1 -- 172.21.15.170:0/10745456 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f183000a040 con 0x5622c9d08120 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f172b606640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17240342b0 0x7f1724036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f172b606640 1 -- 172.21.15.170:0/10745456 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1830049570 con 0x5622c9d08120 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183e0fe640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17240342b0 0x7f1724036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9dfdc80 0x5622c9e00070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e005b0 con 0x5622c9dfdc80 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183f100640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9dfdc80 0x5622c9e00070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183e0fe640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17240342b0 0x7f1724036770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18340ad000 tx=0x7f18340a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183f100640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9dfdc80 0x5622c9e00070 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183f100640 1 -- 172.21.15.170:0/10745456 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9dfdc80 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e00910 con 0x5622c9dfdc80 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183f100640 1 -- 172.21.15.170:0/10745456 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9dfdc80 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e022a0 0x5622c9e046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e04e60 con 0x5622c9e022a0 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183e8ff640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e022a0 0x5622c9e046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183e8ff640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e022a0 0x5622c9e046d0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f183e8ff640 1 -- 172.21.15.170:0/10745456 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f1830014070 con 0x5622c9e022a0 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.859+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e01110 con 0x5622c9dfdc80 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183f100640 1 -- 172.21.15.170:0/10745456 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9dfdc80 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5622c9e06280 0x5622c9e08670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e08d30 con 0x5622c9e06280 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e0fe640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5622c9e06280 0x5622c9e08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e0fe640 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5622c9e06280 0x5622c9e08670 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e0fe640 1 -- 172.21.15.170:0/10745456 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e06280 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 Processor -- start 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 -- start start 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e16c60 0x5622c9e17080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c9baae30 con 0x5622c9e16c60 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e16c60 0x5622c9e17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e16c60 0x5622c9e17080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57194/0 (socket says 172.21.15.170:57194) 2024-03-28T16:49:21.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e8ff640 1 -- 172.21.15.170:0/2597125115 learned_addr learned my addr 172.21.15.170:0/2597125115 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e8ff640 1 -- 172.21.15.170:0/2597125115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c9e175c0 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f183e8ff640 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e16c60 0x5622c9e17080 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1830047ab0 tx=0x7f18300494a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1728e01640 1 -- 172.21.15.170:0/2597125115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183001ab10 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1728e01640 1 -- 172.21.15.170:0/2597125115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1830033040 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5622c9e17810 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1728e01640 1 -- 172.21.15.170:0/2597125115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183001ada0 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.863+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c9e18360 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1728e01640 1 -- 172.21.15.170:0/2597125115 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f1830041320 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1728e01640 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17180342b0 0x7f1718036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1728e01640 1 -- 172.21.15.170:0/2597125115 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1830019030 con 0x5622c9e16c60 2024-03-28T16:49:21.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e0fe640 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17180342b0 0x7f1718036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 Processor -- start 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- start start 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e0fe640 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17180342b0 0x7f1718036770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5622c9e051f0 tx=0x7f1834005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e23fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c9b9e790 con 0x5622c9e23be0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e23fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e23fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57196/0 (socket says 172.21.15.170:57196) 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 -- 172.21.15.170:0/4270763401 learned_addr learned my addr 172.21.15.170:0/4270763401 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 -- 172.21.15.170:0/4270763401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c9e245b0 con 0x5622c9e23be0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 --2- 172.21.15.170:0/4270763401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e23fe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1830016cb0 tx=0x7f1830039bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efbe6ca5beab778 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f17227fc640 1 -- 172.21.15.170:0/4270763401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183002f0b0 con 0x5622c9e23be0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f17227fc640 1 -- 172.21.15.170:0/4270763401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183002c3a0 con 0x5622c9e23be0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f17227fc640 1 -- 172.21.15.170:0/4270763401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1830023ad0 con 0x5622c9e23be0 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- 172.21.15.170:0/4270763401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 msgr2=0x5622c9e23fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 --2- 172.21.15.170:0/4270763401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e23fe0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1830016cb0 tx=0x7f1830039bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- 172.21.15.170:0/4270763401 shutdown_connections 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 --2- 172.21.15.170:0/4270763401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e23fe0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- 172.21.15.170:0/4270763401 >> 172.21.15.170:0/4270763401 conn(0x5622c9e1ff20 msgr2=0x5622c9e1df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- 172.21.15.170:0/4270763401 shutdown_connections 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- 172.21.15.170:0/4270763401 wait complete. 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 Processor -- start 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- start start 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f1840de4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c9e2afb0 con 0x5622c9e23be0 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e2aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57200/0 (socket says 172.21.15.170:57200) 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.867+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 learned_addr learned my addr 172.21.15.170:0/3358872789 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:21.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c9e18360 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183e8ff640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e2aa70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1830039fd0 tx=0x7f1830021f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1720ff9640 1 -- 172.21.15.170:0/3358872789 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f183006a070 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1720ff9640 1 -- 172.21.15.170:0/3358872789 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1830018e80 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5622c9e18360 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1720ff9640 1 -- 172.21.15.170:0/3358872789 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1830039480 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c9e18360 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1720ff9640 1 -- 172.21.15.170:0/3358872789 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f18300233c0 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1720ff9640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17080342b0 0x7f1708036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1720ff9640 1 -- 172.21.15.170:0/3358872789 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1830062040 con 0x5622c9e23be0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183e0fe640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17080342b0 0x7f1708036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9e35cd0 0x5622c9e38190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e388e0 con 0x5622c9e35cd0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183f100640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9e35cd0 0x5622c9e38190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183e0fe640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17080342b0 0x7f1708036770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18340ab670 tx=0x7f18340a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183f100640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9e35cd0 0x5622c9e38190 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 realm 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e38cb0 con 0x5622c9e35cd0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39860 con 0x5622c9e35cd0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e3a210 0x5622c9e3c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e3cb80 con 0x5622c9e3a210 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.871+0000 7f183e8ff640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e3a210 0x5622c9e3c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e8ff640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e3a210 0x5622c9e3c600 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f1840de4980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f1840de4980 20 searching for the correct realm 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5622c9e3ea80 0x5622c9e40e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e41560 con 0x5622c9e3ea80 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e0fe640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5622c9e3ea80 0x5622c9e40e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e0fe640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5622c9e3ea80 0x5622c9e40e90 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e0fe640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f18340da040 0x7f18340cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e41560 con 0x7f18340da040 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f18340da040 0x7f18340cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f18340da040 0x7f18340cc420 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.875+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.879+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.883+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e3d3f0 con 0x5622c9e35cd0 2024-03-28T16:49:21.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e3d3c0 con 0x5622c9e35cd0 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e3d3c0 con 0x7f18340da040 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 zone default found 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 4 Realm: () 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:21.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 zonegroup default 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e3dc40 con 0x5622c9e3a210 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.887+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e3e040 con 0x5622c9e3ea80 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:21.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e43380 con 0x5622c9e3ea80 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 20 started sync module instance, tier type = 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e42c60 con 0x7f18340da040 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94707416049600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e43010 con 0x7f18340da040 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e44110 con 0x5622c9e35cd0 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94707416049600] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94707416053712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e440d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.891+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e44b90 con 0x5622c9e3ea80 2024-03-28T16:49:21.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94707416057424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e453e0 con 0x5622c9e3ea80 2024-03-28T16:49:21.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e45dd0 con 0x7f18340da040 2024-03-28T16:49:21.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94707416053712] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94707416062128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e46180 con 0x7f18340da040 2024-03-28T16:49:21.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e46af0 con 0x5622c9e35cd0 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.895+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94707416057424] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94707416062128] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94707416054768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e46650 con 0x5622c9e35cd0 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e47340 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94707416054768] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94707416067744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e46650 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.899+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e47b40 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94707416067744] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94707416059712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e48340 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e48cb0 con 0x5622c9e3a210 2024-03-28T16:49:21.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94707416059712] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94707416072112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e48cb0 con 0x5622c9e3a210 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 20 add_watcher() i=0 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 20 add_watcher() i=1 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 20 add_watcher() i=2 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 20 add_watcher() i=3 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 20 add_watcher() i=4 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 20 add_watcher() i=5 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.903+0000 7f1840de4980 20 add_watcher() i=6 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94707416072112] v29'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 20 add_watcher() i=7 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:21.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e4c070 con 0x5622c9e35cd0 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94707416074720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e49550 con 0x5622c9e35cd0 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94707416074720] v29'146 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f16f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 5 note: GC not initialized 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f166ffff640 20 reqs_thread_entry: start 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 20 Filter name: none 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f166dffb640 20 reqs_thread_entry: start 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.907+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 6.0 6:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e6d7a0 con 0x5622c9e3a210 2024-03-28T16:49:21.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 [pgnls start_epoch 29 out=254b] v29'35 uv18 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.10 6:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e47b40 con 0x5622c9e3a210 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(63 [pgnls start_epoch 29 out=131b] v29'4 uv4 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x7f183006c070 con 0x5622c9e3a210 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.8 6:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e47b40 con 0x5622c9e3ea80 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 [pgnls start_epoch 29 out=295b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 6.18 6:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e47b40 con 0x5622c9e3a210 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(65 [pgnls start_epoch 29 out=172b] v29'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f18300820c0 con 0x5622c9e3a210 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 6.4 6:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e47b40 con 0x5622c9e35cd0 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(66 [pgnls start_epoch 29 out=335b] v29'146 uv23 ondisk = 1) v8 ==== 144+0+335 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 6.14 6:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(67 [pgnls start_epoch 29 out=213b] v29'10 uv10 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 6.c 6:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(68 [pgnls start_epoch 29 out=254b] v29'10 uv10 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 6.1c 6:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(69 [pgnls start_epoch 29 out=295b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:70 6.2 6:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(70 [pgnls start_epoch 29 out=213b] v29'10 uv10 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.911+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 6.12 6:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(71 [pgnls start_epoch 29 out=172b] v29'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.a 6:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 [pgnls start_epoch 29 out=336b] v29'27 uv19 ondisk = 1) v8 ==== 144+0+336 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 6.1a 6:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(73 [pgnls start_epoch 29 out=131b] v29'17 uv9 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:74 6.6 6:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(74 [pgnls start_epoch 29 out=254b] v29'12 uv12 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 6.16 6:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(75 [pgnls start_epoch 29 out=90b] v29'17 uv9 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.e 6:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(76 [pgnls start_epoch 29 out=213b] v29'19 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.1e 6:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(77 [pgnls start_epoch 29 out=254b] v29'10 uv10 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f18340abb10 con 0x5622c9e3ea80 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:78 6.1 6:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(78 [pgnls start_epoch 29 out=254b] v29'14 uv14 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.11 6:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(79 [pgnls start_epoch 29 out=213b] v29'20 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f18340f8040 con 0x5622c9e3ea80 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:80 6.9 6:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(80 [pgnls start_epoch 29 out=172b] v29'28 uv12 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f18340f8040 con 0x5622c9e3ea80 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.19 6:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(81 [pgnls start_epoch 29 out=295b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:82 6.5 6:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(82 [pgnls start_epoch 29 out=276b] v29'15 uv15 ondisk = 1) v8 ==== 144+0+276 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.15 6:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(83 [pgnls start_epoch 29 out=172b] v29'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:84 6.d 6:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(84 [pgnls start_epoch 29 out=295b] v29'25 uv17 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.915+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:85 6.1d 6:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(85 [pgnls start_epoch 29 out=172b] v29'28 uv12 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f18300820c0 con 0x5622c9e3a210 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:86 6.3 6:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(86 [pgnls start_epoch 29 out=213b] v29'43 uv19 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f183800a4d0 con 0x7f18340da040 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:87 6.13 6:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(87 [pgnls start_epoch 29 out=172b] v29'19 uv11 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f18300820c0 con 0x5622c9e3a210 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:88 6.b 6:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(88 [pgnls start_epoch 29 out=172b] v29'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x7f18340f8040 con 0x5622c9e3ea80 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1b 6:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(89 [pgnls start_epoch 29 out=90b] v29'13 uv5 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x7f18340fe040 con 0x5622c9e3ea80 2024-03-28T16:49:21.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:90 6.7 6:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3ea80 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(90 [pgnls start_epoch 29 out=90b] v29'37 uv13 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x7f18340fe040 con 0x5622c9e3ea80 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:91 6.17 6:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e35cd0 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(91 [pgnls start_epoch 29 out=131b] v29'7 uv6 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x7f183800a4d0 con 0x5622c9e35cd0 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:92 6.f 6:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x7f18340da040 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(92 [pgnls start_epoch 29 out=295b] v29'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x7f183805e130 con 0x7f18340da040 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.919+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:93 6.1f 6:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e39500 con 0x5622c9e3a210 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(93 [pgnls start_epoch 29 out=213b] v29'19 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x7f18300820c0 con 0x5622c9e3a210 2024-03-28T16:49:21.935 INFO:teuthology.orchestra.run.smithi170.stdout:["obj_delete_at_hint.0000000078","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","data_loggenerations_metadata","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","obj_delete_at_hint.0000000027","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","data_log.0","obj_delete_at_hint.0000000012","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f16f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94707416074720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e762c0 con 0x5622c9e35cd0 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94707416074720] v29'147 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f183805e130 con 0x5622c9e35cd0 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94707416049600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e75bf0 con 0x7f18340da040 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94707416049600] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183805e130 con 0x7f18340da040 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f1840de4980 20 remove_watcher() i=0 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f1840de4980 2 removed watcher, disabling cache 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.923+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94707416053712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e427c0 con 0x5622c9e35cd0 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.927+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94707416053712] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183805e130 con 0x5622c9e35cd0 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.927+0000 7f1840de4980 20 remove_watcher() i=1 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.927+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94707416057424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e427c0 con 0x5622c9e3ea80 2024-03-28T16:49:21.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.927+0000 7f183e0fe640 1 -- 172.21.15.170:0/3358872789 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94707416057424] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f18340fe040 con 0x5622c9e3ea80 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.927+0000 7f1840de4980 20 remove_watcher() i=2 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.927+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94707416062128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e44650 con 0x7f18340da040 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.931+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94707416062128] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183805e130 con 0x7f18340da040 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.931+0000 7f1840de4980 20 remove_watcher() i=3 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.931+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94707416054768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e44650 con 0x5622c9e35cd0 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.931+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94707416054768] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f183805e130 con 0x5622c9e35cd0 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.931+0000 7f1840de4980 20 remove_watcher() i=4 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.931+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94707416067744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e44650 con 0x5622c9e3a210 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.935+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94707416067744] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f18300820c0 con 0x5622c9e3a210 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.935+0000 7f1840de4980 20 remove_watcher() i=5 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.935+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94707416059712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e44650 con 0x5622c9e3a210 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.935+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94707416059712] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f18300820c0 con 0x5622c9e3a210 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.935+0000 7f1840de4980 20 remove_watcher() i=6 2024-03-28T16:49:21.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.935+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94707416072112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5622c9e44f40 con 0x5622c9e3a210 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f183e8ff640 1 -- 172.21.15.170:0/3358872789 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94707416072112] v29'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f18300820c0 con 0x5622c9e3a210 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 20 remove_watcher() i=7 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e3a210 msgr2=0x5622c9e3c600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e3a210 0x5622c9e3c600 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9e35cd0 msgr2=0x5622c9e38190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9e35cd0 0x5622c9e38190 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f18340da040 msgr2=0x7f18340cc420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f18340da040 0x7f18340cc420 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5622c9e3ea80 msgr2=0x5622c9e40e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5622c9e3ea80 0x5622c9e40e90 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17080342b0 msgr2=0x7f1708036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17080342b0 0x7f1708036770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18340ab670 tx=0x7f18340a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 msgr2=0x5622c9e2aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/3358872789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e23be0 0x5622c9e2aa70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1830039fd0 tx=0x7f1830021f50 comp rx=0 tx=0).stop 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f183f100640 1 -- 172.21.15.170:0/3358872789 reap_dead start 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 shutdown_connections 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 >> 172.21.15.170:0/3358872789 conn(0x5622c9e1ff20 msgr2=0x5622c9e1d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 shutdown_connections 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/3358872789 wait complete. 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17180342b0 msgr2=0x7f1718036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17180342b0 0x7f1718036770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5622c9e051f0 tx=0x7f1834005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e16c60 msgr2=0x5622c9e17080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e16c60 0x5622c9e17080 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1830047ab0 tx=0x7f18300494a0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 shutdown_connections 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17180342b0 0x7f1718036770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/2597125115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9e16c60 0x5622c9e17080 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 >> 172.21.15.170:0/2597125115 conn(0x5622c9e0ba70 msgr2=0x5622c9e0c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 shutdown_connections 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/2597125115 wait complete. 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e022a0 msgr2=0x5622c9e046d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5622c9e022a0 0x5622c9e046d0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9dfdc80 msgr2=0x5622c9e00070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5622c9dfdc80 0x5622c9e00070 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5622c9e06280 msgr2=0x5622c9e08670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5622c9e06280 0x5622c9e08670 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17240342b0 msgr2=0x7f1724036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f17240342b0 0x7f1724036770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18340ad000 tx=0x7f18340a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 msgr2=0x5622c9df9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 --2- 172.21.15.170:0/10745456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5622c9d08120 0x5622c9df9010 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1830001ee0 tx=0x7f1830000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f183f100640 1 -- 172.21.15.170:0/10745456 reap_dead start 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 shutdown_connections 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 >> 172.21.15.170:0/10745456 conn(0x5622c9c7b840 msgr2=0x5622c9dec7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:21.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 shutdown_connections 2024-03-28T16:49:21.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:21.939+0000 7f1840de4980 1 -- 172.21.15.170:0/10745456 wait complete. 2024-03-28T16:49:21.948 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-03-28T16:49:21.948 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-03-28T16:49:21.949 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-03-28T16:49:21.949 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-03-28T16:49:22.024 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:22.024 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:22.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.023+0000 7fa3ab2bb980 1 Processor -- start 2024-03-28T16:49:22.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.023+0000 7fa3ab2bb980 1 -- start start 2024-03-28T16:49:22.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.023+0000 7fa3ab2bb980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5602cb5f1440 0x5602cb5f3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.023+0000 7fa3ab2bb980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5602cb539520 con 0x5602cb5ecf80 2024-03-28T16:49:22.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.023+0000 7fa3ab2bb980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5602cb5000f0 con 0x5602cb5f1440 2024-03-28T16:49:22.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a85d5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5602cb5f1440 0x5602cb5f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a85d5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5602cb5f1440 0x5602cb5f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57208/0 (socket says 172.21.15.170:57208) 2024-03-28T16:49:22.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a85d5640 1 -- 172.21.15.170:0/1273057835 learned_addr learned my addr 172.21.15.170:0/1273057835 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a85d5640 1 -- 172.21.15.170:0/1273057835 >> v1:172.21.15.119:6789/0 conn(0x5602cb5ecf80 legacy=0x5602cb5ed360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a85d5640 1 -- 172.21.15.170:0/1273057835 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602cb5f4d80 con 0x5602cb5f1440 2024-03-28T16:49:22.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/1273057835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5f1440 0x5602cb5f3830 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00a5c0 tx=0x7fa39c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd20b73ae95146b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a7dd4640 1 -- 172.21.15.170:0/1273057835 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa39c0aa470 con 0x5602cb5f1440 2024-03-28T16:49:22.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a7dd4640 1 -- 172.21.15.170:0/1273057835 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c0aa5d0 con 0x5602cb5f1440 2024-03-28T16:49:22.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.027+0000 7fa3a7dd4640 1 -- 172.21.15.170:0/1273057835 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa39c0aa890 con 0x5602cb5f1440 2024-03-28T16:49:22.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.031+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1273057835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5f1440 msgr2=0x5602cb5f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.031+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/1273057835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5f1440 0x5602cb5f3830 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00a5c0 tx=0x7fa39c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:22.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.031+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1273057835 shutdown_connections 2024-03-28T16:49:22.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.031+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/1273057835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5f1440 0x5602cb5f3830 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.031+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1273057835 >> 172.21.15.170:0/1273057835 conn(0x5602cb5e8850 msgr2=0x5602cb5eac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.031+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1273057835 shutdown_connections 2024-03-28T16:49:22.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.031+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1273057835 wait complete. 2024-03-28T16:49:22.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3ab2bb980 1 Processor -- start 2024-03-28T16:49:22.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3ab2bb980 1 -- start start 2024-03-28T16:49:22.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3ab2bb980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3ab2bb980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602cb4fd620 con 0x5602cb5ecf80 2024-03-28T16:49:22.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6d6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57222/0 (socket says 172.21.15.170:57222) 2024-03-28T16:49:22.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/2264423839 learned_addr learned my addr 172.21.15.170:0/2264423839 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/2264423839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c005df0 con 0x5602cb5ecf80 2024-03-28T16:49:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/2264423839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6d6df0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa398001d60 tx=0x7fa39800b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf484f46abafd42 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.043+0000 7fa296e0d640 1 -- 172.21.15.170:0/2264423839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa398014070 con 0x5602cb5ecf80 2024-03-28T16:49:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa296e0d640 1 -- 172.21.15.170:0/2264423839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39800f540 con 0x5602cb5ecf80 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa296e0d640 1 -- 172.21.15.170:0/2264423839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa39800f820 con 0x5602cb5ecf80 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2264423839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 msgr2=0x5602cb6d6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/2264423839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6d6df0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa398001d60 tx=0x7fa39800b5c0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2264423839 shutdown_connections 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/2264423839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6d6df0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2264423839 >> 172.21.15.170:0/2264423839 conn(0x5602cb5e8850 msgr2=0x5602cb6d2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2264423839 shutdown_connections 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2264423839 wait complete. 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 Processor -- start 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- start start 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6ddc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602cb4f8e00 con 0x5602cb5ecf80 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6ddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6ddc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57228/0 (socket says 172.21.15.170:57228) 2024-03-28T16:49:22.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/829884711 learned_addr learned my addr 172.21.15.170:0/829884711 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602cb539520 con 0x5602cb5ecf80 2024-03-28T16:49:22.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6ddc00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa39800f510 tx=0x7fa398000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa29560a640 1 -- 172.21.15.170:0/829884711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa398014050 con 0x5602cb5ecf80 2024-03-28T16:49:22.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa29560a640 1 -- 172.21.15.170:0/829884711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa398001300 con 0x5602cb5ecf80 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa29560a640 1 -- 172.21.15.170:0/829884711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa39801bb20 con 0x5602cb5ecf80 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5602cb6de140 con 0x5602cb5ecf80 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.047+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602cb6de5d0 con 0x5602cb5ecf80 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa29560a640 1 -- 172.21.15.170:0/829884711 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa398022020 con 0x5602cb5ecf80 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa29560a640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa290034300 0x7fa2900367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa29560a640 1 -- 172.21.15.170:0/829884711 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fa398048c80 con 0x5602cb5ecf80 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa290034300 0x7fa2900367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb5f1440 0x5602cb6e4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb6e4dc0 con 0x5602cb5f1440 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a95d7640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb5f1440 0x5602cb6e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa290034300 0x7fa2900367c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0ae000 tx=0x7fa39c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a95d7640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb5f1440 0x5602cb6e4880 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a95d7640 1 -- 172.21.15.170:0/829884711 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb5f1440 2024-03-28T16:49:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb6e5120 con 0x5602cb5f1440 2024-03-28T16:49:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a95d7640 1 -- 172.21.15.170:0/829884711 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb5f1440 2024-03-28T16:49:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb6e6e90 0x5602cb6e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb6e9800 con 0x5602cb6e6e90 2024-03-28T16:49:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb6e6e90 0x5602cb6e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb6e6e90 0x5602cb6e9280 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/829884711 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fa398016070 con 0x5602cb6e6e90 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb6e5cd0 con 0x5602cb5f1440 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3a95d7640 1 -- 172.21.15.170:0/829884711 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb5f1440 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5602cb6eac00 0x5602cb6ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.051+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb6ed690 con 0x5602cb6eac00 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5602cb6eac00 0x5602cb6ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5602cb6eac00 0x5602cb6ecff0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a85d5640 1 -- 172.21.15.170:0/829884711 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb6eac00 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3ab2bb980 1 Processor -- start 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3ab2bb980 1 -- start start 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3ab2bb980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb6fb560 0x5602cb6fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3ab2bb980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602cb491e30 con 0x5602cb6fb560 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb6fb560 0x5602cb6fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb6fb560 0x5602cb6fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57244/0 (socket says 172.21.15.170:57244) 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/1421334453 learned_addr learned my addr 172.21.15.170:0/1421334453 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/1421334453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602cb6fbec0 con 0x5602cb6fb560 2024-03-28T16:49:22.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb6fb560 0x5602cb6fb980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa39800ee50 tx=0x7fa398018760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa28e7fc640 1 -- 172.21.15.170:0/1421334453 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa398019dc0 con 0x5602cb6fb560 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5602cb6fc110 con 0x5602cb6fb560 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa28e7fc640 1 -- 172.21.15.170:0/1421334453 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa398026920 con 0x5602cb6fb560 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa28e7fc640 1 -- 172.21.15.170:0/1421334453 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa398026be0 con 0x5602cb6fb560 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.055+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602cb6fcc60 con 0x5602cb6fb560 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa28e7fc640 1 -- 172.21.15.170:0/1421334453 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa398026d40 con 0x5602cb6fb560 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa28e7fc640 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa280034260 0x7fa280036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa28e7fc640 1 -- 172.21.15.170:0/1421334453 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fa39803daa0 con 0x5602cb6fb560 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa280034260 0x7fa280036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 Processor -- start 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- start start 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa280034260 0x7fa280036720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5602cb6e6030 tx=0x7fa39c005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb563990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602cb485790 con 0x5602cb563570 2024-03-28T16:49:22.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb563990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb563990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57252/0 (socket says 172.21.15.170:57252) 2024-03-28T16:49:22.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/2087110321 learned_addr learned my addr 172.21.15.170:0/2087110321 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/2087110321 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602cb561e00 con 0x5602cb563570 2024-03-28T16:49:22.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/2087110321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb563990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa398010fd0 tx=0x7fa398025b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efef397645920801 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa27ffff640 1 -- 172.21.15.170:0/2087110321 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa398058070 con 0x5602cb563570 2024-03-28T16:49:22.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa27ffff640 1 -- 172.21.15.170:0/2087110321 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3980120d0 con 0x5602cb563570 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa27ffff640 1 -- 172.21.15.170:0/2087110321 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa39802f450 con 0x5602cb563570 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2087110321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 msgr2=0x5602cb563990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/2087110321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb563990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa398010fd0 tx=0x7fa398025b70 comp rx=0 tx=0).stop 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2087110321 shutdown_connections 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/2087110321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb563990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2087110321 >> 172.21.15.170:0/2087110321 conn(0x5602cb704620 msgr2=0x5602cb702680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2087110321 shutdown_connections 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/2087110321 wait complete. 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 Processor -- start 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- start start 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb711330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3ab2bb980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602cb711870 con 0x5602cb563570 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb711330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb711330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57262/0 (socket says 172.21.15.170:57262) 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.059+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 learned_addr learned my addr 172.21.15.170:0/95721042 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39804c020 con 0x5602cb563570 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb711330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa398036660 tx=0x7fa39800af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa27e7fc640 1 -- 172.21.15.170:0/95721042 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa398025cb0 con 0x5602cb563570 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5602cb711a70 con 0x5602cb563570 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa27e7fc640 1 -- 172.21.15.170:0/95721042 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39802e3e0 con 0x5602cb563570 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602cb711f00 con 0x5602cb563570 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa27e7fc640 1 -- 172.21.15.170:0/95721042 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa398010070 con 0x5602cb563570 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa27e7fc640 1 -- 172.21.15.170:0/95721042 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa3980101d0 con 0x5602cb563570 2024-03-28T16:49:22.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa27e7fc640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa2740342b0 0x7fa274036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa27e7fc640 1 -- 172.21.15.170:0/95721042 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fa39804c810 con 0x5602cb563570 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa2740342b0 0x7fa274036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb71c710 0x5602cb71ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71f320 con 0x5602cb71c710 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a95d7640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb71c710 0x5602cb71ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa2740342b0 0x7fa274036770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa39c000c10 tx=0x7fa39c0c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a95d7640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb71c710 0x5602cb71ebd0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 realm 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71f6f0 con 0x5602cb71c710 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb7202d0 con 0x5602cb71c710 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb720c80 0x5602cb723070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.063+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb723680 con 0x5602cb720c80 2024-03-28T16:49:22.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb720c80 0x5602cb723070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a8dd6640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb720c80 0x5602cb723070 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3ab2bb980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:22.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3ab2bb980 20 searching for the correct realm 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5602cb725860 0x5602cb727d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb7283f0 con 0x5602cb725860 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5602cb725860 0x5602cb727d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5602cb725860 0x5602cb727d20 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a85d5640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa39c0e0040 0x7fa39c0d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb7283f0 con 0x7fa39c0e0040 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a95d7640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa39c0e0040 0x7fa39c0d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a95d7640 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa39c0e0040 0x7fa39c0d3400 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.067+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x7fa39c0e0040 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x7fa39c0e0040 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.071+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x7fa39c0e0040 2024-03-28T16:49:22.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.075+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x7fa39c0e0040 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb725860 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb720c80 2024-03-28T16:49:22.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x5602cb71c710 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb71ff70 con 0x7fa39c0e0040 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb723c60 con 0x5602cb71c710 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb723c60 con 0x5602cb71c710 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb723c30 con 0x7fa39c0e0040 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 zone default found 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 4 Realm: () 2024-03-28T16:49:22.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 zonegroup default 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb7244b0 con 0x5602cb720c80 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.079+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb7248b0 con 0x5602cb725860 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72a330 con 0x5602cb725860 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 20 started sync module instance, tier type = 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb729af0 con 0x7fa39c0e0040 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94570003207760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb729ea0 con 0x7fa39c0e0040 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72b090 con 0x5602cb71c710 2024-03-28T16:49:22.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94570003207760] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94570003212112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72b010 con 0x5602cb71c710 2024-03-28T16:49:22.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.083+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72bac0 con 0x5602cb725860 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94570003215808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72c310 con 0x5602cb725860 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72ccc0 con 0x7fa39c0e0040 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94570003212112] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94570003213168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72d070 con 0x7fa39c0e0040 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72d9e0 con 0x5602cb71c710 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94570003215808] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.087+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94570003213168] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.091+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.091+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94570003220448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72d540 con 0x5602cb71c710 2024-03-28T16:49:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.091+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72e230 con 0x5602cb720c80 2024-03-28T16:49:22.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.091+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94570003220448] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.091+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.091+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94570003226000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72d540 con 0x5602cb720c80 2024-03-28T16:49:22.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.091+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72ea70 con 0x5602cb720c80 2024-03-28T16:49:22.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94570003228048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72f270 con 0x5602cb720c80 2024-03-28T16:49:22.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72fd40 con 0x5602cb720c80 2024-03-28T16:49:22.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94570003226000] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94570003228048] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v29'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94570003232800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72fd00 con 0x5602cb720c80 2024-03-28T16:49:22.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 20 add_watcher() i=0 2024-03-28T16:49:22.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 20 add_watcher() i=1 2024-03-28T16:49:22.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 20 add_watcher() i=2 2024-03-28T16:49:22.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 20 add_watcher() i=3 2024-03-28T16:49:22.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 20 add_watcher() i=4 2024-03-28T16:49:22.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 20 add_watcher() i=5 2024-03-28T16:49:22.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.095+0000 7fa3ab2bb980 20 add_watcher() i=6 2024-03-28T16:49:22.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94570003232800] v29'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 add_watcher() i=7 2024-03-28T16:49:22.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:22.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72ecd0 con 0x5602cb71c710 2024-03-28T16:49:22.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94570003218032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72ecd0 con 0x5602cb71c710 2024-03-28T16:49:22.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94570003218032] v29'148 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa25f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 5 note: GC not initialized 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa25e7e4640 20 reqs_thread_entry: start 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 Filter name: none 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa257fff640 20 reqs_thread_entry: start 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:22.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb754c80 con 0x5602cb71c710 2024-03-28T16:49:22.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:22.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:22.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:22.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:22.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb7571a0 con 0x7fa39c0e0040 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv28 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 disabling bucket name=myfoo 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.099+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb755180 con 0x7fa39c0e0040 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:22.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb7579f0 con 0x5602cb725860 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(65 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:22.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb75f410 con 0x5602cb725860 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'6 uv6 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 7.9 7:93e5b521:::notify.7:head [notify cookie 94570003398240 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb759b60 con 0x5602cb720c80 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== watch-notify(notify (1) cookie 94570003232800 notify 124554051601 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5602cb561e00 con 0x5602cb720c80 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa27ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051601 cookie 94570003232800 notifier 4585 bl.length()=725 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(67 notify.7 [notify cookie 94570003398240 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa27ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa27ffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa27ffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa27ffff640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fa26c003ea0 con 0x5602cb720c80 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(68 notify.7 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify_complete (2) cookie 94570003398240 notify 124554051601 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fa39804c020 con 0x5602cb720c80 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_sources= new_sources= 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential sources added=[] removed=[] 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_dests= new_dests= 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential dests added=[] removed=[] 2024-03-28T16:49:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.103+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb75b7f0 con 0x5602cb71c710 2024-03-28T16:49:22.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v29'14 uv14 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3ab2bb980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3ab2bb980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3ab2bb980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94570003408800 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb75b7f0 con 0x5602cb71c710 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94570003220448 notify 124554051601 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7fa3a004a040 con 0x5602cb71c710 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(70 notify.4 [notify cookie 94570003408800 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa27ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051601 cookie 94570003220448 notifier 4585 bl.length()=533 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa27ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa27ffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa27ffff640 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fa26c004200 con 0x5602cb71c710 2024-03-28T16:49:22.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== watch-notify(notify_complete (2) cookie 94570003408800 notify 124554051601 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fa3a0018b60 con 0x5602cb71c710 2024-03-28T16:49:22.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.111 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:22.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa25f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:22.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94570003218032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb746da0 con 0x5602cb71c710 2024-03-28T16:49:22.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94570003218032] v29'149 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.107+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94570003207760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72be70 con 0x7fa39c0e0040 2024-03-28T16:49:22.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.111+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94570003207760] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.111+0000 7fa3ab2bb980 20 remove_watcher() i=0 2024-03-28T16:49:22.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.111+0000 7fa3ab2bb980 2 removed watcher, disabling cache 2024-03-28T16:49:22.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.111+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94570003212112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72d540 con 0x5602cb71c710 2024-03-28T16:49:22.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.111+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94570003212112] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.111+0000 7fa3ab2bb980 20 remove_watcher() i=1 2024-03-28T16:49:22.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.111+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94570003215808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72d540 con 0x5602cb725860 2024-03-28T16:49:22.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3a85d5640 1 -- 172.21.15.170:0/95721042 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94570003215808] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa39c0b0020 con 0x5602cb725860 2024-03-28T16:49:22.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3ab2bb980 20 remove_watcher() i=2 2024-03-28T16:49:22.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94570003213168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72d540 con 0x7fa39c0e0040 2024-03-28T16:49:22.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94570003213168] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x7fa39c0e0040 2024-03-28T16:49:22.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3ab2bb980 20 remove_watcher() i=3 2024-03-28T16:49:22.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94570003220448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72abf0 con 0x5602cb71c710 2024-03-28T16:49:22.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94570003220448] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa3a000a4d0 con 0x5602cb71c710 2024-03-28T16:49:22.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3ab2bb980 20 remove_watcher() i=4 2024-03-28T16:49:22.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.115+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94570003226000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72abf0 con 0x5602cb720c80 2024-03-28T16:49:22.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.119+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94570003226000] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.119+0000 7fa3ab2bb980 20 remove_watcher() i=5 2024-03-28T16:49:22.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.119+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94570003228048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72abf0 con 0x5602cb720c80 2024-03-28T16:49:22.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.119+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94570003228048] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.119+0000 7fa3ab2bb980 20 remove_watcher() i=6 2024-03-28T16:49:22.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.119+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94570003232800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x5602cb72abf0 con 0x5602cb720c80 2024-03-28T16:49:22.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3a8dd6640 1 -- 172.21.15.170:0/95721042 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94570003232800] v29'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa398016070 con 0x5602cb720c80 2024-03-28T16:49:22.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 20 remove_watcher() i=7 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb720c80 msgr2=0x5602cb723070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb720c80 0x5602cb723070 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb71c710 msgr2=0x5602cb71ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb71c710 0x5602cb71ebd0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa39c0e0040 msgr2=0x7fa39c0d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa39c0e0040 0x7fa39c0d3400 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5602cb725860 msgr2=0x5602cb727d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5602cb725860 0x5602cb727d20 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa2740342b0 msgr2=0x7fa274036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa2740342b0 0x7fa274036770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa39c000c10 tx=0x7fa39c0c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 msgr2=0x5602cb711330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/95721042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb563570 0x5602cb711330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa398036660 tx=0x7fa39800af30 comp rx=0 tx=0).stop 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 reap_dead start 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3a95d7640 1 -- 172.21.15.170:0/95721042 reap_dead start 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 shutdown_connections 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 >> 172.21.15.170:0/95721042 conn(0x5602cb704620 msgr2=0x5602cb701c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 shutdown_connections 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/95721042 wait complete. 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa280034260 msgr2=0x7fa280036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa280034260 0x7fa280036720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5602cb6e6030 tx=0x7fa39c005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb6fb560 msgr2=0x5602cb6fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb6fb560 0x5602cb6fb980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa39800ee50 tx=0x7fa398018760 comp rx=0 tx=0).stop 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 shutdown_connections 2024-03-28T16:49:22.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa280034260 0x7fa280036720 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/1421334453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb6fb560 0x5602cb6fb980 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 >> 172.21.15.170:0/1421334453 conn(0x5602cb6f0370 msgr2=0x5602cb6f0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 shutdown_connections 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/1421334453 wait complete. 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb6e6e90 msgr2=0x5602cb6e9280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5602cb6e6e90 0x5602cb6e9280 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb5f1440 msgr2=0x5602cb6e4880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5602cb5f1440 0x5602cb6e4880 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5602cb6eac00 msgr2=0x5602cb6ecff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5602cb6eac00 0x5602cb6ecff0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa290034300 msgr2=0x7fa2900367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa290034300 0x7fa2900367c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0ae000 tx=0x7fa39c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 msgr2=0x5602cb6ddc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 --2- 172.21.15.170:0/829884711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5602cb5ecf80 0x5602cb6ddc00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa39800f510 tx=0x7fa398000ee0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3a95d7640 1 -- 172.21.15.170:0/829884711 reap_dead start 2024-03-28T16:49:22.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 shutdown_connections 2024-03-28T16:49:22.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 >> 172.21.15.170:0/829884711 conn(0x5602cb5e8850 msgr2=0x5602cb6d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 shutdown_connections 2024-03-28T16:49:22.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.123+0000 7fa3ab2bb980 1 -- 172.21.15.170:0/829884711 wait complete. 2024-03-28T16:49:22.133 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:22.135 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-03-28T16:49:22.135 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-03-28T16:49:22.135 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-03-28T16:49:22.203 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:22.203 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:22.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9dcb06980 1 Processor -- start 2024-03-28T16:49:22.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9dcb06980 1 -- start start 2024-03-28T16:49:22.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9dcb06980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c115f82350 0x55c115f84740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9dcb06980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c115eca520 con 0x55c115ef28b0 2024-03-28T16:49:22.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9dcb06980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c115e910f0 con 0x55c115f82350 2024-03-28T16:49:22.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9d9e20640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c115f82350 0x55c115f84740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9d9e20640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c115f82350 0x55c115f84740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57276/0 (socket says 172.21.15.170:57276) 2024-03-28T16:49:22.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9da621640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55c115ef28b0 0x55c115f80c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:38366/0 (socket says 172.21.15.170:38366) 2024-03-28T16:49:22.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.203+0000 7fb9d9e20640 1 -- 172.21.15.170:0/3353172969 learned_addr learned my addr 172.21.15.170:0/3353172969 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9d9e20640 1 -- 172.21.15.170:0/3353172969 >> v1:172.21.15.119:6789/0 conn(0x55c115ef28b0 legacy=0x55c115f80c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9d9e20640 1 -- 172.21.15.170:0/3353172969 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c115f85cc0 con 0x55c115f82350 2024-03-28T16:49:22.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/3353172969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f82350 0x55c115f84740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400a4e0 tx=0x7fb9c401b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1799b2c87926a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9d961f640 1 -- 172.21.15.170:0/3353172969 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9c40aa540 con 0x55c115f82350 2024-03-28T16:49:22.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9d961f640 1 -- 172.21.15.170:0/3353172969 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c40a6070 con 0x55c115f82350 2024-03-28T16:49:22.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9d961f640 1 -- 172.21.15.170:0/3353172969 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9c40aa850 con 0x55c115f82350 2024-03-28T16:49:22.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9dcb06980 1 -- 172.21.15.170:0/3353172969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f82350 msgr2=0x55c115f84740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/3353172969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f82350 0x55c115f84740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400a4e0 tx=0x7fb9c401b090 comp rx=0 tx=0).stop 2024-03-28T16:49:22.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9dcb06980 1 -- 172.21.15.170:0/3353172969 shutdown_connections 2024-03-28T16:49:22.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/3353172969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f82350 0x55c115f84740 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9dcb06980 1 -- 172.21.15.170:0/3353172969 >> 172.21.15.170:0/3353172969 conn(0x55c115f78710 msgr2=0x55c115f7ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9dcb06980 1 -- 172.21.15.170:0/3353172969 shutdown_connections 2024-03-28T16:49:22.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.207+0000 7fb9dcb06980 1 -- 172.21.15.170:0/3353172969 wait complete. 2024-03-28T16:49:22.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 Processor -- start 2024-03-28T16:49:22.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- start start 2024-03-28T16:49:22.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c116067e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c115e8e620 con 0x55c115ef28b0 2024-03-28T16:49:22.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c116067e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c116067e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57290/0 (socket says 172.21.15.170:57290) 2024-03-28T16:49:22.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9da621640 1 -- 172.21.15.170:0/2851179485 learned_addr learned my addr 172.21.15.170:0/2851179485 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9da621640 1 -- 172.21.15.170:0/2851179485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c400a1c0 con 0x55c115ef28b0 2024-03-28T16:49:22.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9da621640 1 --2- 172.21.15.170:0/2851179485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c116067e70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001cc0 tx=0x7fb9d400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50e4d1bc79ab711 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb8c8608640 1 -- 172.21.15.170:0/2851179485 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4014070 con 0x55c115ef28b0 2024-03-28T16:49:22.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb8c8608640 1 -- 172.21.15.170:0/2851179485 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d400f590 con 0x55c115ef28b0 2024-03-28T16:49:22.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb8c8608640 1 -- 172.21.15.170:0/2851179485 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4007040 con 0x55c115ef28b0 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2851179485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 msgr2=0x55c116067e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2851179485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c116067e70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001cc0 tx=0x7fb9d400b610 comp rx=0 tx=0).stop 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2851179485 shutdown_connections 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2851179485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c116067e70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2851179485 >> 172.21.15.170:0/2851179485 conn(0x55c115f78710 msgr2=0x55c116063a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2851179485 shutdown_connections 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2851179485 wait complete. 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 Processor -- start 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- start start 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c11606ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.223+0000 7fb9dcb06980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c115e89e00 con 0x55c115ef28b0 2024-03-28T16:49:22.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c11606ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c11606ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57302/0 (socket says 172.21.15.170:57302) 2024-03-28T16:49:22.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9da621640 1 -- 172.21.15.170:0/984905632 learned_addr learned my addr 172.21.15.170:0/984905632 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9da621640 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c115eca520 con 0x55c115ef28b0 2024-03-28T16:49:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9da621640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c11606ec80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55c116068d00 tx=0x7fb9d4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb8c6e05640 1 -- 172.21.15.170:0/984905632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4014030 con 0x55c115ef28b0 2024-03-28T16:49:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c11606f1c0 con 0x55c115ef28b0 2024-03-28T16:49:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb8c6e05640 1 -- 172.21.15.170:0/984905632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d401c840 con 0x55c115ef28b0 2024-03-28T16:49:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb8c6e05640 1 -- 172.21.15.170:0/984905632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d401cb20 con 0x55c115ef28b0 2024-03-28T16:49:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c11606f5f0 con 0x55c115ef28b0 2024-03-28T16:49:22.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb8c6e05640 1 -- 172.21.15.170:0/984905632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb9d401cc80 con 0x55c115ef28b0 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb8c6e05640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb8c6e05640 1 -- 172.21.15.170:0/984905632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb9d4006070 con 0x55c115ef28b0 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c115f82350 0x55c116075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c116075e40 con 0x55c115f82350 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dae22640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c115f82350 0x55c116075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40ab000 tx=0x7fb9c40a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dae22640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c115f82350 0x55c116075900 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dae22640 1 -- 172.21.15.170:0/984905632 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c115f82350 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.227+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160761a0 con 0x55c115f82350 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dae22640 1 -- 172.21.15.170:0/984905632 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c115f82350 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c116077ec0 0x55c11607a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c11607a7f0 con 0x55c116077ec0 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9da621640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c116077ec0 0x55c11607a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9da621640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c116077ec0 0x55c11607a2b0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9da621640 1 -- 172.21.15.170:0/984905632 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fb9d4016070 con 0x55c116077ec0 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c116076d00 con 0x55c115f82350 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dae22640 1 -- 172.21.15.170:0/984905632 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c115f82350 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c11607bb60 0x55c11607df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c11607e5f0 con 0x55c11607bb60 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c11607bb60 0x55c11607df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c11607bb60 0x55c11607df50 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9d9e20640 1 -- 172.21.15.170:0/984905632 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c11607bb60 2024-03-28T16:49:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 Processor -- start 2024-03-28T16:49:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 -- start start 2024-03-28T16:49:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c11608c500 0x55c11608c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9dcb06980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c115e22e30 con 0x55c11608c500 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.231+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c11608c500 0x55c11608c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c11608c500 0x55c11608c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57312/0 (socket says 172.21.15.170:57312) 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 -- 172.21.15.170:0/2383332279 learned_addr learned my addr 172.21.15.170:0/2383332279 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 -- 172.21.15.170:0/2383332279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11608ce60 con 0x55c11608c500 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c11608c500 0x55c11608c920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4044010 tx=0x7fb9d4017d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb8bffff640 1 -- 172.21.15.170:0/2383332279 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4041830 con 0x55c11608c500 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c11608d0b0 con 0x55c11608c500 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb8bffff640 1 -- 172.21.15.170:0/2383332279 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4049ba0 con 0x55c11608c500 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c11608dc00 con 0x55c11608c500 2024-03-28T16:49:22.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb8bffff640 1 -- 172.21.15.170:0/2383332279 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4049e40 con 0x55c11608c500 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb8bffff640 1 -- 172.21.15.170:0/2383332279 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb9d40412c0 con 0x55c11608c500 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb8bffff640 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b4034300 0x7fb8b40367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb8bffff640 1 -- 172.21.15.170:0/2383332279 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb9d4048a40 con 0x55c11608c500 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b4034300 0x7fb8b40367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9dcb06980 1 Processor -- start 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9dcb06980 1 -- start start 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b4034300 0x7fb8b40367c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55c116073330 tx=0x7fb9c4005e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9dcb06980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c115f7e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9dcb06980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c115e16790 con 0x55c115f7e1c0 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c115f7e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c115f7e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57322/0 (socket says 172.21.15.170:57322) 2024-03-28T16:49:22.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 -- 172.21.15.170:0/2705741956 learned_addr learned my addr 172.21.15.170:0/2705741956 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.235+0000 7fb9da621640 1 -- 172.21.15.170:0/2705741956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c115f7ca10 con 0x55c115f7e1c0 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9da621640 1 --2- 172.21.15.170:0/2705741956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c115f7e5e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4017fb0 tx=0x7fb9d40268c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650ab7f8b13f0738 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb8bdffb640 1 -- 172.21.15.170:0/2705741956 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4041570 con 0x55c115f7e1c0 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb8bdffb640 1 -- 172.21.15.170:0/2705741956 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d402dbb0 con 0x55c115f7e1c0 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb8bdffb640 1 -- 172.21.15.170:0/2705741956 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4026410 con 0x55c115f7e1c0 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2705741956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 msgr2=0x55c115f7e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2705741956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c115f7e5e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4017fb0 tx=0x7fb9d40268c0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2705741956 shutdown_connections 2024-03-28T16:49:22.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2705741956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c115f7e5e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2705741956 >> 172.21.15.170:0/2705741956 conn(0x55c116095720 msgr2=0x55c116093780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2705741956 shutdown_connections 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2705741956 wait complete. 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 Processor -- start 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- start start 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c1160a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1160a2970 con 0x55c115f7e1c0 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c1160a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9da621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c1160a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:57326/0 (socket says 172.21.15.170:57326) 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 learned_addr learned my addr 172.21.15.170:0/323759287 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11608dc00 con 0x55c115f7e1c0 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9da621640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c1160a2430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4017fb0 tx=0x7fb9d40486b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb8abfff640 1 -- 172.21.15.170:0/323759287 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4048ca0 con 0x55c115f7e1c0 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c1160a2b70 con 0x55c115f7e1c0 2024-03-28T16:49:22.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb8abfff640 1 -- 172.21.15.170:0/323759287 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4048e00 con 0x55c115f7e1c0 2024-03-28T16:49:22.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1160a2f40 con 0x55c115f7e1c0 2024-03-28T16:49:22.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.239+0000 7fb8abfff640 1 -- 172.21.15.170:0/323759287 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9d4026cf0 con 0x55c115f7e1c0 2024-03-28T16:49:22.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb8abfff640 1 -- 172.21.15.170:0/323759287 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb9d4019a40 con 0x55c115f7e1c0 2024-03-28T16:49:22.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb8abfff640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8a40342b0 0x7fb8a4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb8abfff640 1 -- 172.21.15.170:0/323759287 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb9d4072aa0 con 0x55c115f7e1c0 2024-03-28T16:49:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8a40342b0 0x7fb8a4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1160ad730 0x55c1160afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0340 con 0x55c1160ad730 2024-03-28T16:49:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dae22640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1160ad730 0x55c1160afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8a40342b0 0x7fb8a4036770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40aafd0 tx=0x7fb9c40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dae22640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1160ad730 0x55c1160afbf0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 realm 2024-03-28T16:49:22.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0710 con 0x55c1160ad730 2024-03-28T16:49:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b12c0 con 0x55c1160ad730 2024-03-28T16:49:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1160b1c70 0x55c1160b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b4700 con 0x55c1160b1c70 2024-03-28T16:49:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9da621640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1160b1c70 0x55c1160b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9da621640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1160b1c70 0x55c1160b4060 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 20 searching for the correct realm 2024-03-28T16:49:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1160b68c0 0x55c1160b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b9400 con 0x55c1160b68c0 2024-03-28T16:49:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.243+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1160b68c0 0x55c1160b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1160b68c0 0x55c1160b8d80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9d9e20640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9c40cd050 0x7fb9c40cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:22.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b9400 con 0x7fb9c40cd050 2024-03-28T16:49:22.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9c40cd050 0x7fb9c40cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:22.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9c40cd050 0x7fb9c40cd430 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:22.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x7fb9c40cd050 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.247+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x7fb9c40cd050 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.251+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x7fb9c40cd050 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x7fb9c40cd050 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b68c0 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.255+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160b1c70 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 29 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x55c1160ad730 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b0f60 con 0x7fb9c40cd050 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b4cc0 con 0x55c1160ad730 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b4cc0 con 0x55c1160ad730 2024-03-28T16:49:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b4c70 con 0x7fb9c40cd050 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 zone default found 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 4 Realm: () 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:22.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 zonegroup default 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b5510 con 0x55c1160b1c70 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160b58e0 con 0x55c1160b68c0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bb2a0 con 0x55c1160b68c0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 started sync module instance, tier type = 2024-03-28T16:49:22.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bab00 con 0x7fb9c40cd050 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94287786911328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160baa80 con 0x7fb9c40cd050 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.259+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bc000 con 0x55c1160ad730 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.263+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94287786911328] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.263+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.263+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94287786915520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bbf80 con 0x55c1160ad730 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.263+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bc780 con 0x55c1160b68c0 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94287786915520] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94287786916848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bcb30 con 0x55c1160b68c0 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bd4a0 con 0x7fb9c40cd050 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94287786922256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bdb90 con 0x7fb9c40cd050 2024-03-28T16:49:22.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160be640 con 0x55c1160ad730 2024-03-28T16:49:22.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94287786916848] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94287786922256] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94287786926400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160be600 con 0x55c1160ad730 2024-03-28T16:49:22.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.267+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bea20 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94287786926400] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94287786917968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bedd0 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bf740 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94287786932896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0360 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0cd0 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.271+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94287786917968] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v29'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94287786932896] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94287786936368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0c50 con 0x55c1160b1c70 2024-03-28T16:49:22.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=0 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=1 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=2 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=3 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=4 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=5 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=6 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94287786936368] v29'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 20 add_watcher() i=7 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.275+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c40b0 con 0x55c1160ad730 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94287786933952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c1550 con 0x55c1160ad730 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94287786933952] v29'150 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb890fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 5 note: GC not initialized 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb80b7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 Filter name: none 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb8097fa640 20 reqs_thread_entry: start 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160e5f30 con 0x55c1160ad730 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv14 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:22.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160e8450 con 0x7fb9c40cd050 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv28 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 enabling bucket name=myfoo 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160e6430 con 0x7fb9c40cd050 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 : miss 2024-03-28T16:49:22.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rados->read ofs=0 len=0 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160e8c60 con 0x55c1160b68c0 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(65 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv6 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.279+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 8.12 8:4b9ba655:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160f0690 con 0x55c1160b68c0 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v29'7 uv7 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 10 moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1, ofs0, ns] 2024-03-28T16:49:22.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 7.9 7:93e5b521:::notify.7:head [notify cookie 94287787102480 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160eae30 con 0x55c1160b1c70 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== watch-notify(notify (1) cookie 94287786936368 notify 124554051602 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55c11608dc00 con 0x55c1160b1c70 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb8bdffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051602 cookie 94287786936368 notifier 4592 bl.length()=725 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb8bdffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 info.flags=0x17 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb8bdffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1 to cache LRU end 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb8bdffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb8bdffb640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fb8a0003ea0 con 0x55c1160b1c70 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(67 notify.7 [notify cookie 94287787102480 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(68 notify.7 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d4041070 con 0x55c1160b1c70 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify_complete (2) cookie 94287787102480 notify 124554051602 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb9d408c020 con 0x55c1160b1c70 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_sources= new_sources= 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential sources added=[] removed=[] 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): orig_dests= new_dests= 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 20 handle_bi_update(): bucket=:myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.1]): potential dests added=[] removed=[] 2024-03-28T16:49:22.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.283+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160ecac0 con 0x55c1160ad730 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v29'15 uv15 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94287787113072 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160ecac0 con 0x55c1160ad730 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94287786926400 notify 124554051602 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7fb9cc00a930 con 0x55c1160ad730 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb8bd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051602 cookie 94287786926400 notifier 4592 bl.length()=533 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(70 notify.4 [notify cookie 94287787113072 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb8bd7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb8bd7fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb8bd7fa640 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) v8 -- 0x7fb810003430 con 0x55c1160ad730 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94287787113072 notify 124554051602 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fb9cc00aac0 con 0x55c1160ad730 2024-03-28T16:49:22.302 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb890fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94287786933952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160d8070 con 0x55c1160ad730 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94287786933952] v29'151 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94287786911328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160bfec0 con 0x7fb9c40cd050 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94287786911328] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 20 remove_watcher() i=0 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 2 removed watcher, disabling cache 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.287+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94287786915520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0360 con 0x55c1160ad730 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.291+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94287786915520] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.291+0000 7fb9dcb06980 20 remove_watcher() i=1 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.291+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94287786916848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0310 con 0x55c1160b68c0 2024-03-28T16:49:22.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.291+0000 7fb9d9e20640 1 -- 172.21.15.170:0/323759287 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94287786916848] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9c40aaa00 con 0x55c1160b68c0 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.291+0000 7fb9dcb06980 20 remove_watcher() i=2 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.291+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94287786922256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0310 con 0x7fb9c40cd050 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.295+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94287786922256] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x7fb9c40cd050 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.295+0000 7fb9dcb06980 20 remove_watcher() i=3 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.295+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94287786926400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0310 con 0x55c1160ad730 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.295+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94287786926400] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9cc0083f0 con 0x55c1160ad730 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.295+0000 7fb9dcb06980 20 remove_watcher() i=4 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.295+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94287786917968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0310 con 0x55c1160b1c70 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.299+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94287786917968] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d408e040 con 0x55c1160b1c70 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.299+0000 7fb9dcb06980 20 remove_watcher() i=5 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.299+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94287786932896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0310 con 0x55c1160b1c70 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.299+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94287786932896] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d408e040 con 0x55c1160b1c70 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.299+0000 7fb9dcb06980 20 remove_watcher() i=6 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.299+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94287786936368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) v8 -- 0x55c1160c0310 con 0x55c1160b1c70 2024-03-28T16:49:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9da621640 1 -- 172.21.15.170:0/323759287 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94287786936368] v29'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d408e040 con 0x55c1160b1c70 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 20 remove_watcher() i=7 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1160b1c70 msgr2=0x55c1160b4060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1160b1c70 0x55c1160b4060 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1160ad730 msgr2=0x55c1160afbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1160ad730 0x55c1160afbf0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9c40cd050 msgr2=0x7fb9c40cd430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9c40cd050 0x7fb9c40cd430 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1160b68c0 msgr2=0x55c1160b8d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1160b68c0 0x55c1160b8d80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8a40342b0 msgr2=0x7fb8a4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8a40342b0 0x7fb8a4036770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40aafd0 tx=0x7fb9c40a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 msgr2=0x55c1160a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/323759287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115f7e1c0 0x55c1160a2430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4017fb0 tx=0x7fb9d40486b0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 reap_dead start 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dae22640 1 -- 172.21.15.170:0/323759287 reap_dead start 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 shutdown_connections 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 >> 172.21.15.170:0/323759287 conn(0x55c116095720 msgr2=0x55c116092d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 shutdown_connections 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/323759287 wait complete. 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b4034300 msgr2=0x7fb8b40367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b4034300 0x7fb8b40367c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55c116073330 tx=0x7fb9c4005e50 comp rx=0 tx=0).stop 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c11608c500 msgr2=0x55c11608c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c11608c500 0x55c11608c920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4044010 tx=0x7fb9d4017d30 comp rx=0 tx=0).stop 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 shutdown_connections 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b4034300 0x7fb8b40367c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/2383332279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c11608c500 0x55c11608c920 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 >> 172.21.15.170:0/2383332279 conn(0x55c116081310 msgr2=0x55c116081e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 shutdown_connections 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/2383332279 wait complete. 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c116077ec0 msgr2=0x55c11607a2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c116077ec0 0x55c11607a2b0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c115f82350 msgr2=0x55c116075900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c115f82350 0x55c116075900 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c11607bb60 msgr2=0x55c11607df50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c11607bb60 0x55c11607df50 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 msgr2=0x7fb8c0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40ab000 tx=0x7fb9c40a5000 comp rx=0 tx=0).stop 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 msgr2=0x55c11606ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 --2- 172.21.15.170:0/984905632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c115ef28b0 0x55c11606ec80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55c116068d00 tx=0x7fb9d4000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dae22640 1 -- 172.21.15.170:0/984905632 reap_dead start 2024-03-28T16:49:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 shutdown_connections 2024-03-28T16:49:22.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 >> 172.21.15.170:0/984905632 conn(0x55c115f78710 msgr2=0x55c116062510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:22.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 shutdown_connections 2024-03-28T16:49:22.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:22.303+0000 7fb9dcb06980 1 -- 172.21.15.170:0/984905632 wait complete. 2024-03-28T16:49:22.313 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:22.320 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-03-28T16:49:22.669 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-03-28T16:49:22.686 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:37.686 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-03-28T16:49:37.686 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-03-28T16:49:37.687 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-03-28T16:49:37.725 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:37.725 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:37.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c2a9e4980 1 Processor -- start 2024-03-28T16:49:37.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c2a9e4980 1 -- start start 2024-03-28T16:49:37.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c2a9e4980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556efdc50630 0x556efdc52a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c2a9e4980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556efdb96520 con 0x556efdc4c130 2024-03-28T16:49:37.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c2a9e4980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x556efdb5d0f0 con 0x556efdc50630 2024-03-28T16:49:37.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c27cfe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556efdc50630 0x556efdc52a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c27cfe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556efdc50630 0x556efdc52a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49710/0 (socket says 172.21.15.170:49710) 2024-03-28T16:49:37.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c27cfe640 1 -- 172.21.15.170:0/435426203 learned_addr learned my addr 172.21.15.170:0/435426203 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:37.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c284ff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x556efdc4c130 0x556efdc4c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:39082/0 (socket says 172.21.15.170:39082) 2024-03-28T16:49:37.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c27cfe640 1 -- 172.21.15.170:0/435426203 >> v1:172.21.15.119:6789/0 conn(0x556efdc4c130 legacy=0x556efdc4c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.726+0000 7f8c27cfe640 1 -- 172.21.15.170:0/435426203 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556efdc53f70 con 0x556efdc50630 2024-03-28T16:49:37.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/435426203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc50630 0x556efdc52a20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00a5c0 tx=0x7f8c1c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40216c20a3ff2b46 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c274fd640 1 -- 172.21.15.170:0/435426203 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c1c0aa470 con 0x556efdc50630 2024-03-28T16:49:37.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c274fd640 1 -- 172.21.15.170:0/435426203 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1c0aa5d0 con 0x556efdc50630 2024-03-28T16:49:37.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c274fd640 1 -- 172.21.15.170:0/435426203 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c1c0aa890 con 0x556efdc50630 2024-03-28T16:49:37.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/435426203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc50630 msgr2=0x556efdc52a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/435426203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc50630 0x556efdc52a20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00a5c0 tx=0x7f8c1c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:37.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/435426203 shutdown_connections 2024-03-28T16:49:37.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/435426203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc50630 0x556efdc52a20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/435426203 >> 172.21.15.170:0/435426203 conn(0x556efdbbf850 msgr2=0x556efdbbfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:37.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/435426203 shutdown_connections 2024-03-28T16:49:37.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.730+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/435426203 wait complete. 2024-03-28T16:49:37.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c2a9e4980 1 Processor -- start 2024-03-28T16:49:37.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c2a9e4980 1 -- start start 2024-03-28T16:49:37.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c2a9e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c2a9e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efdb5a620 con 0x556efdc4c130 2024-03-28T16:49:37.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd35fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49726/0 (socket says 172.21.15.170:49726) 2024-03-28T16:49:37.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c284ff640 1 -- 172.21.15.170:0/2446885324 learned_addr learned my addr 172.21.15.170:0/2446885324 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c284ff640 1 -- 172.21.15.170:0/2446885324 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c005df0 con 0x556efdc4c130 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8c284ff640 1 --2- 172.21.15.170:0/2446885324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd35fe0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8c20006fc0 tx=0x7f8c200019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1e4016b18b5da7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8b1660c640 1 -- 172.21.15.170:0/2446885324 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c20013420 con 0x556efdc4c130 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8b1660c640 1 -- 172.21.15.170:0/2446885324 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20013580 con 0x556efdc4c130 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.746+0000 7f8b1660c640 1 -- 172.21.15.170:0/2446885324 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c20013880 con 0x556efdc4c130 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/2446885324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 msgr2=0x556efdd35fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/2446885324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd35fe0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8c20006fc0 tx=0x7f8c200019b0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/2446885324 shutdown_connections 2024-03-28T16:49:37.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/2446885324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd35fe0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/2446885324 >> 172.21.15.170:0/2446885324 conn(0x556efdbbf850 msgr2=0x556efdd31b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:37.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/2446885324 shutdown_connections 2024-03-28T16:49:37.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/2446885324 wait complete. 2024-03-28T16:49:37.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 Processor -- start 2024-03-28T16:49:37.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- start start 2024-03-28T16:49:37.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd3ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efdb55e00 con 0x556efdc4c130 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd3ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd3ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49728/0 (socket says 172.21.15.170:49728) 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c284ff640 1 -- 172.21.15.170:0/652104308 learned_addr learned my addr 172.21.15.170:0/652104308 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c284ff640 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556efdb96520 con 0x556efdc4c130 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c284ff640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd3ce30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8c200012f0 tx=0x7f8c20001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8b14e09640 1 -- 172.21.15.170:0/652104308 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c20006320 con 0x556efdc4c130 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8b14e09640 1 -- 172.21.15.170:0/652104308 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20006840 con 0x556efdc4c130 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556efdd3d370 con 0x556efdc4c130 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8b14e09640 1 -- 172.21.15.170:0/652104308 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c20004c90 con 0x556efdc4c130 2024-03-28T16:49:37.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556efdd3d7a0 con 0x556efdc4c130 2024-03-28T16:49:37.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8b14e09640 1 -- 172.21.15.170:0/652104308 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8c2001e3f0 con 0x556efdc4c130 2024-03-28T16:49:37.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.750+0000 7f8b14e09640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b100342b0 0x7f8b10036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8b14e09640 1 -- 172.21.15.170:0/652104308 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8c200497b0 con 0x556efdc4c130 2024-03-28T16:49:37.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b100342b0 0x7f8b10036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdc50630 0x556efdd43ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd43ff0 con 0x556efdc50630 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c28d00640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdc50630 0x556efdd43ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b100342b0 0x7f8b10036770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0ae000 tx=0x7f8c1c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c28d00640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdc50630 0x556efdd43ab0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c28d00640 1 -- 172.21.15.170:0/652104308 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdc50630 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd44350 con 0x556efdc50630 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c28d00640 1 -- 172.21.15.170:0/652104308 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdc50630 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd46070 0x556efdd48460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd489a0 con 0x556efdd46070 2024-03-28T16:49:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c284ff640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd46070 0x556efdd48460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c284ff640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd46070 0x556efdd48460 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c284ff640 1 -- 172.21.15.170:0/652104308 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd46070 2024-03-28T16:49:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd44ee0 con 0x556efdc50630 2024-03-28T16:49:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c28d00640 1 -- 172.21.15.170:0/652104308 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdc50630 2024-03-28T16:49:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556efdd49d80 0x556efdd4c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd4c7c0 con 0x556efdd49d80 2024-03-28T16:49:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.754+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556efdd49d80 0x556efdd4c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556efdd49d80 0x556efdd4c170 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c27cfe640 1 -- 172.21.15.170:0/652104308 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd49d80 2024-03-28T16:49:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c2a9e4980 1 Processor -- start 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c2a9e4980 1 -- start start 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c2a9e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd5a6f0 0x556efdd5ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c2a9e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efdaeee30 con 0x556efdd5a6f0 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd5a6f0 0x556efdd5ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd5a6f0 0x556efdd5ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49732/0 (socket says 172.21.15.170:49732) 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c284ff640 1 -- 172.21.15.170:0/470283888 learned_addr learned my addr 172.21.15.170:0/470283888 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c284ff640 1 -- 172.21.15.170:0/470283888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556efdd5b050 con 0x556efdd5a6f0 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c284ff640 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd5a6f0 0x556efdd5ab10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8c20047bc0 tx=0x7f8c20011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8b0dffb640 1 -- 172.21.15.170:0/470283888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c20026a90 con 0x556efdd5a6f0 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8b0dffb640 1 -- 172.21.15.170:0/470283888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20026c10 con 0x556efdd5a6f0 2024-03-28T16:49:37.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556efdd5b2a0 con 0x556efdd5a6f0 2024-03-28T16:49:37.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8b0dffb640 1 -- 172.21.15.170:0/470283888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c200490c0 con 0x556efdd5a6f0 2024-03-28T16:49:37.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.758+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556efdd5bdf0 con 0x556efdd5a6f0 2024-03-28T16:49:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8b0dffb640 1 -- 172.21.15.170:0/470283888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8c20049220 con 0x556efdd5a6f0 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8b0dffb640 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b000342b0 0x7f8b00036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8b0dffb640 1 -- 172.21.15.170:0/470283888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8c2003ebd0 con 0x556efdd5a6f0 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b000342b0 0x7f8b00036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 Processor -- start 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- start start 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd67ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efdae2790 con 0x556efdd67690 2024-03-28T16:49:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b000342b0 0x7f8b00036770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x556efdd45240 tx=0x7f8c1c005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd67ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd67ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49742/0 (socket says 172.21.15.170:49742) 2024-03-28T16:49:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c284ff640 1 -- 172.21.15.170:0/4112270178 learned_addr learned my addr 172.21.15.170:0/4112270178 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c284ff640 1 -- 172.21.15.170:0/4112270178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556efdd68080 con 0x556efdd67690 2024-03-28T16:49:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c284ff640 1 --2- 172.21.15.170:0/4112270178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd67ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8c20048fd0 tx=0x7f8c20027cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fa83d7d7e4bc4e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8b077fe640 1 -- 172.21.15.170:0/4112270178 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c2003ee60 con 0x556efdd67690 2024-03-28T16:49:37.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8b077fe640 1 -- 172.21.15.170:0/4112270178 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c200411e0 con 0x556efdd67690 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8b077fe640 1 -- 172.21.15.170:0/4112270178 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c200494d0 con 0x556efdd67690 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/4112270178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 msgr2=0x556efdd67ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/4112270178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd67ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8c20048fd0 tx=0x7f8c20027cd0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/4112270178 shutdown_connections 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/4112270178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd67ab0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/4112270178 >> 172.21.15.170:0/4112270178 conn(0x556efdd639d0 msgr2=0x556efdd61a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/4112270178 shutdown_connections 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/4112270178 wait complete. 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 Processor -- start 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- start start 2024-03-28T16:49:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd6e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.762+0000 7f8c2a9e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556efdd6e860 con 0x556efdd67690 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd6e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c284ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd6e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49748/0 (socket says 172.21.15.170:49748) 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 learned_addr learned my addr 172.21.15.170:0/3197727917 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556efdd5bdf0 con 0x556efdd67690 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c284ff640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd6e320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8c200469c0 tx=0x7f8c2002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8b05ffb640 1 -- 172.21.15.170:0/3197727917 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c20064070 con 0x556efdd67690 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556efdd5bdf0 con 0x556efdd67690 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8b05ffb640 1 -- 172.21.15.170:0/3197727917 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2001cc00 con 0x556efdd67690 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8b05ffb640 1 -- 172.21.15.170:0/3197727917 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c20012700 con 0x556efdd67690 2024-03-28T16:49:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556efdd6ec40 con 0x556efdd67690 2024-03-28T16:49:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8b05ffb640 1 -- 172.21.15.170:0/3197727917 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8c20012920 con 0x556efdd67690 2024-03-28T16:49:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8b05ffb640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8af8034260 0x7f8af8036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8b05ffb640 1 -- 172.21.15.170:0/3197727917 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8c20073bf0 con 0x556efdd67690 2024-03-28T16:49:37.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8af8034260 0x7f8af8036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdd79590 0x556efdd7ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7c180 con 0x556efdd79590 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c28d00640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdd79590 0x556efdd7ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8af8034260 0x7f8af8036720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c000c10 tx=0x7f8c1c0c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.766+0000 7f8c28d00640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdd79590 0x556efdd7ba30 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 realm 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7c550 con 0x556efdd79590 2024-03-28T16:49:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7cda0 con 0x556efdd79590 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd7de10 0x556efdd80200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd80780 con 0x556efdd7de10 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c284ff640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd7de10 0x556efdd80200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c284ff640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd7de10 0x556efdd80200 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 20 searching for the correct realm 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556efdd82680 0x556efdd84ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd851a0 con 0x556efdd82680 2024-03-28T16:49:37.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556efdd82680 0x556efdd84ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556efdd82680 0x556efdd84ad0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c27cfe640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8c1c0e0040 0x7f8c1c0d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd851a0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.770+0000 7f8c28d00640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8c1c0e0040 0x7f8c1c0d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8c1c0e0040 0x7f8c1c0d3400 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x7f8c1c0e0040 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.774+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x7f8c1c0e0040 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.778+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x7f8c1c0e0040 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x7f8c1c0e0040 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.782+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd82680 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd7de10 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x556efdd79590 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd7d100 con 0x7f8c1c0e0040 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd80ff0 con 0x556efdd79590 2024-03-28T16:49:37.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd80fc0 con 0x556efdd79590 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd80fc0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 zone default found 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 4 Realm: () 2024-03-28T16:49:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 zonegroup default 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd81840 con 0x556efdd7de10 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd81c40 con 0x556efdd82680 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd86fc0 con 0x556efdd82680 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 started sync module instance, tier type = 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.786+0000 7f8c2a9e4980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd868a0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93935193580544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd86c50 con 0x7f8c1c0e0040 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd87cf0 con 0x556efdd79590 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93935193580544] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93935193584560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd87d30 con 0x556efdd79590 2024-03-28T16:49:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.790+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd88800 con 0x556efdd82680 2024-03-28T16:49:37.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93935193588448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd89050 con 0x556efdd82680 2024-03-28T16:49:37.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd89a00 con 0x7f8c1c0e0040 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93935193584560] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93935193585616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd89de0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8a750 con 0x556efdd79590 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.794+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93935193588448] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.798+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93935193585616] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.798+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.798+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93935193593120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8a2b0 con 0x556efdd79590 2024-03-28T16:49:37.801 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.798+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8afa0 con 0x556efdd7de10 2024-03-28T16:49:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.798+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93935193593120] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.802+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.802+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93935193598720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8a2b0 con 0x556efdd7de10 2024-03-28T16:49:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.802+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8b7a0 con 0x556efdd7de10 2024-03-28T16:49:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.802+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93935193598720] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.802+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.802+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93935193590704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8bfa0 con 0x556efdd7de10 2024-03-28T16:49:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.802+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8c910 con 0x556efdd7de10 2024-03-28T16:49:37.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93935193590704] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93935193605232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8cd30 con 0x556efdd7de10 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=0 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=1 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=2 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=3 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=4 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=5 2024-03-28T16:49:37.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=6 2024-03-28T16:49:37.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93935193605232] v31'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 20 add_watcher() i=7 2024-03-28T16:49:37.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:37.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd90010 con 0x556efdd79590 2024-03-28T16:49:37.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.806+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93935193606656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd8d2c0 con 0x556efdd79590 2024-03-28T16:49:37.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.810+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93935193606656] v31'152 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.810+0000 7f8ade7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:37.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.810+0000 7f8c2a9e4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:37.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.810+0000 7f8c2a9e4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:37.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.810+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd98520 con 0x556efdd79590 2024-03-28T16:49:37.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.810+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.810+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd98520 con 0x556efdd7de10 2024-03-28T16:49:37.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.814+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.814+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd986f0 con 0x556efdd82680 2024-03-28T16:49:37.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.814+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.814+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd990c0 con 0x556efdd79590 2024-03-28T16:49:37.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.818+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'153 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.818+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd966d0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.818+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'44 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.818+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd966d0 con 0x556efdd82680 2024-03-28T16:49:37.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.818+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'29 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.818+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd99e00 con 0x556efdd79590 2024-03-28T16:49:37.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.822+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.822+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd99e00 con 0x556efdd7de10 2024-03-28T16:49:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.822+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'29 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.822+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9a330 con 0x7f8c1c0e0040 2024-03-28T16:49:37.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.826+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.826+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9a330 con 0x556efdd82680 2024-03-28T16:49:37.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.826+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.826+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9a860 con 0x7f8c1c0e0040 2024-03-28T16:49:37.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.830+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'45 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.830+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9a860 con 0x556efdd7de10 2024-03-28T16:49:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.830+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.830+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ad00 con 0x7f8c1c0e0040 2024-03-28T16:49:37.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.834+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.834+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ad00 con 0x556efdd7de10 2024-03-28T16:49:37.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.834+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'36 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.834+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ad00 con 0x556efdd79590 2024-03-28T16:49:37.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.838+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'154 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.838+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ad00 con 0x556efdd82680 2024-03-28T16:49:37.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.838+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'38 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.838+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x7f8c1c0e0040 2024-03-28T16:49:37.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.842+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'46 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.842+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x556efdd82680 2024-03-28T16:49:37.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.842+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.842+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ac00 con 0x556efdd79590 2024-03-28T16:49:37.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.846+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'26 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.846+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ac00 con 0x556efdd79590 2024-03-28T16:49:37.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.846+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'155 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.846+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9abd0 con 0x556efdd82680 2024-03-28T16:49:37.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.850+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'39 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.850+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9abd0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.850+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.850+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x556efdd82680 2024-03-28T16:49:37.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.854+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.854+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9abd0 con 0x556efdd82680 2024-03-28T16:49:37.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.854+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.854+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9abd0 con 0x556efdd82680 2024-03-28T16:49:37.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.854+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.854+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9abd0 con 0x556efdd82680 2024-03-28T16:49:37.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.858+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'30 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.858+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x556efdd7de10 2024-03-28T16:49:37.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.858+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'21 uv20 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.858+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x556efdd82680 2024-03-28T16:49:37.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.862+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.862+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x556efdd7de10 2024-03-28T16:49:37.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.866+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'30 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.866+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x556efdd82680 2024-03-28T16:49:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.866+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'40 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.866+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x556efdd7de10 2024-03-28T16:49:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.870+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'37 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.870+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd9ab40 con 0x7f8c1c0e0040 2024-03-28T16:49:37.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8add7e2640 20 reqs_thread_entry: start 2024-03-28T16:49:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c2a9e4980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c2a9e4980 20 Filter name: none 2024-03-28T16:49:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8a4effd640 20 reqs_thread_entry: start 2024-03-28T16:49:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95920 con 0x556efdd79590 2024-03-28T16:49:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.874+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd958e0 con 0x556efdd7de10 2024-03-28T16:49:37.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95920 con 0x556efdd82680 2024-03-28T16:49:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x556efdd82680 2024-03-28T16:49:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x556efdd79590 2024-03-28T16:49:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.878+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x556efdd79590 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x556efdd79590 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x7f8c1c0e0040 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x7f8c1c0e0040 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x7f8c1c0e0040 2024-03-28T16:49:37.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95d00 con 0x556efdd82680 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd94550 con 0x556efdd82680 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 31 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd79590 2024-03-28T16:49:37.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.882+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd7de10 2024-03-28T16:49:37.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv6 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x7f8c1c0e0040 2024-03-28T16:49:37.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.886+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 32 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd82680 2024-03-28T16:49:37.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 33 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 34 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 25 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95910 con 0x7f8c1c0e0040 2024-03-28T16:49:37.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 26 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 27 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.890+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95910 con 0x556efdd7de10 2024-03-28T16:49:37.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 28 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95910 con 0x7f8c1c0e0040 2024-03-28T16:49:37.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 29 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 30 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.894+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95910 con 0x556efdd7de10 2024-03-28T16:49:37.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95910 con 0x556efdd79590 2024-03-28T16:49:37.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.898+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 35 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95910 con 0x556efdd82680 2024-03-28T16:49:37.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 36 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 37 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 31 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd94550 con 0x7f8c1c0e0040 2024-03-28T16:49:37.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 32 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 33 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 38 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd82680 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 39 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 40 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.902+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd79590 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd79590 2024-03-28T16:49:37.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd79590 2024-03-28T16:49:37.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 41 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd82680 2024-03-28T16:49:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 42 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 43 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 34 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.906+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x7f8c1c0e0040 2024-03-28T16:49:37.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 35 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 36 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 44 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 45 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 46 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 47 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd82680 2024-03-28T16:49:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 48 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 49 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 50 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 51 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 52 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.910+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 53 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 54 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 55 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95940 con 0x556efdd7de10 2024-03-28T16:49:37.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.914+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(174 gc.26 [call out=1701b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+1701 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 56 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd94550 con 0x556efdd82680 2024-03-28T16:49:37.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 57 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 58 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.918+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd94550 con 0x556efdd7de10 2024-03-28T16:49:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 41 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 42 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 59 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd94550 con 0x556efdd82680 2024-03-28T16:49:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 60 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.922+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd82680 2024-03-28T16:49:37.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 61 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 43 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd94550 con 0x556efdd7de10 2024-03-28T16:49:37.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 44 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x556efdd7de10 2024-03-28T16:49:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 45 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 37 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd94550 con 0x7f8c1c0e0040 2024-03-28T16:49:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 38 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.926+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95560 con 0x7f8c1c0e0040 2024-03-28T16:49:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.930+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 39 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.930+0000 7f8ade7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:37.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.930+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93935193606656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x556efdd79590 2024-03-28T16:49:37.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.934+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 93935193606656] v31'156 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.934+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93935193580544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.934+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 40 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 93935193580544] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.934+0000 7f8c2a9e4980 20 remove_watcher() i=0 2024-03-28T16:49:37.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.934+0000 7f8c2a9e4980 2 removed watcher, disabling cache 2024-03-28T16:49:37.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.934+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93935193584560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x556efdd79590 2024-03-28T16:49:37.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.938+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 93935193584560] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.938+0000 7f8c2a9e4980 20 remove_watcher() i=1 2024-03-28T16:49:37.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.938+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93935193588448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x556efdd82680 2024-03-28T16:49:37.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.938+0000 7f8c27cfe640 1 -- 172.21.15.170:0/3197727917 <== osd.3 v2:172.21.15.170:6816/2102412426 62 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 93935193588448] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c1c0b0020 con 0x556efdd82680 2024-03-28T16:49:37.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.938+0000 7f8c2a9e4980 20 remove_watcher() i=2 2024-03-28T16:49:37.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.938+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93935193585616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.942+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.2 v2:172.21.15.170:6800/447751618 41 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 93935193585616] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x7f8c1c0e0040 2024-03-28T16:49:37.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.942+0000 7f8c2a9e4980 20 remove_watcher() i=3 2024-03-28T16:49:37.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.942+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93935193593120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x556efdd79590 2024-03-28T16:49:37.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.942+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 <== osd.1 v2:172.21.15.170:6805/11349688 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 93935193593120] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c180083f0 con 0x556efdd79590 2024-03-28T16:49:37.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.942+0000 7f8c2a9e4980 20 remove_watcher() i=4 2024-03-28T16:49:37.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.942+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93935193598720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x556efdd7de10 2024-03-28T16:49:37.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.946+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 46 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 93935193598720] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.946+0000 7f8c2a9e4980 20 remove_watcher() i=5 2024-03-28T16:49:37.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.946+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93935193590704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x556efdd7de10 2024-03-28T16:49:37.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 47 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 93935193590704] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 20 remove_watcher() i=6 2024-03-28T16:49:37.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93935193605232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556efdd95ce0 con 0x556efdd7de10 2024-03-28T16:49:37.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c284ff640 1 -- 172.21.15.170:0/3197727917 <== osd.0 v2:172.21.15.119:6800/2028815515 48 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 93935193605232] v31'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8c20024070 con 0x556efdd7de10 2024-03-28T16:49:37.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 20 remove_watcher() i=7 2024-03-28T16:49:37.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd7de10 msgr2=0x556efdd80200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd7de10 0x556efdd80200 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdd79590 msgr2=0x556efdd7ba30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdd79590 0x556efdd7ba30 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8c1c0e0040 msgr2=0x7f8c1c0d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8c1c0e0040 0x7f8c1c0d3400 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556efdd82680 msgr2=0x556efdd84ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x556efdd82680 0x556efdd84ad0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8af8034260 msgr2=0x7f8af8036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8af8034260 0x7f8af8036720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c000c10 tx=0x7f8c1c0c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c28d00640 1 -- 172.21.15.170:0/3197727917 reap_dead start 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 msgr2=0x556efdd6e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd6e320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8c200469c0 tx=0x7f8c2002bf70 comp rx=0 tx=0).stop 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 shutdown_connections 2024-03-28T16:49:37.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/3197727917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd67690 0x556efdd6e320 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 >> 172.21.15.170:0/3197727917 conn(0x556efdd639d0 msgr2=0x556efdd61030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:37.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 shutdown_connections 2024-03-28T16:49:37.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.950+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/3197727917 wait complete. 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b000342b0 msgr2=0x7f8b00036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b000342b0 0x7f8b00036770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x556efdd45240 tx=0x7f8c1c005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd5a6f0 msgr2=0x556efdd5ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd5a6f0 0x556efdd5ab10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8c20047bc0 tx=0x7f8c20011680 comp rx=0 tx=0).stop 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 shutdown_connections 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b000342b0 0x7f8b00036770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/470283888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdd5a6f0 0x556efdd5ab10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 >> 172.21.15.170:0/470283888 conn(0x556efdd4f500 msgr2=0x556efdd50010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 shutdown_connections 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/470283888 wait complete. 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd46070 msgr2=0x556efdd48460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556efdd46070 0x556efdd48460 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdc50630 msgr2=0x556efdd43ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556efdc50630 0x556efdd43ab0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556efdd49d80 msgr2=0x556efdd4c170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x556efdd49d80 0x556efdd4c170 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b100342b0 msgr2=0x7f8b10036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8b100342b0 0x7f8b10036770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0ae000 tx=0x7f8c1c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 msgr2=0x556efdd3ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 --2- 172.21.15.170:0/652104308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556efdc4c130 0x556efdd3ce30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8c200012f0 tx=0x7f8c20001260 comp rx=0 tx=0).stop 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c28d00640 1 -- 172.21.15.170:0/652104308 reap_dead start 2024-03-28T16:49:37.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 shutdown_connections 2024-03-28T16:49:37.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 >> 172.21.15.170:0/652104308 conn(0x556efdbbf850 msgr2=0x556efdd30680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:37.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 shutdown_connections 2024-03-28T16:49:37.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:37.954+0000 7f8c2a9e4980 1 -- 172.21.15.170:0/652104308 wait complete. 2024-03-28T16:49:37.972 INFO:teuthology.orchestra.run.smithi170.stdout:[{"tag":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.2115683582490547709\u0000","time":"2024-03-28T16:49:37.686355+0000","objs":[{"pool":"default.rgw.buckets.data","oid":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_1","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_2","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_3","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_4","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_5","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_6","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_7","key":"","instance":""}]}] 2024-03-28T16:49:37.972 DEBUG:tasks.util.rgw: json result: [{'tag': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.2115683582490547709\x00', 'time': '2024-03-28T16:49:37.686355+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': 'b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_7', 'key': '', 'instance': ''}]}] 2024-03-28T16:49:37.972 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-03-28T16:49:37.972 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-03-28T16:49:37.973 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-03-28T16:49:38.051 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:38.052 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:38.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6d6f5980 1 Processor -- start 2024-03-28T16:49:38.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6d6f5980 1 -- start start 2024-03-28T16:49:38.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6d6f5980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b5dc3552c0 0x55b5dc3576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6d6f5980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b5dc29d510 con 0x55b5dc2c50a0 2024-03-28T16:49:38.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6d6f5980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b5dc2640f0 con 0x55b5dc3552c0 2024-03-28T16:49:38.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6aa0f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b5dc3552c0 0x55b5dc3576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6aa0f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b5dc3552c0 0x55b5dc3576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49750/0 (socket says 172.21.15.170:49750) 2024-03-28T16:49:38.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/2418821212 learned_addr learned my addr 172.21.15.170:0/2418821212 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.050+0000 7f0c6b210640 1 --1- 172.21.15.170:0/2418821212 >> v1:172.21.15.119:6789/0 conn(0x55b5dc2c50a0 0x55b5dc353ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:39090/0 (socket says 172.21.15.170:39090) 2024-03-28T16:49:38.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/2418821212 >> v1:172.21.15.119:6789/0 conn(0x55b5dc2c50a0 legacy=0x55b5dc353ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/2418821212 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dc358cc0 con 0x55b5dc3552c0 2024-03-28T16:49:38.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/2418821212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc3552c0 0x55b5dc3576b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000a5c0 tx=0x7f0c6001b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a96c550968f4268 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6a20e640 1 -- 172.21.15.170:0/2418821212 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c600aa470 con 0x55b5dc3552c0 2024-03-28T16:49:38.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6a20e640 1 -- 172.21.15.170:0/2418821212 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c600aa5d0 con 0x55b5dc3552c0 2024-03-28T16:49:38.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6a20e640 1 -- 172.21.15.170:0/2418821212 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c600aa890 con 0x55b5dc3552c0 2024-03-28T16:49:38.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2418821212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc3552c0 msgr2=0x55b5dc3576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/2418821212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc3552c0 0x55b5dc3576b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000a5c0 tx=0x7f0c6001b340 comp rx=0 tx=0).stop 2024-03-28T16:49:38.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2418821212 shutdown_connections 2024-03-28T16:49:38.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/2418821212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc3552c0 0x55b5dc3576b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2418821212 >> 172.21.15.170:0/2418821212 conn(0x55b5dc34b6c0 msgr2=0x55b5dc34dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2418821212 shutdown_connections 2024-03-28T16:49:38.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.054+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2418821212 wait complete. 2024-03-28T16:49:38.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 Processor -- start 2024-03-28T16:49:38.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- start start 2024-03-28T16:49:38.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc43aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dc261620 con 0x55b5dc2c50a0 2024-03-28T16:49:38.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc43aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc43aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49756/0 (socket says 172.21.15.170:49756) 2024-03-28T16:49:38.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6b210640 1 -- 172.21.15.170:0/4206114742 learned_addr learned my addr 172.21.15.170:0/4206114742 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6b210640 1 -- 172.21.15.170:0/4206114742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60005df0 con 0x55b5dc2c50a0 2024-03-28T16:49:38.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6b210640 1 --2- 172.21.15.170:0/4206114742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc43aec0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c006fc0 tx=0x7f0c5c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4723f60bf74f99b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0b5960a640 1 -- 172.21.15.170:0/4206114742 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c013420 con 0x55b5dc2c50a0 2024-03-28T16:49:38.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0b5960a640 1 -- 172.21.15.170:0/4206114742 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c013580 con 0x55b5dc2c50a0 2024-03-28T16:49:38.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0b5960a640 1 -- 172.21.15.170:0/4206114742 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c013880 con 0x55b5dc2c50a0 2024-03-28T16:49:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/4206114742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 msgr2=0x55b5dc43aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/4206114742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc43aec0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c006fc0 tx=0x7f0c5c0019b0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/4206114742 shutdown_connections 2024-03-28T16:49:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/4206114742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc43aec0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/4206114742 >> 172.21.15.170:0/4206114742 conn(0x55b5dc34b6c0 msgr2=0x55b5dc436a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/4206114742 shutdown_connections 2024-03-28T16:49:38.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/4206114742 wait complete. 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 Processor -- start 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- start start 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc351300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.074+0000 7f0c6d6f5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dc25ce00 con 0x55b5dc2c50a0 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc351300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc351300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49764/0 (socket says 172.21.15.170:49764) 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6b210640 1 -- 172.21.15.170:0/3856381917 learned_addr learned my addr 172.21.15.170:0/3856381917 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6b210640 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dc29d510 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6b210640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc351300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0012f0 tx=0x7f0c5c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0b57e07640 1 -- 172.21.15.170:0/3856381917 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c006320 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b5dc351840 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0b57e07640 1 -- 172.21.15.170:0/3856381917 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c006840 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0b57e07640 1 -- 172.21.15.170:0/3856381917 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c004c90 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5dc34fce0 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0b57e07640 1 -- 172.21.15.170:0/3856381917 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0c5c01e3f0 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0b57e07640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b500342b0 0x7f0b50036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0b57e07640 1 -- 172.21.15.170:0/3856381917 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0c5c0496f0 con 0x55b5dc2c50a0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b500342b0 0x7f0b50036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc3552c0 0x55b5dc44ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc44b190 con 0x55b5dc3552c0 2024-03-28T16:49:38.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6ba11640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc3552c0 0x55b5dc44ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b500342b0 0x7f0b50036770 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c600ae000 tx=0x7f0c600a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6ba11640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc3552c0 0x55b5dc44ac50 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6ba11640 1 -- 172.21.15.170:0/3856381917 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc3552c0 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.078+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc44b4f0 con 0x55b5dc3552c0 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6ba11640 1 -- 172.21.15.170:0/3856381917 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc3552c0 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc44ce40 0x55b5dc44f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc44f9f0 con 0x55b5dc44ce40 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6b210640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc44ce40 0x55b5dc44f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6b210640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc44ce40 0x55b5dc44f260 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6b210640 1 -- 172.21.15.170:0/3856381917 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc44ce40 2024-03-28T16:49:38.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc44bcf0 con 0x55b5dc3552c0 2024-03-28T16:49:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6ba11640 1 -- 172.21.15.170:0/3856381917 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc3552c0 2024-03-28T16:49:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b5dc450e10 0x55b5dc453200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4538f0 con 0x55b5dc450e10 2024-03-28T16:49:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b5dc450e10 0x55b5dc453200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b5dc450e10 0x55b5dc453200 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/3856381917 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc450e10 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6d6f5980 1 Processor -- start 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.082+0000 7f0c6d6f5980 1 -- start start 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc461820 0x55b5dc461c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dc1f5e30 con 0x55b5dc461820 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc461820 0x55b5dc461c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc461820 0x55b5dc461c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49780/0 (socket says 172.21.15.170:49780) 2024-03-28T16:49:38.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6b210640 1 -- 172.21.15.170:0/1363800130 learned_addr learned my addr 172.21.15.170:0/1363800130 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6b210640 1 -- 172.21.15.170:0/1363800130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dc462180 con 0x55b5dc461820 2024-03-28T16:49:38.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6b210640 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc461820 0x55b5dc461c40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c047bc0 tx=0x7f0c5c010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0b55602640 1 -- 172.21.15.170:0/1363800130 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c026ca0 con 0x55b5dc461820 2024-03-28T16:49:38.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0b55602640 1 -- 172.21.15.170:0/1363800130 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c026e20 con 0x55b5dc461820 2024-03-28T16:49:38.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b5dc4623d0 con 0x55b5dc461820 2024-03-28T16:49:38.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0b55602640 1 -- 172.21.15.170:0/1363800130 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c049230 con 0x55b5dc461820 2024-03-28T16:49:38.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5dc462f20 con 0x55b5dc461820 2024-03-28T16:49:38.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0b55602640 1 -- 172.21.15.170:0/1363800130 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0c5c049390 con 0x55b5dc461820 2024-03-28T16:49:38.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0b55602640 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b440342b0 0x7f0b44036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0b55602640 1 -- 172.21.15.170:0/1363800130 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0c5c03ebf0 con 0x55b5dc461820 2024-03-28T16:49:38.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b440342b0 0x7f0b44036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 Processor -- start 2024-03-28T16:49:38.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 -- start start 2024-03-28T16:49:38.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc46eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b440342b0 0x7f0b44036770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55b5dc44fd80 tx=0x7f0c60005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.086+0000 7f0c6d6f5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dc1e9790 con 0x55b5dc46e770 2024-03-28T16:49:38.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc46eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc46eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49782/0 (socket says 172.21.15.170:49782) 2024-03-28T16:49:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 -- 172.21.15.170:0/2275911104 learned_addr learned my addr 172.21.15.170:0/2275911104 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 -- 172.21.15.170:0/2275911104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dc46f120 con 0x55b5dc46e770 2024-03-28T16:49:38.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 --2- 172.21.15.170:0/2275911104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc46eb50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c026ff0 tx=0x7f0c5c027bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea27489ec987dba2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0b4effd640 1 -- 172.21.15.170:0/2275911104 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c012af0 con 0x55b5dc46e770 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0b4effd640 1 -- 172.21.15.170:0/2275911104 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c0411e0 con 0x55b5dc46e770 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0b4effd640 1 -- 172.21.15.170:0/2275911104 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c01e6f0 con 0x55b5dc46e770 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2275911104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 msgr2=0x55b5dc46eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/2275911104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc46eb50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c026ff0 tx=0x7f0c5c027bb0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2275911104 shutdown_connections 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/2275911104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc46eb50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2275911104 >> 172.21.15.170:0/2275911104 conn(0x55b5dc46ab00 msgr2=0x55b5dc468b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2275911104 shutdown_connections 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/2275911104 wait complete. 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 Processor -- start 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 -- start start 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc475450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6d6f5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dc475990 con 0x55b5dc46e770 2024-03-28T16:49:38.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc475450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc475450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49792/0 (socket says 172.21.15.170:49792) 2024-03-28T16:49:38.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 learned_addr learned my addr 172.21.15.170:0/1317095849 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.090+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dc462f20 con 0x55b5dc46e770 2024-03-28T16:49:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6b210640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc475450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c026ff0 tx=0x7f0c5c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0b4d7fa640 1 -- 172.21.15.170:0/1317095849 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c03fe40 con 0x55b5dc46e770 2024-03-28T16:49:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0b4d7fa640 1 -- 172.21.15.170:0/1317095849 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c01e6f0 con 0x55b5dc46e770 2024-03-28T16:49:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b5dc462f20 con 0x55b5dc46e770 2024-03-28T16:49:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0b4d7fa640 1 -- 172.21.15.170:0/1317095849 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c5c01c9f0 con 0x55b5dc46e770 2024-03-28T16:49:38.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5dc462f20 con 0x55b5dc46e770 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0b4d7fa640 1 -- 172.21.15.170:0/1317095849 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0c5c01c2e0 con 0x55b5dc46e770 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0b4d7fa640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b34034300 0x7f0b340367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0b4d7fa640 1 -- 172.21.15.170:0/1317095849 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0c5c06cbf0 con 0x55b5dc46e770 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b34034300 0x7f0b340367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc4806c0 0x55b5dc482b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4832b0 con 0x55b5dc4806c0 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6ba11640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc4806c0 0x55b5dc482b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b34034300 0x7f0b340367c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0c60000c10 tx=0x7f0c600c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6ba11640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc4806c0 0x55b5dc482b60 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 realm 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483680 con 0x55b5dc4806c0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.094+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc484230 con 0x55b5dc4806c0 2024-03-28T16:49:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:38.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc484be0 0x55b5dc486fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc487670 con 0x55b5dc484be0 2024-03-28T16:49:38.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6b210640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc484be0 0x55b5dc486fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6b210640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc484be0 0x55b5dc486fd0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc484be0 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 20 searching for the correct realm 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b5dc489830 0x55b5dc48bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48c370 con 0x55b5dc489830 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b5dc489830 0x55b5dc48bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b5dc489830 0x55b5dc48bcf0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6aa0f640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0c600e0040 0x7f0c600d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48c370 con 0x7f0c600e0040 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6ba11640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0c600e0040 0x7f0c600d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6ba11640 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0c600e0040 0x7f0c600d3400 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.098+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc484be0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc484be0 2024-03-28T16:49:38.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc484be0 2024-03-28T16:49:38.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x7f0c600e0040 2024-03-28T16:49:38.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.102+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc484be0 2024-03-28T16:49:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x7f0c600e0040 2024-03-28T16:49:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc484be0 2024-03-28T16:49:38.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.106+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c024070 con 0x55b5dc484be0 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x7f0c600e0040 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x7f0c600e0040 2024-03-28T16:49:38.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc489830 2024-03-28T16:49:38.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.110+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x55b5dc4806c0 2024-03-28T16:49:38.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc483ed0 con 0x7f0c600e0040 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc487c30 con 0x55b5dc4806c0 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc487c30 con 0x55b5dc4806c0 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc487be0 con 0x7f0c600e0040 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 zone default found 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 4 Realm: () 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 zonegroup default 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc488480 con 0x55b5dc484be0 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4888e0 con 0x55b5dc489830 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48e2c0 con 0x55b5dc489830 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 started sync module instance, tier type = 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.114+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48da70 con 0x7f0c600e0040 2024-03-28T16:49:38.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.118+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.118+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94239573202384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48de20 con 0x7f0c600e0040 2024-03-28T16:49:38.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.118+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48f020 con 0x55b5dc4806c0 2024-03-28T16:49:38.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.118+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94239573202384] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.118+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.118+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94239573206752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48efa0 con 0x55b5dc4806c0 2024-03-28T16:49:38.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.118+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc48f9b0 con 0x55b5dc489830 2024-03-28T16:49:38.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94239573210272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc490200 con 0x55b5dc489830 2024-03-28T16:49:38.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc490bd0 con 0x7f0c600e0040 2024-03-28T16:49:38.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94239573206752] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94239573214928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc490f80 con 0x7f0c600e0040 2024-03-28T16:49:38.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4918f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94239573210272] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.122+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94239573214928] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.126+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.126+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94239573207808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc491450 con 0x55b5dc4806c0 2024-03-28T16:49:38.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.126+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc491c50 con 0x55b5dc484be0 2024-03-28T16:49:38.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.126+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94239573207808] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.126+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.126+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94239573218384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc491870 con 0x55b5dc484be0 2024-03-28T16:49:38.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.126+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc492940 con 0x55b5dc484be0 2024-03-28T16:49:38.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.130+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94239573218384] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.130+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.130+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94239573221856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc493140 con 0x55b5dc484be0 2024-03-28T16:49:38.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.130+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc493ab0 con 0x55b5dc484be0 2024-03-28T16:49:38.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.130+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94239573221856] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94239573227024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc493ed0 con 0x55b5dc484be0 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=0 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=1 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=2 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=3 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=4 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=5 2024-03-28T16:49:38.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=6 2024-03-28T16:49:38.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94239573227024] v31'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 20 add_watcher() i=7 2024-03-28T16:49:38.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:38.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4971b0 con 0x55b5dc4806c0 2024-03-28T16:49:38.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.134+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94239573228448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc494460 con 0x55b5dc4806c0 2024-03-28T16:49:38.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.138+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94239573228448] v31'157 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.138+0000 7f0b21fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:38.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.138+0000 7f0c6d6f5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:38.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.138+0000 7f0c6d6f5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:38.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.138+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49f870 con 0x55b5dc4806c0 2024-03-28T16:49:38.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.138+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.138+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49fcb0 con 0x55b5dc484be0 2024-03-28T16:49:38.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.142+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.142+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49fcb0 con 0x55b5dc489830 2024-03-28T16:49:38.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.142+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.142+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc496540 con 0x55b5dc4806c0 2024-03-28T16:49:38.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.146+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'158 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.146+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc496540 con 0x7f0c600e0040 2024-03-28T16:49:38.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.146+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'47 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.146+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc496540 con 0x55b5dc489830 2024-03-28T16:49:38.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.146+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'31 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.146+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a11f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.150+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.150+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a11f0 con 0x55b5dc484be0 2024-03-28T16:49:38.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.150+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'31 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.150+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a1720 con 0x7f0c600e0040 2024-03-28T16:49:38.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.154+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.154+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a1720 con 0x55b5dc489830 2024-03-28T16:49:38.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.154+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.154+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a1c50 con 0x7f0c600e0040 2024-03-28T16:49:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.158+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'48 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.158+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a1c50 con 0x55b5dc484be0 2024-03-28T16:49:38.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.158+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'22 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.158+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a2180 con 0x7f0c600e0040 2024-03-28T16:49:38.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.162+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.162+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a2180 con 0x55b5dc484be0 2024-03-28T16:49:38.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.162+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'38 uv12 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.162+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a22c0 con 0x55b5dc4806c0 2024-03-28T16:49:38.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.166+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'159 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.166+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a20f0 con 0x55b5dc489830 2024-03-28T16:49:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.166+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'41 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.166+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e960 con 0x7f0c600e0040 2024-03-28T16:49:38.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.170+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'49 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.170+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e960 con 0x55b5dc489830 2024-03-28T16:49:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.170+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.170+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e960 con 0x55b5dc4806c0 2024-03-28T16:49:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.174+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.174+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc4806c0 2024-03-28T16:49:38.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.174+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'160 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.174+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc489830 2024-03-28T16:49:38.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.178+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'42 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.178+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x7f0c600e0040 2024-03-28T16:49:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.178+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'29 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.178+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc489830 2024-03-28T16:49:38.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.182+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.182+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc489830 2024-03-28T16:49:38.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.182+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.182+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc489830 2024-03-28T16:49:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.182+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.182+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc489830 2024-03-28T16:49:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.186+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'32 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.186+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc484be0 2024-03-28T16:49:38.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.190+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'22 uv20 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.190+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc489830 2024-03-28T16:49:38.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.190+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.190+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc484be0 2024-03-28T16:49:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.194+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'32 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.194+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc489830 2024-03-28T16:49:38.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.194+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'43 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.194+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x55b5dc484be0 2024-03-28T16:49:38.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.198+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'39 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.198+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e700 con 0x7f0c600e0040 2024-03-28T16:49:38.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.198+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.198+0000 7f0b20fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:38.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6d6f5980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:38.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6d6f5980 20 Filter name: none 2024-03-28T16:49:38.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0a9a7fc640 20 reqs_thread_entry: start 2024-03-28T16:49:38.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-03-28T16:49:38.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:94 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(94 gc.31 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(95 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:96 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.202+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(96 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:97 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(97 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(98 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:99 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(99 gc.31 [call] v31'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-03-28T16:49:38.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.206+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(100 gc.0 [call] v31'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:101 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(101 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:102 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(102 gc.0 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:103 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(103 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:104 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(104 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.210+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.214+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(105 gc.0 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.214+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-03-28T16:49:38.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.214+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:106 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.214+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(106 gc.1 [call] v31'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.214+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:107 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(107 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:108 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(108 gc.1 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:109 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(109 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:110 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(110 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.218+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:111 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.222+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(111 gc.1 [call] v31'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.222+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-03-28T16:49:38.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.222+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:112 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.222+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(112 gc.2 [call] v31'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.222+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:113 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(113 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:114 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(114 gc.2 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:115 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(115 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:116 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(116 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:117 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 31 ==== osd_op_reply(117 gc.2 [call] v31'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-03-28T16:49:38.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.226+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:118 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(118 gc.3 [call] v31'161 uv161 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:119 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(119 gc.3 [call out=15b] v0'0 uv161 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:120 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(120 gc.3 [call out=24b] v0'0 uv161 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:121 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 36 ==== osd_op_reply(121 gc.3 [call out=15b] v0'0 uv161 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:122 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 37 ==== osd_op_reply(122 gc.3 [call out=15b] v0'0 uv161 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.230+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:123 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 38 ==== osd_op_reply(123 gc.3 [call] v31'162 uv162 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-03-28T16:49:38.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:124 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 25 ==== osd_op_reply(124 gc.4 [call] v31'50 uv50 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:125 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 26 ==== osd_op_reply(125 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:126 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 27 ==== osd_op_reply(126 gc.4 [call out=24b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:127 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 28 ==== osd_op_reply(127 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.234+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:128 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.238+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 29 ==== osd_op_reply(128 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.238+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.238+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:129 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.238+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 30 ==== osd_op_reply(129 gc.4 [call] v31'51 uv51 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.238+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-03-28T16:49:38.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.238+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:130 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 32 ==== osd_op_reply(130 gc.5 [call] v31'33 uv33 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:131 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 33 ==== osd_op_reply(131 gc.5 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:132 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 34 ==== osd_op_reply(132 gc.5 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:133 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 35 ==== osd_op_reply(133 gc.5 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:134 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 36 ==== osd_op_reply(134 gc.5 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:135 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 37 ==== osd_op_reply(135 gc.5 [call] v31'34 uv34 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-03-28T16:49:38.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.242+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:136 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.246+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 39 ==== osd_op_reply(136 gc.6 [call] v31'22 uv22 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.246+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:137 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.246+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 40 ==== osd_op_reply(137 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.246+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.246+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:138 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.246+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 41 ==== osd_op_reply(138 gc.6 [call out=24b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.246+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:139 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 42 ==== osd_op_reply(139 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:140 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 43 ==== osd_op_reply(140 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:141 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 44 ==== osd_op_reply(141 gc.6 [call] v31'23 uv23 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-03-28T16:49:38.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.250+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:142 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.254+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(142 gc.7 [call] v31'33 uv33 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.254+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:143 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.254+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(143 gc.7 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.254+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.254+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:144 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.254+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(144 gc.7 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.254+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:145 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(145 gc.7 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:146 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(146 gc.7 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:147 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== osd_op_reply(147 gc.7 [call] v31'34 uv34 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-03-28T16:49:38.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.258+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:148 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 31 ==== osd_op_reply(148 gc.8 [call] v31'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:149 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 32 ==== osd_op_reply(149 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:150 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 33 ==== osd_op_reply(150 gc.8 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:151 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 34 ==== osd_op_reply(151 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.262+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:152 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.266+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 35 ==== osd_op_reply(152 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.266+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.266+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:153 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.266+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 36 ==== osd_op_reply(153 gc.8 [call] v31'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.266+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-03-28T16:49:38.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.266+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:154 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 38 ==== osd_op_reply(154 gc.9 [call] v31'26 uv26 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:155 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 39 ==== osd_op_reply(155 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:156 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 40 ==== osd_op_reply(156 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:157 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 41 ==== osd_op_reply(157 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:158 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 42 ==== osd_op_reply(158 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.270+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:159 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.274+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 43 ==== osd_op_reply(159 gc.9 [call] v31'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.274+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-03-28T16:49:38.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.274+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:160 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.274+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 37 ==== osd_op_reply(160 gc.10 [call] v31'52 uv52 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.274+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:161 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 38 ==== osd_op_reply(161 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:162 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 39 ==== osd_op_reply(162 gc.10 [call out=24b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:163 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 40 ==== osd_op_reply(163 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:164 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 41 ==== osd_op_reply(164 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:165 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 42 ==== osd_op_reply(165 gc.10 [call] v31'53 uv53 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-03-28T16:49:38.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.278+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:166 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.282+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(166 gc.11 [call] v31'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.282+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:167 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.282+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(167 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.282+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.282+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:168 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.282+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(168 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.282+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:169 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== osd_op_reply(169 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:170 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 41 ==== osd_op_reply(170 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:171 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 42 ==== osd_op_reply(171 gc.11 [call] v31'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.286+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-03-28T16:49:38.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:172 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 43 ==== osd_op_reply(172 gc.12 [call] v31'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:173 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 44 ==== osd_op_reply(173 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:174 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 45 ==== osd_op_reply(174 gc.12 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:175 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 46 ==== osd_op_reply(175 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.290+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:176 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.294+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 47 ==== osd_op_reply(176 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.294+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.294+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:177 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.294+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 48 ==== osd_op_reply(177 gc.12 [call] v31'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.294+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-03-28T16:49:38.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.294+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:178 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 43 ==== osd_op_reply(178 gc.13 [call] v31'40 uv40 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:179 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 44 ==== osd_op_reply(179 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:180 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 45 ==== osd_op_reply(180 gc.13 [call out=24b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:181 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 46 ==== osd_op_reply(181 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:182 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 47 ==== osd_op_reply(182 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.298+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:183 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.302+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 48 ==== osd_op_reply(183 gc.13 [call] v31'41 uv41 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.302+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-03-28T16:49:38.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.302+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:184 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 45 ==== osd_op_reply(184 gc.14 [call] v31'163 uv163 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:185 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 46 ==== osd_op_reply(185 gc.14 [call out=15b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:186 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 47 ==== osd_op_reply(186 gc.14 [call out=24b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:187 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 48 ==== osd_op_reply(187 gc.14 [call out=15b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:188 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 49 ==== osd_op_reply(188 gc.14 [call out=15b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.306+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:189 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.310+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 50 ==== osd_op_reply(189 gc.14 [call] v31'164 uv164 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.310+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-03-28T16:49:38.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.310+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:190 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.310+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 44 ==== osd_op_reply(190 gc.15 [call] v31'44 uv44 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.310+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:191 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.310+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 45 ==== osd_op_reply(191 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.310+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:192 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 46 ==== osd_op_reply(192 gc.15 [call out=24b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:193 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 47 ==== osd_op_reply(193 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:194 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 48 ==== osd_op_reply(194 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.314+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:195 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 49 ==== osd_op_reply(195 gc.15 [call] v31'45 uv45 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-03-28T16:49:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:196 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 49 ==== osd_op_reply(196 gc.16 [call] v31'54 uv54 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:197 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 50 ==== osd_op_reply(197 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:198 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 51 ==== osd_op_reply(198 gc.16 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.318+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:199 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 52 ==== osd_op_reply(199 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:200 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 53 ==== osd_op_reply(200 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:201 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 54 ==== osd_op_reply(201 gc.16 [call] v31'55 uv55 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-03-28T16:49:38.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.322+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:202 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 50 ==== osd_op_reply(202 gc.17 [call] v31'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:203 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 51 ==== osd_op_reply(203 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:204 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 52 ==== osd_op_reply(204 gc.17 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:205 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 53 ==== osd_op_reply(205 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:206 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 54 ==== osd_op_reply(206 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:207 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 55 ==== osd_op_reply(207 gc.17 [call] v31'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-03-28T16:49:38.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.326+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:208 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 51 ==== osd_op_reply(208 gc.18 [call] v31'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:209 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 52 ==== osd_op_reply(209 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:210 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 53 ==== osd_op_reply(210 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:211 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 54 ==== osd_op_reply(211 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.330+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:212 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.334+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 55 ==== osd_op_reply(212 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.334+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.334+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:213 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.334+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 56 ==== osd_op_reply(213 gc.18 [call] v31'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.334+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-03-28T16:49:38.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.334+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:214 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 57 ==== osd_op_reply(214 gc.19 [call] v31'165 uv165 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:215 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 58 ==== osd_op_reply(215 gc.19 [call out=15b] v0'0 uv165 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:216 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 59 ==== osd_op_reply(216 gc.19 [call out=24b] v0'0 uv165 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:217 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 60 ==== osd_op_reply(217 gc.19 [call out=15b] v0'0 uv165 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.338+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:218 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.342+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 61 ==== osd_op_reply(218 gc.19 [call out=15b] v0'0 uv165 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.342+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.342+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:219 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc4806c0 2024-03-28T16:49:38.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.342+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 62 ==== osd_op_reply(219 gc.19 [call] v31'166 uv166 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.342+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-03-28T16:49:38.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.342+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:220 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 56 ==== osd_op_reply(220 gc.20 [call] v31'46 uv46 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:221 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 57 ==== osd_op_reply(221 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.348 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:222 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 58 ==== osd_op_reply(222 gc.20 [call out=24b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:223 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 59 ==== osd_op_reply(223 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.346+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:224 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.350+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 60 ==== osd_op_reply(224 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.350+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.350+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:225 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.350+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 61 ==== osd_op_reply(225 gc.20 [call] v31'47 uv47 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.350+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-03-28T16:49:38.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.350+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:226 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 55 ==== osd_op_reply(226 gc.21 [call] v31'30 uv30 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:227 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 56 ==== osd_op_reply(227 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:228 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 57 ==== osd_op_reply(228 gc.21 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:229 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 58 ==== osd_op_reply(229 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:230 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 59 ==== osd_op_reply(230 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.354+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:231 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x7f0c600e0040 2024-03-28T16:49:38.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.358+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 60 ==== osd_op_reply(231 gc.21 [call] v31'31 uv31 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.358+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-03-28T16:49:38.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.358+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:232 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.358+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 62 ==== osd_op_reply(232 gc.22 [call] v31'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.358+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:233 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 63 ==== osd_op_reply(233 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:234 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 64 ==== osd_op_reply(234 gc.22 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:235 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 65 ==== osd_op_reply(235 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:236 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 66 ==== osd_op_reply(236 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.362+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:237 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 67 ==== osd_op_reply(237 gc.22 [call] v31'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-03-28T16:49:38.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:238 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 68 ==== osd_op_reply(238 gc.23 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:239 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 69 ==== osd_op_reply(239 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:240 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 70 ==== osd_op_reply(240 gc.23 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:241 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 71 ==== osd_op_reply(241 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:242 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 72 ==== osd_op_reply(242 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.366+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:243 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.370+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 73 ==== osd_op_reply(243 gc.23 [call] v31'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.370+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-03-28T16:49:38.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.370+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:244 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 74 ==== osd_op_reply(244 gc.24 [call] v31'16 uv16 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:245 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 75 ==== osd_op_reply(245 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:246 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 76 ==== osd_op_reply(246 gc.24 [call out=24b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:247 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 77 ==== osd_op_reply(247 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:248 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 78 ==== osd_op_reply(248 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.374+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:249 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 79 ==== osd_op_reply(249 gc.24 [call] v31'17 uv17 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-03-28T16:49:38.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:250 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 80 ==== osd_op_reply(250 gc.25 [call] v31'35 uv35 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:251 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 81 ==== osd_op_reply(251 gc.25 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:252 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 82 ==== osd_op_reply(252 gc.25 [call out=24b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:253 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 83 ==== osd_op_reply(253 gc.25 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.378+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:254 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.382+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 84 ==== osd_op_reply(254 gc.25 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.382+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.382+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:255 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc489830 2024-03-28T16:49:38.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.382+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 85 ==== osd_op_reply(255 gc.25 [call] v31'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.382+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-03-28T16:49:38.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.382+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:256 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 49 ==== osd_op_reply(256 gc.26 [call] v31'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:257 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 50 ==== osd_op_reply(257 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:258 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 51 ==== osd_op_reply(258 gc.26 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.388 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:259 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 52 ==== osd_op_reply(259 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.386+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:260 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc49e580 con 0x55b5dc484be0 2024-03-28T16:49:38.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 53 ==== osd_op_reply(260 gc.26 [call out=1701b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+1701 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process iterating over entry tag='b88c2392-6ef8-4660-b080-19480b748fc0.4221.2115683582490547709', time=2024-03-28T16:49:37.686355+0000, chain.objs.size()=7 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_1 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:261 3.37 3:edca1b53:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_1:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4bdc90 con 0x55b5dc4806c0 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_2 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:262 3.16 3:69a1f5fa:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_2:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c0790 con 0x7f0c600e0040 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_3 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:263 3.3 3:c034bcd3:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_3:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c3350 con 0x7f0c600e0040 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_4 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:264 3.23 3:c5d85860:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_4:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c5cb0 con 0x55b5dc484be0 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_5 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:265 3.c 3:32c693c3:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_5:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c3350 con 0x55b5dc489830 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_6 2024-03-28T16:49:38.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:266 3.3b 3:debcdfb8:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_6:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4caa50 con 0x55b5dc489830 2024-03-28T16:49:38.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_7 2024-03-28T16:49:38.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.390+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:267 3.28 3:14c5be5a:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_7:head [call refcount.put in=84b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4cd310 con 0x55b5dc489830 2024-03-28T16:49:38.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.394+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 86 ==== osd_op_reply(267 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_7 [call] v31'4 uv4 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.394+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 61 ==== osd_op_reply(262 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_2 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.394+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 63 ==== osd_op_reply(261 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_1 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.398+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 87 ==== osd_op_reply(265 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_5 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.398+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 54 ==== osd_op_reply(264 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_4 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.398+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 62 ==== osd_op_reply(263 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_3 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.398+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 88 ==== osd_op_reply(266 b88c2392-6ef8-4660-b080-19480b748fc0.4221.1__shadow_.2aPO96I7EWf9_qnvyhAb9xQ-jUUG5TN_6 [call] v31'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.398+0000 7f0c6d6f5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-03-28T16:49:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.398+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:268 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.402+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 55 ==== osd_op_reply(268 gc.26 [call] v31'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.402+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:269 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.406+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 56 ==== osd_op_reply(269 gc.26 [call] v31'25 uv25 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.406+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-03-28T16:49:38.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.406+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:270 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.406+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 89 ==== osd_op_reply(270 gc.27 [call] v31'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.406+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:271 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 90 ==== osd_op_reply(271 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:272 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 91 ==== osd_op_reply(272 gc.27 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:273 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 92 ==== osd_op_reply(273 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:274 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 93 ==== osd_op_reply(274 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:275 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 94 ==== osd_op_reply(275 gc.27 [call] v31'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-03-28T16:49:38.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.410+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:276 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.414+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 57 ==== osd_op_reply(276 gc.28 [call] v31'35 uv35 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.414+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:277 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.414+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 58 ==== osd_op_reply(277 gc.28 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.414+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.414+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:278 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.414+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 59 ==== osd_op_reply(278 gc.28 [call out=24b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.414+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:279 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.418+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 60 ==== osd_op_reply(279 gc.28 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.418+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.418+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:280 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.418+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 61 ==== osd_op_reply(280 gc.28 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.418+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.418+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:281 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc484be0 2024-03-28T16:49:38.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.422+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 62 ==== osd_op_reply(281 gc.28 [call] v31'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.422+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-03-28T16:49:38.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.422+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:282 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.422+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 95 ==== osd_op_reply(282 gc.29 [call] v31'48 uv48 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.422+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:283 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 96 ==== osd_op_reply(283 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:284 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4ca9b0 con 0x55b5dc489830 2024-03-28T16:49:38.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 97 ==== osd_op_reply(284 gc.29 [call out=24b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:285 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc489830 2024-03-28T16:49:38.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 98 ==== osd_op_reply(285 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:286 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc489830 2024-03-28T16:49:38.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 99 ==== osd_op_reply(286 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.426+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:287 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc489830 2024-03-28T16:49:38.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.430+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 100 ==== osd_op_reply(287 gc.29 [call] v31'49 uv49 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.430+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-03-28T16:49:38.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.430+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:288 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc484be0 2024-03-28T16:49:38.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.430+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 63 ==== osd_op_reply(288 gc.30 [call] v31'42 uv42 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.430+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:289 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc484be0 2024-03-28T16:49:38.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 64 ==== osd_op_reply(289 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:290 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc484be0 2024-03-28T16:49:38.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 65 ==== osd_op_reply(290 gc.30 [call out=24b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:291 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc484be0 2024-03-28T16:49:38.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 66 ==== osd_op_reply(291 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:49:38.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:292 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc484be0 2024-03-28T16:49:38.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 67 ==== osd_op_reply(292 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6d6f5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:49:38.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.434+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:293 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4c2cd0 con 0x55b5dc484be0 2024-03-28T16:49:38.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.438+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 68 ==== osd_op_reply(293 gc.30 [call] v31'43 uv43 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.438+0000 7f0b21fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:38.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.438+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94239573228448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x55b5dc4806c0 2024-03-28T16:49:38.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.442+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 64 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 94239573228448] v31'167 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.442+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94239573202384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x7f0c600e0040 2024-03-28T16:49:38.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.442+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 63 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 94239573202384] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.442+0000 7f0c6d6f5980 20 remove_watcher() i=0 2024-03-28T16:49:38.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.442+0000 7f0c6d6f5980 2 removed watcher, disabling cache 2024-03-28T16:49:38.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.442+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94239573206752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x55b5dc4806c0 2024-03-28T16:49:38.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.446+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 65 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 94239573206752] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.446+0000 7f0c6d6f5980 20 remove_watcher() i=1 2024-03-28T16:49:38.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.446+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94239573210272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x55b5dc489830 2024-03-28T16:49:38.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.450+0000 7f0c6aa0f640 1 -- 172.21.15.170:0/1317095849 <== osd.3 v2:172.21.15.170:6816/2102412426 101 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 94239573210272] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c600b0020 con 0x55b5dc489830 2024-03-28T16:49:38.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.450+0000 7f0c6d6f5980 20 remove_watcher() i=2 2024-03-28T16:49:38.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.450+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94239573214928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x7f0c600e0040 2024-03-28T16:49:38.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.450+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.2 v2:172.21.15.170:6800/447751618 64 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 94239573214928] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x7f0c600e0040 2024-03-28T16:49:38.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.450+0000 7f0c6d6f5980 20 remove_watcher() i=3 2024-03-28T16:49:38.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.450+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94239573207808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x55b5dc4806c0 2024-03-28T16:49:38.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.454+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 <== osd.1 v2:172.21.15.170:6805/11349688 66 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 94239573207808] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c640083f0 con 0x55b5dc4806c0 2024-03-28T16:49:38.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.454+0000 7f0c6d6f5980 20 remove_watcher() i=4 2024-03-28T16:49:38.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.454+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94239573218384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x55b5dc484be0 2024-03-28T16:49:38.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.454+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 69 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 94239573218384] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.454+0000 7f0c6d6f5980 20 remove_watcher() i=5 2024-03-28T16:49:38.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.454+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94239573221856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x55b5dc484be0 2024-03-28T16:49:38.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.458+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 70 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 94239573221856] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.458+0000 7f0c6d6f5980 20 remove_watcher() i=6 2024-03-28T16:49:38.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.458+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94239573227024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b5dc4a0e50 con 0x55b5dc484be0 2024-03-28T16:49:38.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.458+0000 7f0c6b210640 1 -- 172.21.15.170:0/1317095849 <== osd.0 v2:172.21.15.119:6800/2028815515 71 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 94239573227024] v31'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0c5c082040 con 0x55b5dc484be0 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.458+0000 7f0c6d6f5980 20 remove_watcher() i=7 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc484be0 msgr2=0x55b5dc486fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc484be0 0x55b5dc486fd0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc4806c0 msgr2=0x55b5dc482b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc4806c0 0x55b5dc482b60 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0c600e0040 msgr2=0x7f0c600d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0c600e0040 0x7f0c600d3400 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b5dc489830 msgr2=0x55b5dc48bcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b5dc489830 0x55b5dc48bcf0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b34034300 msgr2=0x7f0b340367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b34034300 0x7f0b340367c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0c60000c10 tx=0x7f0c600c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6ba11640 1 -- 172.21.15.170:0/1317095849 reap_dead start 2024-03-28T16:49:38.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 msgr2=0x55b5dc475450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc475450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c026ff0 tx=0x7f0c5c032f40 comp rx=0 tx=0).stop 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 shutdown_connections 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1317095849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc46e770 0x55b5dc475450 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 >> 172.21.15.170:0/1317095849 conn(0x55b5dc46ab00 msgr2=0x55b5dc468160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 shutdown_connections 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1317095849 wait complete. 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b440342b0 msgr2=0x7f0b44036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b440342b0 0x7f0b44036770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55b5dc44fd80 tx=0x7f0c60005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc461820 msgr2=0x55b5dc461c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc461820 0x55b5dc461c40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c047bc0 tx=0x7f0c5c010650 comp rx=0 tx=0).stop 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 shutdown_connections 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b440342b0 0x7f0b44036770 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/1363800130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc461820 0x55b5dc461c40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 >> 172.21.15.170:0/1363800130 conn(0x55b5dc456630 msgr2=0x55b5dc457140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 shutdown_connections 2024-03-28T16:49:38.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/1363800130 wait complete. 2024-03-28T16:49:38.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc44ce40 msgr2=0x55b5dc44f260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b5dc44ce40 0x55b5dc44f260 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc3552c0 msgr2=0x55b5dc44ac50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b5dc3552c0 0x55b5dc44ac50 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b5dc450e10 msgr2=0x55b5dc453200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b5dc450e10 0x55b5dc453200 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b500342b0 msgr2=0x7f0b50036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0b500342b0 0x7f0b50036770 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c600ae000 tx=0x7f0c600a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 msgr2=0x55b5dc351300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 --2- 172.21.15.170:0/3856381917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b5dc2c50a0 0x55b5dc351300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0012f0 tx=0x7f0c5c001260 comp rx=0 tx=0).stop 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6ba11640 1 -- 172.21.15.170:0/3856381917 reap_dead start 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 shutdown_connections 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 >> 172.21.15.170:0/3856381917 conn(0x55b5dc34b6c0 msgr2=0x55b5dc435560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 shutdown_connections 2024-03-28T16:49:38.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.462+0000 7f0c6d6f5980 1 -- 172.21.15.170:0/3856381917 wait complete. 2024-03-28T16:49:38.484 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-03-28T16:49:38.484 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-03-28T16:49:38.484 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-03-28T16:49:38.561 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:38.561 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:38.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d5d9d980 1 Processor -- start 2024-03-28T16:49:38.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d5d9d980 1 -- start start 2024-03-28T16:49:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d5d9d980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55f8dd6a0f40 0x55f8dd6a1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d5d9d980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f8dd5ed520 con 0x55f8dd6a1860 2024-03-28T16:49:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d5d9d980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f8dd5b40f0 con 0x55f8dd6a0f40 2024-03-28T16:49:38.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d38b8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55f8dd6a0f40 0x55f8dd6a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d30b7640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55f8dd6a1860 0x55f8dd6a2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:39092/0 (socket says 172.21.15.170:39092) 2024-03-28T16:49:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d38b8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55f8dd6a0f40 0x55f8dd6a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49808/0 (socket says 172.21.15.170:49808) 2024-03-28T16:49:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d30b7640 1 -- 172.21.15.170:0/4002123583 learned_addr learned my addr 172.21.15.170:0/4002123583 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d38b8640 1 -- 172.21.15.170:0/4002123583 >> v1:172.21.15.119:6789/0 conn(0x55f8dd6a1860 legacy=0x55f8dd6a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.562+0000 7fc4d38b8640 1 -- 172.21.15.170:0/4002123583 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dd6a2a50 con 0x55f8dd6a0f40 2024-03-28T16:49:38.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/4002123583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd6a1320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400a420 tx=0x7fc4c401afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374915b0e13a942c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d28b6640 1 -- 172.21.15.170:0/4002123583 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40aa530 con 0x55f8dd6a0f40 2024-03-28T16:49:38.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d28b6640 1 -- 172.21.15.170:0/4002123583 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40a6050 con 0x55f8dd6a0f40 2024-03-28T16:49:38.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d28b6640 1 -- 172.21.15.170:0/4002123583 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40aa800 con 0x55f8dd6a0f40 2024-03-28T16:49:38.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4002123583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 msgr2=0x55f8dd6a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4002123583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd6a1320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400a420 tx=0x7fc4c401afd0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4002123583 shutdown_connections 2024-03-28T16:49:38.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4002123583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd6a1320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4002123583 >> 172.21.15.170:0/4002123583 conn(0x55f8dd69c810 msgr2=0x55f8dd69ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4002123583 shutdown_connections 2024-03-28T16:49:38.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.566+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4002123583 wait complete. 2024-03-28T16:49:38.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d5d9d980 1 Processor -- start 2024-03-28T16:49:38.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d5d9d980 1 -- start start 2024-03-28T16:49:38.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d5d9d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd78b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d5d9d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dd5b1620 con 0x55f8dd6a0f40 2024-03-28T16:49:38.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd78b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd78b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49822/0 (socket says 172.21.15.170:49822) 2024-03-28T16:49:38.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d38b8640 1 -- 172.21.15.170:0/2952005150 learned_addr learned my addr 172.21.15.170:0/2952005150 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d38b8640 1 -- 172.21.15.170:0/2952005150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c400a100 con 0x55f8dd6a0f40 2024-03-28T16:49:38.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.582+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/2952005150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd78b3a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40ad040 tx=0x7fc4c40033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d979b54627d4a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc3c1e0b640 1 -- 172.21.15.170:0/2952005150 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40aa6c0 con 0x55f8dd6a0f40 2024-03-28T16:49:38.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc3c1e0b640 1 -- 172.21.15.170:0/2952005150 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40aa820 con 0x55f8dd6a0f40 2024-03-28T16:49:38.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc3c1e0b640 1 -- 172.21.15.170:0/2952005150 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40aaac0 con 0x55f8dd6a0f40 2024-03-28T16:49:38.589 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/2952005150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 msgr2=0x55f8dd78b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/2952005150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd78b3a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40ad040 tx=0x7fc4c40033b0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/2952005150 shutdown_connections 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/2952005150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd78b3a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/2952005150 >> 172.21.15.170:0/2952005150 conn(0x55f8dd69c810 msgr2=0x55f8dd786f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/2952005150 shutdown_connections 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/2952005150 wait complete. 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.586+0000 7fc4d5d9d980 1 Processor -- start 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d5d9d980 1 -- start start 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d5d9d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd7920f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d5d9d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dd5ace00 con 0x55f8dd6a0f40 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd7920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd7920f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49826/0 (socket says 172.21.15.170:49826) 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d38b8640 1 -- 172.21.15.170:0/949842355 learned_addr learned my addr 172.21.15.170:0/949842355 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d38b8640 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dd6a2a50 con 0x55f8dd6a0f40 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd7920f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4003430 tx=0x7fc4c4019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc3bbfff640 1 -- 172.21.15.170:0/949842355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40b6920 con 0x55f8dd6a0f40 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc3bbfff640 1 -- 172.21.15.170:0/949842355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40b6a80 con 0x55f8dd6a0f40 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f8dd792630 con 0x55f8dd6a0f40 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc3bbfff640 1 -- 172.21.15.170:0/949842355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40b6d20 con 0x55f8dd6a0f40 2024-03-28T16:49:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8dd792a60 con 0x55f8dd6a0f40 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc3bbfff640 1 -- 172.21.15.170:0/949842355 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc4c40c0450 con 0x55f8dd6a0f40 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc3bbfff640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3bc034300 0x7fc3bc0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc3bbfff640 1 -- 172.21.15.170:0/949842355 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc4c40e3f90 con 0x55f8dd6a0f40 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3bc034300 0x7fc3bc0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd6a1860 0x55f8dd798d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd799260 con 0x55f8dd6a1860 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d40b9640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd6a1860 0x55f8dd798d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.590+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3bc034300 0x7fc3bc0367c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8001da0 tx=0x7fc4c8001300 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d40b9640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd6a1860 0x55f8dd798d20 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d40b9640 1 -- 172.21.15.170:0/949842355 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd6a1860 2024-03-28T16:49:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7995c0 con 0x55f8dd6a1860 2024-03-28T16:49:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d40b9640 1 -- 172.21.15.170:0/949842355 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd6a1860 2024-03-28T16:49:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd79b2e0 0x55f8dd79d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd79dc10 con 0x55f8dd79b2e0 2024-03-28T16:49:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd79b2e0 0x55f8dd79d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd79b2e0 0x55f8dd79d6d0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d38b8640 1 -- 172.21.15.170:0/949842355 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fc4c40c6070 con 0x55f8dd79b2e0 2024-03-28T16:49:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd79a150 con 0x55f8dd6a1860 2024-03-28T16:49:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d40b9640 1 -- 172.21.15.170:0/949842355 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd6a1860 2024-03-28T16:49:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f8dd79efd0 0x55f8dd7a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7a1a10 con 0x55f8dd79efd0 2024-03-28T16:49:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f8dd79efd0 0x55f8dd7a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f8dd79efd0 0x55f8dd7a13c0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.594+0000 7fc4d30b7640 1 -- 172.21.15.170:0/949842355 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd79efd0 2024-03-28T16:49:38.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d5d9d980 1 Processor -- start 2024-03-28T16:49:38.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d5d9d980 1 -- start start 2024-03-28T16:49:38.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d5d9d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7af940 0x55f8dd7afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d5d9d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dd545e30 con 0x55f8dd7af940 2024-03-28T16:49:38.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7af940 0x55f8dd7afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7af940 0x55f8dd7afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49830/0 (socket says 172.21.15.170:49830) 2024-03-28T16:49:38.599 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d38b8640 1 -- 172.21.15.170:0/4141763258 learned_addr learned my addr 172.21.15.170:0/4141763258 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d38b8640 1 -- 172.21.15.170:0/4141763258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dd7b02a0 con 0x55f8dd7af940 2024-03-28T16:49:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7af940 0x55f8dd7afd60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40e2320 tx=0x7fc4c40b3f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc3b97fa640 1 -- 172.21.15.170:0/4141763258 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c4007cd0 con 0x55f8dd7af940 2024-03-28T16:49:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc3b97fa640 1 -- 172.21.15.170:0/4141763258 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40db040 con 0x55f8dd7af940 2024-03-28T16:49:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f8dd7b04f0 con 0x55f8dd7af940 2024-03-28T16:49:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc3b97fa640 1 -- 172.21.15.170:0/4141763258 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40071f0 con 0x55f8dd7af940 2024-03-28T16:49:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8dd7b1040 con 0x55f8dd7af940 2024-03-28T16:49:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc3b97fa640 1 -- 172.21.15.170:0/4141763258 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc4c4007350 con 0x55f8dd7af940 2024-03-28T16:49:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc3b97fa640 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3ac0342b0 0x7fc3ac036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc3b97fa640 1 -- 172.21.15.170:0/4141763258 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc4c40e8ba0 con 0x55f8dd7af940 2024-03-28T16:49:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.598+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3ac0342b0 0x7fc3ac036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 Processor -- start 2024-03-28T16:49:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- start start 2024-03-28T16:49:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd7bcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dd539790 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3ac0342b0 0x7fc3ac036770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55f8dd796790 tx=0x7fc4c800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd7bcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd7bcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49842/0 (socket says 172.21.15.170:49842) 2024-03-28T16:49:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 -- 172.21.15.170:0/4234403866 learned_addr learned my addr 172.21.15.170:0/4234403866 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 -- 172.21.15.170:0/4234403866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dd7bd270 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/4234403866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd7bcca0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40d5bd0 tx=0x7fc4c40d5420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1798e6829027c314 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc3b2ffd640 1 -- 172.21.15.170:0/4234403866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40076a0 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc3b2ffd640 1 -- 172.21.15.170:0/4234403866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40e2dc0 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc3b2ffd640 1 -- 172.21.15.170:0/4234403866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40cad30 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4234403866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 msgr2=0x55f8dd7bcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4234403866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd7bcca0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40d5bd0 tx=0x7fc4c40d5420 comp rx=0 tx=0).stop 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4234403866 shutdown_connections 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4234403866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd7bcca0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4234403866 >> 172.21.15.170:0/4234403866 conn(0x55f8dd7b8c00 msgr2=0x55f8dd7b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4234403866 shutdown_connections 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4234403866 wait complete. 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 Processor -- start 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- start start 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd617050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d5d9d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dd617590 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd617050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd617050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49848/0 (socket says 172.21.15.170:49848) 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.602+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 learned_addr learned my addr 172.21.15.170:0/394250188 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dd7b1040 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd617050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40a9710 tx=0x7fc4c40f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc3b17fa640 1 -- 172.21.15.170:0/394250188 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40fe040 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f8dd615770 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc3b17fa640 1 -- 172.21.15.170:0/394250188 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40076a0 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc3b17fa640 1 -- 172.21.15.170:0/394250188 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c40c1df0 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8dd615ba0 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc3b17fa640 1 -- 172.21.15.170:0/394250188 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc4c40ca7a0 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc3b17fa640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc39c034260 0x7fc39c036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc3b17fa640 1 -- 172.21.15.170:0/394250188 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc4c410fbf0 con 0x55f8dd7bc8c0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc39c034260 0x7fc39c036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd7d0cd0 0x55f8dd7d3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d38c0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d40b9640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd7d0cd0 0x55f8dd7d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc39c034260 0x7fc39c036720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55f8dd7a1dd0 tx=0x7fc4c8010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d40b9640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd7d0cd0 0x55f8dd7d3170 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 20 realm 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d3c90 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.606+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d4840 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd7d51f0 0x55f8dd7d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d7b60 con 0x55f8dd7d51f0 2024-03-28T16:49:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd7d51f0 0x55f8dd7d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d38b8640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd7d51f0 0x55f8dd7d75e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc4c40c6070 con 0x55f8dd7d51f0 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 20 searching for the correct realm 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f8dd7d9a60 0x55f8dd7dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7dc580 con 0x55f8dd7d9a60 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f8dd7d9a60 0x55f8dd7dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f8dd7d9a60 0x55f8dd7dbeb0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d30b7640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc4c8037040 0x7fc4c8036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7dc580 con 0x7fc4c8037040 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d40b9640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc4c8037040 0x7fc4c8036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.610+0000 7fc4d40b9640 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc4c8037040 0x7fc4c8036410 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c40c6070 con 0x55f8dd7d51f0 2024-03-28T16:49:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c40c6070 con 0x55f8dd7d51f0 2024-03-28T16:49:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x7fc4c8037040 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.614+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x7fc4c8037040 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.618+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x7fc4c8037040 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x7fc4c8037040 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.622+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d44e0 con 0x7fc4c8037040 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:38.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d83d0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d83d0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d83d0 con 0x7fc4c8037040 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 zone default found 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 4 Realm: () 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 zonegroup default 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d8c20 con 0x55f8dd7d51f0 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7d9000 con 0x55f8dd7d9a60 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7de4a0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 started sync module instance, tier type = 2024-03-28T16:49:38.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:38.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.626+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ddc80 con 0x7fc4c8037040 2024-03-28T16:49:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.630+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.630+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94527356262368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7de030 con 0x7fc4c8037040 2024-03-28T16:49:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.630+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7df200 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.630+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94527356262368] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.630+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.630+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94527356266688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7df220 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.630+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7dfc90 con 0x55f8dd7d9a60 2024-03-28T16:49:38.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94527356270448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e04e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94527356266688] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e0e90 con 0x7fc4c8037040 2024-03-28T16:49:38.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94527356267744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e1240 con 0x7fc4c8037040 2024-03-28T16:49:38.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e1bb0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94527356270448] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.634+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94527356267744] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.638+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.638+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94527356275120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e1710 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.638+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e2400 con 0x55f8dd7d51f0 2024-03-28T16:49:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.638+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94527356275120] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.638+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.638+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94527356280672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e1710 con 0x55f8dd7d51f0 2024-03-28T16:49:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.638+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e2c00 con 0x55f8dd7d51f0 2024-03-28T16:49:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94527356284768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e3820 con 0x55f8dd7d51f0 2024-03-28T16:49:38.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e41d0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94527356280672] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94527356284768] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94527356285824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e3d30 con 0x55f8dd7d51f0 2024-03-28T16:49:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 20 add_watcher() i=0 2024-03-28T16:49:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 20 add_watcher() i=1 2024-03-28T16:49:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 20 add_watcher() i=2 2024-03-28T16:49:38.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 20 add_watcher() i=3 2024-03-28T16:49:38.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 20 add_watcher() i=4 2024-03-28T16:49:38.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 20 add_watcher() i=5 2024-03-28T16:49:38.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.642+0000 7fc4d5d9d980 20 add_watcher() i=6 2024-03-28T16:49:38.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94527356285824] v31'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d5d9d980 20 add_watcher() i=7 2024-03-28T16:49:38.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d5d9d980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:38.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e7440 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94527356272704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e4980 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94527356272704] v31'168 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc389fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:38.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d5d9d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:38.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d5d9d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:38.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.646+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e6cd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.650+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.650+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e6830 con 0x55f8dd7d51f0 2024-03-28T16:49:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.650+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.650+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e6c70 con 0x55f8dd7d9a60 2024-03-28T16:49:38.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.654+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.654+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f0470 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.654+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'169 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.654+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f0470 con 0x7fc4c8037040 2024-03-28T16:49:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.654+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'56 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.654+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f0470 con 0x55f8dd7d9a60 2024-03-28T16:49:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.658+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'37 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.658+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f14f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.658+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.658+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f14f0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.662+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'37 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.662+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f1a20 con 0x7fc4c8037040 2024-03-28T16:49:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.662+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.662+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f1a20 con 0x55f8dd7d9a60 2024-03-28T16:49:38.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.662+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.662+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f1f50 con 0x7fc4c8037040 2024-03-28T16:49:38.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.666+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'57 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.666+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f1f50 con 0x55f8dd7d51f0 2024-03-28T16:49:38.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.666+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'25 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.666+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f2480 con 0x7fc4c8037040 2024-03-28T16:49:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.670+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.670+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f2480 con 0x55f8dd7d51f0 2024-03-28T16:49:38.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.670+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'44 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.670+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f25c0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.670+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'170 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.670+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7f23f0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.674+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'50 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.674+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7efb60 con 0x7fc4c8037040 2024-03-28T16:49:38.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.674+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'58 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.674+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7efb60 con 0x55f8dd7d9a60 2024-03-28T16:49:38.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.678+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.678+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7efb60 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.678+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.678+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.678+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'171 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.678+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7efb60 con 0x55f8dd7d9a60 2024-03-28T16:49:38.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.682+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'51 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.682+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x7fc4c8037040 2024-03-28T16:49:38.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.682+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'32 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.682+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d9a60 2024-03-28T16:49:38.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.686+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'25 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.686+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d9a60 2024-03-28T16:49:38.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.686+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.686+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d9a60 2024-03-28T16:49:38.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.686+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'18 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.686+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d9a60 2024-03-28T16:49:38.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.690+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'38 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.690+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d51f0 2024-03-28T16:49:38.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.690+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'26 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.690+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d9a60 2024-03-28T16:49:38.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.694+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.694+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d51f0 2024-03-28T16:49:38.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.694+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'38 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.694+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d9a60 2024-03-28T16:49:38.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.698+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'52 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.698+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x55f8dd7d51f0 2024-03-28T16:49:38.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.698+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'45 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.698+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ef460 con 0x7fc4c8037040 2024-03-28T16:49:38.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc388fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:38.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d5d9d980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:38.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d5d9d980 20 Filter name: none 2024-03-28T16:49:38.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc3027fc640 20 reqs_thread_entry: start 2024-03-28T16:49:38.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.702+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv162 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv162 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.706+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv162 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x7fc4c8037040 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv51 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 31 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.710+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x7fc4c8037040 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 32 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 33 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 34 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 25 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.714+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x7fc4c8037040 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 26 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 27 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 28 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x7fc4c8037040 2024-03-28T16:49:38.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 29 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.718+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 30 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv164 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv164 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv164 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 35 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.722+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 36 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 37 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 31 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x7fc4c8037040 2024-03-28T16:49:38.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 32 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 33 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 38 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 39 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 40 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.726+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv166 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv166 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv166 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 41 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 42 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 43 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 34 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.730+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x7fc4c8037040 2024-03-28T16:49:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 35 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 36 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 44 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 45 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 46 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 47 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 48 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 49 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.734+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 50 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 51 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 52 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 53 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 54 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 55 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.738+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv25 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 56 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 57 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 58 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 41 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 42 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.742+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 59 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 60 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d9a60 2024-03-28T16:49:38.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 61 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 43 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d51f0 2024-03-28T16:49:38.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 44 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x55f8dd7d51f0 2024-03-28T16:49:38.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 45 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv43 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 37 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.746+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x7fc4c8037040 2024-03-28T16:49:38.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.750+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 38 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.750+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd80d170 con 0x7fc4c8037040 2024-03-28T16:49:38.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.750+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 39 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.750+0000 7fc389fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:38.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.750+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94527356272704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7eebd0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.750+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 94527356272704] v31'172 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.750+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94527356262368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7e0040 con 0x7fc4c8037040 2024-03-28T16:49:38.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.754+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 40 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 94527356262368] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.754+0000 7fc4d5d9d980 20 remove_watcher() i=0 2024-03-28T16:49:38.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.754+0000 7fc4d5d9d980 2 removed watcher, disabling cache 2024-03-28T16:49:38.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.754+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94527356266688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7dd7e0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.754+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 94527356266688] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.754+0000 7fc4d5d9d980 20 remove_watcher() i=1 2024-03-28T16:49:38.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.754+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94527356270448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7dd7e0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.758+0000 7fc4d30b7640 1 -- 172.21.15.170:0/394250188 <== osd.3 v2:172.21.15.170:6816/2102412426 62 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 94527356270448] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c800e6c0 con 0x55f8dd7d9a60 2024-03-28T16:49:38.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.758+0000 7fc4d5d9d980 20 remove_watcher() i=2 2024-03-28T16:49:38.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.758+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94527356267744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7df770 con 0x7fc4c8037040 2024-03-28T16:49:38.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.758+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.2 v2:172.21.15.170:6800/447751618 41 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 94527356267744] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x7fc4c8037040 2024-03-28T16:49:38.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.758+0000 7fc4d5d9d980 20 remove_watcher() i=3 2024-03-28T16:49:38.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.758+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94527356275120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ded60 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.762+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 <== osd.1 v2:172.21.15.170:6805/11349688 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 94527356275120] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4cc0083f0 con 0x55f8dd7d0cd0 2024-03-28T16:49:38.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.762+0000 7fc4d5d9d980 20 remove_watcher() i=4 2024-03-28T16:49:38.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.762+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94527356280672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ded60 con 0x55f8dd7d51f0 2024-03-28T16:49:38.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.762+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 46 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 94527356280672] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.762+0000 7fc4d5d9d980 20 remove_watcher() i=5 2024-03-28T16:49:38.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.762+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94527356284768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ded60 con 0x55f8dd7d51f0 2024-03-28T16:49:38.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.766+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 47 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 94527356284768] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.766+0000 7fc4d5d9d980 20 remove_watcher() i=6 2024-03-28T16:49:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.766+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94527356285824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55f8dd7ded60 con 0x55f8dd7d51f0 2024-03-28T16:49:38.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.766+0000 7fc4d38b8640 1 -- 172.21.15.170:0/394250188 <== osd.0 v2:172.21.15.119:6800/2028815515 48 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 94527356285824] v31'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc4c4118040 con 0x55f8dd7d51f0 2024-03-28T16:49:38.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 20 remove_watcher() i=7 2024-03-28T16:49:38.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd7d51f0 msgr2=0x55f8dd7d75e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd7d51f0 0x55f8dd7d75e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd7d0cd0 msgr2=0x55f8dd7d3170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd7d0cd0 0x55f8dd7d3170 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc4c8037040 msgr2=0x7fc4c8036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc4c8037040 0x7fc4c8036410 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f8dd7d9a60 msgr2=0x55f8dd7dbeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55f8dd7d9a60 0x55f8dd7dbeb0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc39c034260 msgr2=0x7fc39c036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc39c034260 0x7fc39c036720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55f8dd7a1dd0 tx=0x7fc4c8010000 comp rx=0 tx=0).stop 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 msgr2=0x55f8dd617050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d40b9640 1 -- 172.21.15.170:0/394250188 reap_dead start 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd617050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40a9710 tx=0x7fc4c40f2000 comp rx=0 tx=0).stop 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 shutdown_connections 2024-03-28T16:49:38.773 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/394250188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7bc8c0 0x55f8dd617050 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 >> 172.21.15.170:0/394250188 conn(0x55f8dd7b8c00 msgr2=0x55f8dd7b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 shutdown_connections 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/394250188 wait complete. 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3ac0342b0 msgr2=0x7fc3ac036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3ac0342b0 0x7fc3ac036770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55f8dd796790 tx=0x7fc4c800f000 comp rx=0 tx=0).stop 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7af940 msgr2=0x55f8dd7afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7af940 0x55f8dd7afd60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40e2320 tx=0x7fc4c40b3f00 comp rx=0 tx=0).stop 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 shutdown_connections 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3ac0342b0 0x7fc3ac036770 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/4141763258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd7af940 0x55f8dd7afd60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 >> 172.21.15.170:0/4141763258 conn(0x55f8dd7a4750 msgr2=0x55f8dd7a5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 shutdown_connections 2024-03-28T16:49:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/4141763258 wait complete. 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd79b2e0 msgr2=0x55f8dd79d6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55f8dd79b2e0 0x55f8dd79d6d0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd6a1860 msgr2=0x55f8dd798d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55f8dd6a1860 0x55f8dd798d20 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f8dd79efd0 msgr2=0x55f8dd7a13c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55f8dd79efd0 0x55f8dd7a13c0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3bc034300 msgr2=0x7fc3bc0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc3bc034300 0x7fc3bc0367c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8001da0 tx=0x7fc4c8001300 comp rx=0 tx=0).stop 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 msgr2=0x55f8dd7920f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 --2- 172.21.15.170:0/949842355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8dd6a0f40 0x55f8dd7920f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4003430 tx=0x7fc4c4019810 comp rx=0 tx=0).stop 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d40b9640 1 -- 172.21.15.170:0/949842355 reap_dead start 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 shutdown_connections 2024-03-28T16:49:38.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 >> 172.21.15.170:0/949842355 conn(0x55f8dd69c810 msgr2=0x55f8dd785a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 shutdown_connections 2024-03-28T16:49:38.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.770+0000 7fc4d5d9d980 1 -- 172.21.15.170:0/949842355 wait complete. 2024-03-28T16:49:38.785 INFO:teuthology.orchestra.run.smithi170.stdout:[] 2024-03-28T16:49:38.785 DEBUG:tasks.util.rgw: json result: [] 2024-03-28T16:49:38.785 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-03-28T16:49:38.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-03-28T16:49:38.786 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-03-28T16:49:38.864 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:38.864 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:38.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.862+0000 7fb9ea63b980 1 Processor -- start 2024-03-28T16:49:38.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.862+0000 7fb9ea63b980 1 -- start start 2024-03-28T16:49:38.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9ea63b980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ddffa8c130 0x55ddffa8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9ea63b980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ddff9d6520 con 0x55ddffa8ca50 2024-03-28T16:49:38.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9ea63b980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ddff99d0f0 con 0x55ddffa8c130 2024-03-28T16:49:38.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e8156640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ddffa8c130 0x55ddffa8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e8156640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55ddffa8c130 0x55ddffa8c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49850/0 (socket says 172.21.15.170:49850) 2024-03-28T16:49:38.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e8156640 1 -- 172.21.15.170:0/1932254923 learned_addr learned my addr 172.21.15.170:0/1932254923 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e8156640 1 -- 172.21.15.170:0/1932254923 >> v1:172.21.15.119:6789/0 conn(0x55ddffa8ca50 legacy=0x55ddffa8d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e8156640 1 -- 172.21.15.170:0/1932254923 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddffa8dc30 con 0x55ddffa8c130 2024-03-28T16:49:38.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e8156640 1 --2- 172.21.15.170:0/1932254923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffa8c510 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000a5c0 tx=0x7fb9e001b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23265e4e4b59d3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e7154640 1 -- 172.21.15.170:0/1932254923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00aa470 con 0x55ddffa8c130 2024-03-28T16:49:38.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e7154640 1 -- 172.21.15.170:0/1932254923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00aa5d0 con 0x55ddffa8c130 2024-03-28T16:49:38.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.866+0000 7fb9e7154640 1 -- 172.21.15.170:0/1932254923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00aa890 con 0x55ddffa8c130 2024-03-28T16:49:38.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.870+0000 7fb9ea63b980 1 -- 172.21.15.170:0/1932254923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 msgr2=0x55ddffa8c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.870+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/1932254923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffa8c510 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000a5c0 tx=0x7fb9e001b340 comp rx=0 tx=0).stop 2024-03-28T16:49:38.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.870+0000 7fb9ea63b980 1 -- 172.21.15.170:0/1932254923 shutdown_connections 2024-03-28T16:49:38.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.870+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/1932254923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffa8c510 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.870+0000 7fb9ea63b980 1 -- 172.21.15.170:0/1932254923 >> 172.21.15.170:0/1932254923 conn(0x55ddff9ff850 msgr2=0x55ddff9ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.870+0000 7fb9ea63b980 1 -- 172.21.15.170:0/1932254923 shutdown_connections 2024-03-28T16:49:38.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.870+0000 7fb9ea63b980 1 -- 172.21.15.170:0/1932254923 wait complete. 2024-03-28T16:49:38.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 Processor -- start 2024-03-28T16:49:38.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 -- start start 2024-03-28T16:49:38.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb76600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddff99a620 con 0x55ddffa8c130 2024-03-28T16:49:38.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb76600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb76600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49866/0 (socket says 172.21.15.170:49866) 2024-03-28T16:49:38.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9e8156640 1 -- 172.21.15.170:0/668436932 learned_addr learned my addr 172.21.15.170:0/668436932 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9e8156640 1 -- 172.21.15.170:0/668436932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0005df0 con 0x55ddffa8c130 2024-03-28T16:49:38.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9e8156640 1 --2- 172.21.15.170:0/668436932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb76600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0018f70 tx=0x7fb9e0003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d53a35b86e86bc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb8d660c640 1 -- 172.21.15.170:0/668436932 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00aa9c0 con 0x55ddffa8c130 2024-03-28T16:49:38.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb8d660c640 1 -- 172.21.15.170:0/668436932 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00aab20 con 0x55ddffa8c130 2024-03-28T16:49:38.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb8d660c640 1 -- 172.21.15.170:0/668436932 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00aadc0 con 0x55ddffa8c130 2024-03-28T16:49:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 -- 172.21.15.170:0/668436932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 msgr2=0x55ddffb76600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/668436932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb76600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0018f70 tx=0x7fb9e0003030 comp rx=0 tx=0).stop 2024-03-28T16:49:38.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 -- 172.21.15.170:0/668436932 shutdown_connections 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/668436932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb76600 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 -- 172.21.15.170:0/668436932 >> 172.21.15.170:0/668436932 conn(0x55ddff9ff850 msgr2=0x55ddffb72190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 -- 172.21.15.170:0/668436932 shutdown_connections 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.886+0000 7fb9ea63b980 1 -- 172.21.15.170:0/668436932 wait complete. 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 Processor -- start 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 -- start start 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb7d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddff995e00 con 0x55ddffa8c130 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb7d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49868/0 (socket says 172.21.15.170:49868) 2024-03-28T16:49:38.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9e8156640 1 -- 172.21.15.170:0/79112632 learned_addr learned my addr 172.21.15.170:0/79112632 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9e8156640 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddffa8dc30 con 0x55ddffa8c130 2024-03-28T16:49:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9e8156640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb7d350 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0018f70 tx=0x7fb9e00180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb8d4e09640 1 -- 172.21.15.170:0/79112632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00cb070 con 0x55ddffa8c130 2024-03-28T16:49:38.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ddffb7d890 con 0x55ddffa8c130 2024-03-28T16:49:38.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb8d4e09640 1 -- 172.21.15.170:0/79112632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00c6690 con 0x55ddffa8c130 2024-03-28T16:49:38.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb8d4e09640 1 -- 172.21.15.170:0/79112632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00c6930 con 0x55ddffa8c130 2024-03-28T16:49:38.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddffb7dcc0 con 0x55ddffa8c130 2024-03-28T16:49:38.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb8d4e09640 1 -- 172.21.15.170:0/79112632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb9e00c6a90 con 0x55ddffa8c130 2024-03-28T16:49:38.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb8d4e09640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8d0034300 0x7fb8d00367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb8d4e09640 1 -- 172.21.15.170:0/79112632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb9e00e4960 con 0x55ddffa8c130 2024-03-28T16:49:38.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9e7955640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8d0034300 0x7fb8d00367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffa8ca50 0x55ddffb83f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.890+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffb844c0 con 0x55ddffa8ca50 2024-03-28T16:49:38.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8957640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffa8ca50 0x55ddffb83f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e7955640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8d0034300 0x7fb8d00367c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0001e20 tx=0x7fb9d00013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8957640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffa8ca50 0x55ddffb83f80 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8957640 1 -- 172.21.15.170:0/79112632 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffa8ca50 2024-03-28T16:49:38.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffb84820 con 0x55ddffa8ca50 2024-03-28T16:49:38.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8957640 1 -- 172.21.15.170:0/79112632 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffa8ca50 2024-03-28T16:49:38.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffb861b0 0x55ddffb885a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffb88d30 con 0x55ddffb861b0 2024-03-28T16:49:38.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8156640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffb861b0 0x55ddffb885a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8156640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffb861b0 0x55ddffb885a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8156640 1 -- 172.21.15.170:0/79112632 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fb9e00cd070 con 0x55ddffb861b0 2024-03-28T16:49:38.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffb85020 con 0x55ddffa8ca50 2024-03-28T16:49:38.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e8957640 1 -- 172.21.15.170:0/79112632 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffa8ca50 2024-03-28T16:49:38.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ddffb8a150 0x55ddffb8c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffb8cc00 con 0x55ddffb8a150 2024-03-28T16:49:38.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e7955640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ddffb8a150 0x55ddffb8c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.894+0000 7fb9e7955640 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ddffb8a150 0x55ddffb8c540 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9e7955640 1 -- 172.21.15.170:0/79112632 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fb9d0010630 con 0x55ddffb8a150 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9ea63b980 1 Processor -- start 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9ea63b980 1 -- start start 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9ea63b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffb9ab30 0x55ddffb9af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9ea63b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddff92ee30 con 0x55ddffb9ab30 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffb9ab30 0x55ddffb9af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffb9ab30 0x55ddffb9af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49878/0 (socket says 172.21.15.170:49878) 2024-03-28T16:49:38.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9e8156640 1 -- 172.21.15.170:0/3130982332 learned_addr learned my addr 172.21.15.170:0/3130982332 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9e8156640 1 -- 172.21.15.170:0/3130982332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddffb9b490 con 0x55ddffb9ab30 2024-03-28T16:49:38.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9e8156640 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffb9ab30 0x55ddffb9af50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00c6fd0 tx=0x7fb9e00d0bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb8cdffb640 1 -- 172.21.15.170:0/3130982332 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00e4cf0 con 0x55ddffb9ab30 2024-03-28T16:49:38.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb8cdffb640 1 -- 172.21.15.170:0/3130982332 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00e2040 con 0x55ddffb9ab30 2024-03-28T16:49:38.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ddffb9b6e0 con 0x55ddffb9ab30 2024-03-28T16:49:38.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb8cdffb640 1 -- 172.21.15.170:0/3130982332 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00c5760 con 0x55ddffb9ab30 2024-03-28T16:49:38.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.898+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddffb9c230 con 0x55ddffb9ab30 2024-03-28T16:49:38.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb8cdffb640 1 -- 172.21.15.170:0/3130982332 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb9e00c58c0 con 0x55ddffb9ab30 2024-03-28T16:49:38.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb8cdffb640 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb8cdffb640 1 -- 172.21.15.170:0/3130982332 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb9e00e4040 con 0x55ddffb9ab30 2024-03-28T16:49:38.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9e7955640 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 Processor -- start 2024-03-28T16:49:38.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- start start 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffba7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddff922790 con 0x55ddffba7ab0 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9e7955640 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55ddffb890c0 tx=0x7fb9d0006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffba7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffba7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49892/0 (socket says 172.21.15.170:49892) 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9e8156640 1 -- 172.21.15.170:0/529582031 learned_addr learned my addr 172.21.15.170:0/529582031 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9e8156640 1 -- 172.21.15.170:0/529582031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddffba8460 con 0x55ddffba7ab0 2024-03-28T16:49:38.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9e8156640 1 --2- 172.21.15.170:0/529582031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffba7e90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00c56f0 tx=0x7fb9e00cd4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6f183fbd8a18e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb8c77fe640 1 -- 172.21.15.170:0/529582031 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00fa050 con 0x55ddffba7ab0 2024-03-28T16:49:38.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb8c77fe640 1 -- 172.21.15.170:0/529582031 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00fb040 con 0x55ddffba7ab0 2024-03-28T16:49:38.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb8c77fe640 1 -- 172.21.15.170:0/529582031 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00aac90 con 0x55ddffba7ab0 2024-03-28T16:49:38.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- 172.21.15.170:0/529582031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 msgr2=0x55ddffba7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:38.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/529582031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffba7e90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00c56f0 tx=0x7fb9e00cd4a0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- 172.21.15.170:0/529582031 shutdown_connections 2024-03-28T16:49:38.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/529582031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffba7e90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- 172.21.15.170:0/529582031 >> 172.21.15.170:0/529582031 conn(0x55ddffba3df0 msgr2=0x55ddffba1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- 172.21.15.170:0/529582031 shutdown_connections 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- 172.21.15.170:0/529582031 wait complete. 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 Processor -- start 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- start start 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffbae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.902+0000 7fb9ea63b980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddffbaee00 con 0x55ddffba7ab0 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffbae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e8156640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffbae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49902/0 (socket says 172.21.15.170:49902) 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 learned_addr learned my addr 172.21.15.170:0/825553845 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:38.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddffb9c230 con 0x55ddffba7ab0 2024-03-28T16:49:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e8156640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffbae8c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00bf040 tx=0x7fb9e00daf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb8c5ffb640 1 -- 172.21.15.170:0/825553845 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e0002de0 con 0x55ddffba7ab0 2024-03-28T16:49:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ddffb9c230 con 0x55ddffba7ab0 2024-03-28T16:49:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb8c5ffb640 1 -- 172.21.15.170:0/825553845 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00dc420 con 0x55ddffba7ab0 2024-03-28T16:49:38.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb8c5ffb640 1 -- 172.21.15.170:0/825553845 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb9e00dfdc0 con 0x55ddffba7ab0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddffbaf1e0 con 0x55ddffba7ab0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb8c5ffb640 1 -- 172.21.15.170:0/825553845 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb9e00d41b0 con 0x55ddffba7ab0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb8c5ffb640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b8034260 0x7fb8b8036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb8c5ffb640 1 -- 172.21.15.170:0/825553845 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb9e00f0430 con 0x55ddffba7ab0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e7955640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b8034260 0x7fb8b8036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffbb9b20 0x55ddffbbbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbc730 con 0x55ddffbb9b20 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e8957640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffbb9b20 0x55ddffbbbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e7955640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b8034260 0x7fb8b8036720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55ddffb8cfc0 tx=0x7fb9d0011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.906+0000 7fb9e8957640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffbb9b20 0x55ddffbbbfe0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 realm 2024-03-28T16:49:38.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbcb00 con 0x55ddffbb9b20 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd6e0 con 0x55ddffbb9b20 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:38.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffbbe090 0x55ddffbc0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc0a90 con 0x55ddffbbe090 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e8156640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffbbe090 0x55ddffbc0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e8156640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffbbe090 0x55ddffbc0480 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 20 searching for the correct realm 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ddffbc2c60 0x55ddffbc5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc57f0 con 0x55ddffbc2c60 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e7955640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ddffbc2c60 0x55ddffbc5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e7955640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ddffbc2c60 0x55ddffbc5120 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.910+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e7955640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9d000f3d0 0x7fb9d000f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:38.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc57f0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9d000f3d0 0x7fb9d000f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:38.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9d000f3d0 0x7fb9d000f820 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:38.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x7fb9d000f3d0 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.914+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x7fb9d000f3d0 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.918+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x7fb9d000f3d0 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x7fb9d000f3d0 2024-03-28T16:49:38.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.922+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbc2c60 2024-03-28T16:49:38.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbbe090 2024-03-28T16:49:38.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x55ddffbb9b20 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbbd380 con 0x7fb9d000f3d0 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc1040 con 0x55ddffbb9b20 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc1040 con 0x55ddffbb9b20 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc1040 con 0x7fb9d000f3d0 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 zone default found 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 4 Realm: () 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:38.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 zonegroup default 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc1890 con 0x55ddffbbe090 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc1cc0 con 0x55ddffbc2c60 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc7790 con 0x55ddffbc2c60 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 started sync module instance, tier type = 2024-03-28T16:49:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.926+0000 7fb9ea63b980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.930+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc6ef0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.930+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.930+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94411966671440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc72a0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.930+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc84f0 con 0x55ddffbb9b20 2024-03-28T16:49:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.930+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94411966671440] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.930+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94411966675888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc8470 con 0x55ddffbb9b20 2024-03-28T16:49:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc8f10 con 0x55ddffbc2c60 2024-03-28T16:49:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94411966675888] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94411966676944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc83f0 con 0x55ddffbc2c60 2024-03-28T16:49:38.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc9c00 con 0x7fb9d000f3d0 2024-03-28T16:49:38.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94411966683280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbca450 con 0x7fb9d000f3d0 2024-03-28T16:49:38.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.934+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbcae00 con 0x55ddffbb9b20 2024-03-28T16:49:38.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.938+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94411966676944] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.938+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94411966683280] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.938+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.938+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94411966681792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc9f60 con 0x55ddffbb9b20 2024-03-28T16:49:38.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.938+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbcb650 con 0x55ddffbbe090 2024-03-28T16:49:38.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.938+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94411966681792] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.938+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.942+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94411966689712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc9f60 con 0x55ddffbbe090 2024-03-28T16:49:38.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.942+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbcbe50 con 0x55ddffbbe090 2024-03-28T16:49:38.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.942+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94411966689712] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.942+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.942+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94411966687520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbcc650 con 0x55ddffbbe090 2024-03-28T16:49:38.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.942+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbccfc0 con 0x55ddffbbe090 2024-03-28T16:49:38.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94411966687520] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94411966694288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbccfc0 con 0x55ddffbbe090 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=0 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=1 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=2 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=3 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=4 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=5 2024-03-28T16:49:38.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=6 2024-03-28T16:49:38.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94411966694288] v31'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 20 add_watcher() i=7 2024-03-28T16:49:38.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:38.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd03b0 con 0x55ddffbb9b20 2024-03-28T16:49:38.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.946+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94411966696688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbcd8f0 con 0x55ddffbb9b20 2024-03-28T16:49:38.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.950+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94411966696688] v31'173 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.950+0000 7fb89e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:38.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.950+0000 7fb9ea63b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:38.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.950+0000 7fb9ea63b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:38.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.950+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd89c0 con 0x55ddffbb9b20 2024-03-28T16:49:38.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.950+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.950+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd8a10 con 0x55ddffbbe090 2024-03-28T16:49:38.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.954+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.954+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd8f00 con 0x55ddffbc2c60 2024-03-28T16:49:38.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.954+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.954+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd98d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.954+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'174 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.954+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd6a40 con 0x7fb9d000f3d0 2024-03-28T16:49:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.958+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'59 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.958+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd6a40 con 0x55ddffbc2c60 2024-03-28T16:49:38.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.958+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'39 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.958+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbda610 con 0x55ddffbb9b20 2024-03-28T16:49:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.958+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.958+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbda610 con 0x55ddffbbe090 2024-03-28T16:49:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.962+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'39 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.962+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdab40 con 0x7fb9d000f3d0 2024-03-28T16:49:38.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.962+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.962+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdab40 con 0x55ddffbc2c60 2024-03-28T16:49:38.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.966+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.966+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdb070 con 0x7fb9d000f3d0 2024-03-28T16:49:38.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.966+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'60 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.966+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdb070 con 0x55ddffbbe090 2024-03-28T16:49:38.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.970+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'26 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.970+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdb5a0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.970+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.971 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.970+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdb5a0 con 0x55ddffbbe090 2024-03-28T16:49:38.973 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.970+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'46 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.974 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.974+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdba40 con 0x55ddffbb9b20 2024-03-28T16:49:38.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.974+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'175 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.975 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.974+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdb070 con 0x55ddffbc2c60 2024-03-28T16:49:38.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.974+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'53 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.977 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.974+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdba40 con 0x7fb9d000f3d0 2024-03-28T16:49:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.978+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'61 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.978+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdba40 con 0x55ddffbc2c60 2024-03-28T16:49:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.978+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.978+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdba40 con 0x55ddffbb9b20 2024-03-28T16:49:38.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.978+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.981 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.978+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdba40 con 0x55ddffbb9b20 2024-03-28T16:49:38.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.982+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'176 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.982+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdba40 con 0x55ddffbc2c60 2024-03-28T16:49:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.982+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'54 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.982+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.986+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'33 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.986+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x55ddffbc2c60 2024-03-28T16:49:38.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.986+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'26 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.986+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x55ddffbc2c60 2024-03-28T16:49:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.986+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.986+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x55ddffbc2c60 2024-03-28T16:49:38.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.990+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'19 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.990+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x55ddffbc2c60 2024-03-28T16:49:38.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.990+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'40 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.990+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x55ddffbbe090 2024-03-28T16:49:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.990+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'27 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.990+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x55ddffbc2c60 2024-03-28T16:49:38.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.994+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.994+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbb30 con 0x55ddffbbe090 2024-03-28T16:49:38.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.994+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'40 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:38.996 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.994+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbb30 con 0x55ddffbc2c60 2024-03-28T16:49:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.998+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'55 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.998+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x55ddffbbe090 2024-03-28T16:49:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.998+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'47 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:38.998+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbdbad0 con 0x7fb9d000f3d0 2024-03-28T16:49:39.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:39.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb89d7e2640 20 reqs_thread_entry: start 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 20 Filter name: none 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb80effd640 20 reqs_thread_entry: start 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd4d60 con 0x55ddffbb9b20 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbf8130 con 0x7fb9d000f3d0 2024-03-28T16:49:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv28 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-03-28T16:49:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb89e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.002+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94411966696688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbd50e0 con 0x55ddffbb9b20 2024-03-28T16:49:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.006+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94411966696688] v31'177 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.006+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94411966671440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbcccc0 con 0x7fb9d000f3d0 2024-03-28T16:49:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.006+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94411966671440] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x7fb9d000f3d0 2024-03-28T16:49:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.006+0000 7fb9ea63b980 20 remove_watcher() i=0 2024-03-28T16:49:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.006+0000 7fb9ea63b980 2 removed watcher, disabling cache 2024-03-28T16:49:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.006+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94411966675888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc6a50 con 0x55ddffbb9b20 2024-03-28T16:49:39.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.010+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94411966675888] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d800a4d0 con 0x55ddffbb9b20 2024-03-28T16:49:39.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.010+0000 7fb9ea63b980 20 remove_watcher() i=1 2024-03-28T16:49:39.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.010+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94411966676944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc6a50 con 0x55ddffbc2c60 2024-03-28T16:49:39.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.010+0000 7fb9e7955640 1 -- 172.21.15.170:0/825553845 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94411966676944] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d0032040 con 0x55ddffbc2c60 2024-03-28T16:49:39.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.010+0000 7fb9ea63b980 20 remove_watcher() i=2 2024-03-28T16:49:39.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.010+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94411966683280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc8050 con 0x7fb9d000f3d0 2024-03-28T16:49:39.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.014+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94411966683280] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d8054040 con 0x7fb9d000f3d0 2024-03-28T16:49:39.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.014+0000 7fb9ea63b980 20 remove_watcher() i=3 2024-03-28T16:49:39.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.014+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94411966681792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc8050 con 0x55ddffbb9b20 2024-03-28T16:49:39.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.014+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94411966681792] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9d8054040 con 0x55ddffbb9b20 2024-03-28T16:49:39.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.014+0000 7fb9ea63b980 20 remove_watcher() i=4 2024-03-28T16:49:39.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.014+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94411966689712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc8050 con 0x55ddffbbe090 2024-03-28T16:49:39.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.018+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94411966689712] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:39.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.018+0000 7fb9ea63b980 20 remove_watcher() i=5 2024-03-28T16:49:39.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.018+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94411966687520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc8050 con 0x55ddffbbe090 2024-03-28T16:49:39.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.018+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94411966687520] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:39.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.018+0000 7fb9ea63b980 20 remove_watcher() i=6 2024-03-28T16:49:39.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.018+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94411966694288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55ddffbc8050 con 0x55ddffbbe090 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9e8156640 1 -- 172.21.15.170:0/825553845 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94411966694288] v31'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb9e0107040 con 0x55ddffbbe090 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 20 remove_watcher() i=7 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffbbe090 msgr2=0x55ddffbc0480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffbbe090 0x55ddffbc0480 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffbb9b20 msgr2=0x55ddffbbbfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffbb9b20 0x55ddffbbbfe0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9d000f3d0 msgr2=0x7fb9d000f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb9d000f3d0 0x7fb9d000f820 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ddffbc2c60 msgr2=0x55ddffbc5120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55ddffbc2c60 0x55ddffbc5120 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b8034260 msgr2=0x7fb8b8036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8b8034260 0x7fb8b8036720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55ddffb8cfc0 tx=0x7fb9d0011000 comp rx=0 tx=0).stop 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 msgr2=0x55ddffbae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffbae8c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00bf040 tx=0x7fb9e00daf60 comp rx=0 tx=0).stop 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9e8957640 1 -- 172.21.15.170:0/825553845 reap_dead start 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 shutdown_connections 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/825553845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffba7ab0 0x55ddffbae8c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 >> 172.21.15.170:0/825553845 conn(0x55ddffba3df0 msgr2=0x55ddffba1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 shutdown_connections 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/825553845 wait complete. 2024-03-28T16:49:39.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 msgr2=0x7fb8c0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55ddffb890c0 tx=0x7fb9d0006000 comp rx=0 tx=0).stop 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffb9ab30 msgr2=0x55ddffb9af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffb9ab30 0x55ddffb9af50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00c6fd0 tx=0x7fb9e00d0bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 shutdown_connections 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8c00342b0 0x7fb8c0036770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/3130982332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffb9ab30 0x55ddffb9af50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 >> 172.21.15.170:0/3130982332 conn(0x55ddffb8f940 msgr2=0x55ddffb90450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 shutdown_connections 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/3130982332 wait complete. 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffb861b0 msgr2=0x55ddffb885a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55ddffb861b0 0x55ddffb885a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffa8ca50 msgr2=0x55ddffb83f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55ddffa8ca50 0x55ddffb83f80 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ddffb8a150 msgr2=0x55ddffb8c540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55ddffb8a150 0x55ddffb8c540 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8d0034300 msgr2=0x7fb8d00367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8d0034300 0x7fb8d00367c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0001e20 tx=0x7fb9d00013a0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 msgr2=0x55ddffb7d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 --2- 172.21.15.170:0/79112632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ddffa8c130 0x55ddffb7d350 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0018f70 tx=0x7fb9e00180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9e8957640 1 -- 172.21.15.170:0/79112632 reap_dead start 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 shutdown_connections 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 >> 172.21.15.170:0/79112632 conn(0x55ddff9ff850 msgr2=0x55ddffb70ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 shutdown_connections 2024-03-28T16:49:39.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.022+0000 7fb9ea63b980 1 -- 172.21.15.170:0/79112632 wait complete. 2024-03-28T16:49:39.035 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-03-28T16:49:39.046 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-03-28T16:49:39.052 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:39.093 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:39.111 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:39.159 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-03-28T16:49:39.168 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:39.168 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-03-28T16:49:39.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-03-28T16:49:39.169 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-03-28T16:49:39.208 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:39.208 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:39.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.206+0000 7fba20b88980 1 Processor -- start 2024-03-28T16:49:39.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.206+0000 7fba20b88980 1 -- start start 2024-03-28T16:49:39.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.206+0000 7fba20b88980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x557d044c1360 0x557d044c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.206+0000 7fba20b88980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x557d044094c0 con 0x557d044318c0 2024-03-28T16:49:39.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.206+0000 7fba20b88980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x557d043d00f0 con 0x557d044c1360 2024-03-28T16:49:39.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1dea2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x557d044c1360 0x557d044c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1dea2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x557d044c1360 0x557d044c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49904/0 (socket says 172.21.15.170:49904) 2024-03-28T16:49:39.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1dea2640 1 -- 172.21.15.170:0/2523924221 learned_addr learned my addr 172.21.15.170:0/2523924221 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1e6a3640 1 --1- 172.21.15.170:0/2523924221 >> v1:172.21.15.119:6789/0 conn(0x557d044318c0 0x557d044bfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:39110/0 (socket says 172.21.15.170:39110) 2024-03-28T16:49:39.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1dea2640 1 -- 172.21.15.170:0/2523924221 >> v1:172.21.15.119:6789/0 conn(0x557d044318c0 legacy=0x557d044bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1dea2640 1 -- 172.21.15.170:0/2523924221 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d044c4cd0 con 0x557d044c1360 2024-03-28T16:49:39.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1dea2640 1 --2- 172.21.15.170:0/2523924221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044c1360 0x557d044c3750 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fba0800a5c0 tx=0x7fba0801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d798537944e4534 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1d6a1640 1 -- 172.21.15.170:0/2523924221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba080aa470 con 0x557d044c1360 2024-03-28T16:49:39.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1d6a1640 1 -- 172.21.15.170:0/2523924221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba080aa5d0 con 0x557d044c1360 2024-03-28T16:49:39.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.210+0000 7fba1d6a1640 1 -- 172.21.15.170:0/2523924221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba080aa890 con 0x557d044c1360 2024-03-28T16:49:39.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.214+0000 7fba20b88980 1 -- 172.21.15.170:0/2523924221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044c1360 msgr2=0x557d044c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.214+0000 7fba20b88980 1 --2- 172.21.15.170:0/2523924221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044c1360 0x557d044c3750 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fba0800a5c0 tx=0x7fba0801b340 comp rx=0 tx=0).stop 2024-03-28T16:49:39.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.214+0000 7fba20b88980 1 -- 172.21.15.170:0/2523924221 shutdown_connections 2024-03-28T16:49:39.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.214+0000 7fba20b88980 1 --2- 172.21.15.170:0/2523924221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044c1360 0x557d044c3750 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.214+0000 7fba20b88980 1 -- 172.21.15.170:0/2523924221 >> 172.21.15.170:0/2523924221 conn(0x557d044b7720 msgr2=0x557d044b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.214+0000 7fba20b88980 1 -- 172.21.15.170:0/2523924221 shutdown_connections 2024-03-28T16:49:39.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.214+0000 7fba20b88980 1 -- 172.21.15.170:0/2523924221 wait complete. 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 Processor -- start 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- start start 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d044bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d043cd620 con 0x557d044318c0 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d044bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d044bd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49906/0 (socket says 172.21.15.170:49906) 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1965626141 learned_addr learned my addr 172.21.15.170:0/1965626141 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1965626141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba08005df0 con 0x557d044318c0 2024-03-28T16:49:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/1965626141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d044bd3a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fba18001cc0 tx=0x7fba1800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef76fc329b7685dd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fb90c608640 1 -- 172.21.15.170:0/1965626141 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba18014070 con 0x557d044318c0 2024-03-28T16:49:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fb90c608640 1 -- 172.21.15.170:0/1965626141 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800f590 con 0x557d044318c0 2024-03-28T16:49:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fb90c608640 1 -- 172.21.15.170:0/1965626141 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba18007040 con 0x557d044318c0 2024-03-28T16:49:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- 172.21.15.170:0/1965626141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 msgr2=0x557d044bd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 --2- 172.21.15.170:0/1965626141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d044bd3a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fba18001cc0 tx=0x7fba1800b610 comp rx=0 tx=0).stop 2024-03-28T16:49:39.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- 172.21.15.170:0/1965626141 shutdown_connections 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 --2- 172.21.15.170:0/1965626141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d044bd3a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- 172.21.15.170:0/1965626141 >> 172.21.15.170:0/1965626141 conn(0x557d044b7720 msgr2=0x557d045a2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- 172.21.15.170:0/1965626141 shutdown_connections 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- 172.21.15.170:0/1965626141 wait complete. 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 Processor -- start 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- start start 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d045afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.230+0000 7fba20b88980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d043c8e00 con 0x557d044318c0 2024-03-28T16:49:39.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d045afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d045afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49914/0 (socket says 172.21.15.170:49914) 2024-03-28T16:49:39.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1e6a3640 1 -- 172.21.15.170:0/539458598 learned_addr learned my addr 172.21.15.170:0/539458598 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1e6a3640 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d044094c0 con 0x557d044318c0 2024-03-28T16:49:39.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d045afff0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x557d044bcbd0 tx=0x7fba18000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fb90ae05640 1 -- 172.21.15.170:0/539458598 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba18014030 con 0x557d044318c0 2024-03-28T16:49:39.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fb90ae05640 1 -- 172.21.15.170:0/539458598 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1801c840 con 0x557d044318c0 2024-03-28T16:49:39.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fb90ae05640 1 -- 172.21.15.170:0/539458598 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba1801cb40 con 0x557d044318c0 2024-03-28T16:49:39.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557d045b0530 con 0x557d044318c0 2024-03-28T16:49:39.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d045b0960 con 0x557d044318c0 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fb90ae05640 1 -- 172.21.15.170:0/539458598 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fba1801cca0 con 0x557d044318c0 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fb90ae05640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb9040342b0 0x7fb904036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fb90ae05640 1 -- 172.21.15.170:0/539458598 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fba18006070 con 0x557d044318c0 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1dea2640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb9040342b0 0x7fb904036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d044c1360 0x557d045b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045b71b0 con 0x557d044c1360 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1eea4640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d044c1360 0x557d045b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1dea2640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb9040342b0 0x7fb904036770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba080ae000 tx=0x7fba080a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1eea4640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d044c1360 0x557d045b6c70 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba1eea4640 1 -- 172.21.15.170:0/539458598 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d044c1360 2024-03-28T16:49:39.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.234+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045b7510 con 0x557d044c1360 2024-03-28T16:49:39.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1eea4640 1 -- 172.21.15.170:0/539458598 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d044c1360 2024-03-28T16:49:39.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045b9230 0x557d045bb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045bbb60 con 0x557d045b9230 2024-03-28T16:49:39.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045b9230 0x557d045bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045b9230 0x557d045bb620 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1e6a3640 1 -- 172.21.15.170:0/539458598 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fba18016070 con 0x557d045b9230 2024-03-28T16:49:39.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045b8010 con 0x557d044c1360 2024-03-28T16:49:39.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1eea4640 1 -- 172.21.15.170:0/539458598 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d044c1360 2024-03-28T16:49:39.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x557d045bced0 0x557d045bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045bf8d0 con 0x557d045bced0 2024-03-28T16:49:39.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1dea2640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x557d045bced0 0x557d045bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1dea2640 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x557d045bced0 0x557d045bf2c0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba1dea2640 1 -- 172.21.15.170:0/539458598 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fba080b0020 con 0x557d045bced0 2024-03-28T16:49:39.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 Processor -- start 2024-03-28T16:49:39.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 -- start start 2024-03-28T16:49:39.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045cd780 0x557d045cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.238+0000 7fba20b88980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d04361e30 con 0x557d045cd780 2024-03-28T16:49:39.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045cd780 0x557d045cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045cd780 0x557d045cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49928/0 (socket says 172.21.15.170:49928) 2024-03-28T16:49:39.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba1e6a3640 1 -- 172.21.15.170:0/2521567567 learned_addr learned my addr 172.21.15.170:0/2521567567 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba1e6a3640 1 -- 172.21.15.170:0/2521567567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d045ce0e0 con 0x557d045cd780 2024-03-28T16:49:39.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045cd780 0x557d045cdba0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fba18000f70 tx=0x7fba180187d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fb903fff640 1 -- 172.21.15.170:0/2521567567 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba18017c90 con 0x557d045cd780 2024-03-28T16:49:39.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fb903fff640 1 -- 172.21.15.170:0/2521567567 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba18017df0 con 0x557d045cd780 2024-03-28T16:49:39.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557d045ce330 con 0x557d045cd780 2024-03-28T16:49:39.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fb903fff640 1 -- 172.21.15.170:0/2521567567 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba180419d0 con 0x557d045cd780 2024-03-28T16:49:39.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d045cee80 con 0x557d045cd780 2024-03-28T16:49:39.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fb903fff640 1 -- 172.21.15.170:0/2521567567 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fba18041b30 con 0x557d045cd780 2024-03-28T16:49:39.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fb903fff640 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8f80342b0 0x7fb8f8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fb903fff640 1 -- 172.21.15.170:0/2521567567 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fba18045050 con 0x557d045cd780 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba1dea2640 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8f80342b0 0x7fb8f8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba20b88980 1 Processor -- start 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba20b88980 1 -- start start 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba1dea2640 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8f80342b0 0x7fb8f8036770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x557d045b8370 tx=0x7fba08005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba20b88980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045daac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.242+0000 7fba20b88980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d04355790 con 0x557d045da6e0 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045daac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045daac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49936/0 (socket says 172.21.15.170:49936) 2024-03-28T16:49:39.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1909852255 learned_addr learned my addr 172.21.15.170:0/1909852255 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1909852255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d045db090 con 0x557d045da6e0 2024-03-28T16:49:39.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/1909852255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045daac0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fba18011980 tx=0x7fba180412d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d20a4c55fc27a2f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fb901ffb640 1 -- 172.21.15.170:0/1909852255 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba1803cc90 con 0x557d045da6e0 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fb901ffb640 1 -- 172.21.15.170:0/1909852255 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba18046b40 con 0x557d045da6e0 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fb901ffb640 1 -- 172.21.15.170:0/1909852255 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba18049800 con 0x557d045da6e0 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- 172.21.15.170:0/1909852255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 msgr2=0x557d045daac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 --2- 172.21.15.170:0/1909852255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045daac0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fba18011980 tx=0x7fba180412d0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- 172.21.15.170:0/1909852255 shutdown_connections 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 --2- 172.21.15.170:0/1909852255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045daac0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- 172.21.15.170:0/1909852255 >> 172.21.15.170:0/1909852255 conn(0x557d045d6a20 msgr2=0x557d045d4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- 172.21.15.170:0/1909852255 shutdown_connections 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- 172.21.15.170:0/1909852255 wait complete. 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 Processor -- start 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- start start 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045e1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d045e1ac0 con 0x557d045da6e0 2024-03-28T16:49:39.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045e1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:49942/0 (socket says 172.21.15.170:49942) 2024-03-28T16:49:39.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 learned_addr learned my addr 172.21.15.170:0/1468315873 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d045cee80 con 0x557d045da6e0 2024-03-28T16:49:39.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045e1580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fba1800ffd0 tx=0x7fba18028f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fb8effff640 1 -- 172.21.15.170:0/1468315873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba18064070 con 0x557d045da6e0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557d045e1cc0 con 0x557d045da6e0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fb8effff640 1 -- 172.21.15.170:0/1468315873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba18041670 con 0x557d045da6e0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fb8effff640 1 -- 172.21.15.170:0/1468315873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba18046950 con 0x557d045da6e0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.246+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d045e2150 con 0x557d045da6e0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fb8effff640 1 -- 172.21.15.170:0/1468315873 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fba1805b020 con 0x557d045da6e0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fb8effff640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8e8034300 0x7fb8e80367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fb8effff640 1 -- 172.21.15.170:0/1468315873 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fba18069bc0 con 0x557d045da6e0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1dea2640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8e8034300 0x7fb8e80367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d045ec8b0 0x557d045eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ef4a0 con 0x557d045ec8b0 2024-03-28T16:49:39.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1eea4640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d045ec8b0 0x557d045eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1dea2640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8e8034300 0x7fb8e80367c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba0801b380 tx=0x7fba080c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1eea4640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d045ec8b0 0x557d045eed50 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 realm 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ef870 con 0x557d045ec8b0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f0420 con 0x557d045ec8b0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045f0dd0 0x557d045f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.250+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f3810 con 0x557d045f0dd0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045f0dd0 0x557d045f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1e6a3640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045f0dd0 0x557d045f31c0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba20b88980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba20b88980 20 searching for the correct realm 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x557d045f59f0 0x557d045f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f8530 con 0x557d045f59f0 2024-03-28T16:49:39.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1dea2640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x557d045f59f0 0x557d045f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1dea2640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x557d045f59f0 0x557d045f7eb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1dea2640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fba080e0040 0x7fba080d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f8530 con 0x7fba080e0040 2024-03-28T16:49:39.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1eea4640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fba080e0040 0x7fba080d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1eea4640 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fba080e0040 0x7fba080d3400 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.254+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x7fba080e0040 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x7fba080e0040 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.258+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x7fba080e0040 2024-03-28T16:49:39.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.262+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x7fba080e0040 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f59f0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045f0dd0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x557d045ec8b0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f00c0 con 0x7fba080e0040 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f3df0 con 0x557d045ec8b0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f3df0 con 0x557d045ec8b0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f3df0 con 0x7fba080e0040 2024-03-28T16:49:39.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.266+0000 7fba20b88980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 zone default found 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 4 Realm: () 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 zonegroup default 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f4640 con 0x557d045f0dd0 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f4a70 con 0x557d045f59f0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fa420 con 0x557d045f59f0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 started sync module instance, tier type = 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:39.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f9c30 con 0x7fba080e0040 2024-03-28T16:49:39.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93995432646544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045f9fe0 con 0x7fba080e0040 2024-03-28T16:49:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.270+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fb180 con 0x557d045ec8b0 2024-03-28T16:49:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93995432646544] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93995432650816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fb100 con 0x557d045ec8b0 2024-03-28T16:49:39.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fbbd0 con 0x557d045f59f0 2024-03-28T16:49:39.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93995432650816] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93995432651872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fb080 con 0x557d045f59f0 2024-03-28T16:49:39.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.274+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fc8c0 con 0x7fba080e0040 2024-03-28T16:49:39.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93995432658224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fd110 con 0x7fba080e0040 2024-03-28T16:49:39.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fdae0 con 0x557d045ec8b0 2024-03-28T16:49:39.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93995432651872] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93995432658224] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93995432662496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fcbf0 con 0x557d045ec8b0 2024-03-28T16:49:39.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.278+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fe330 con 0x557d045f0dd0 2024-03-28T16:49:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.282+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93995432662496] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.282+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.282+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93995432664720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fcbf0 con 0x557d045f0dd0 2024-03-28T16:49:39.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.282+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045feb30 con 0x557d045f0dd0 2024-03-28T16:49:39.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.282+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93995432664720] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.282+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93995432656768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ff330 con 0x557d045f0dd0 2024-03-28T16:49:39.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ffca0 con 0x557d045f0dd0 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93995432656768] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93995432668816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ffe80 con 0x557d045f0dd0 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=0 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=1 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=2 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=3 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=4 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=5 2024-03-28T16:49:39.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=6 2024-03-28T16:49:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93995432668816] v31'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 20 add_watcher() i=7 2024-03-28T16:49:39.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:39.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.286+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d046030b0 con 0x557d045ec8b0 2024-03-28T16:49:39.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93995432671808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d04600300 con 0x557d045ec8b0 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93995432671808] v31'178 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fb8d4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 5 note: GC not initialized 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fb84f7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 Filter name: none 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fb84d7fa640 20 reqs_thread_entry: start 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d046255e0 con 0x7fba080e0040 2024-03-28T16:49:39.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:39.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:39.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 : miss 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:63 8.11 8:8b360716:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d04626eb0 con 0x557d045f59f0 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(63 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 info.flags=0x17 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 to cache LRU end 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 get_obj_state: octx=0x7fba1000aac0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x557d04629318 s->prefetch_data=0 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 3.39 3:9e9db662:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d0462a470 con 0x557d045f59f0 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(64 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=848b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 262+0+864 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 10 manifest: total_size = 5 2024-03-28T16:49:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.290+0000 7fba20b88980 20 get_obj_state: setting s->obj_tag to _T_1UzmKXWCJygCWb5b5SAfWwI_1nGkR 2024-03-28T16:49:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fb8d4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93995432671808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d04602b90 con 0x557d045ec8b0 2024-03-28T16:49:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93995432671808] v31'179 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93995432646544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ffa40 con 0x7fba080e0040 2024-03-28T16:49:39.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93995432646544] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fba20b88980 20 remove_watcher() i=0 2024-03-28T16:49:39.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fba20b88980 2 removed watcher, disabling cache 2024-03-28T16:49:39.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.294+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93995432650816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ffa40 con 0x557d045ec8b0 2024-03-28T16:49:39.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.298+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93995432650816] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.298+0000 7fba20b88980 20 remove_watcher() i=1 2024-03-28T16:49:39.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.298+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93995432651872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fa840 con 0x557d045f59f0 2024-03-28T16:49:39.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.298+0000 7fba1dea2640 1 -- 172.21.15.170:0/1468315873 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93995432651872] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba080b0020 con 0x557d045f59f0 2024-03-28T16:49:39.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.298+0000 7fba20b88980 20 remove_watcher() i=2 2024-03-28T16:49:39.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.298+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93995432658224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ffa40 con 0x7fba080e0040 2024-03-28T16:49:39.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.302+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93995432658224] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x7fba080e0040 2024-03-28T16:49:39.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.302+0000 7fba20b88980 20 remove_watcher() i=3 2024-03-28T16:49:39.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.302+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93995432662496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fc530 con 0x557d045ec8b0 2024-03-28T16:49:39.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.302+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93995432662496] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba1000a4d0 con 0x557d045ec8b0 2024-03-28T16:49:39.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.302+0000 7fba20b88980 20 remove_watcher() i=4 2024-03-28T16:49:39.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.302+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93995432664720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045ffa40 con 0x557d045f0dd0 2024-03-28T16:49:39.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.306+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93995432664720] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.306+0000 7fba20b88980 20 remove_watcher() i=5 2024-03-28T16:49:39.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.306+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93995432656768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fd5e0 con 0x557d045f0dd0 2024-03-28T16:49:39.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.306+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93995432656768] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.306+0000 7fba20b88980 20 remove_watcher() i=6 2024-03-28T16:49:39.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.306+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93995432668816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x557d045fc530 con 0x557d045f0dd0 2024-03-28T16:49:39.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba1e6a3640 1 -- 172.21.15.170:0/1468315873 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93995432668816] v31'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fba18016070 con 0x557d045f0dd0 2024-03-28T16:49:39.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 20 remove_watcher() i=7 2024-03-28T16:49:39.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045f0dd0 msgr2=0x557d045f31c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045f0dd0 0x557d045f31c0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d045ec8b0 msgr2=0x557d045eed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d045ec8b0 0x557d045eed50 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fba080e0040 msgr2=0x7fba080d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fba080e0040 0x7fba080d3400 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x557d045f59f0 msgr2=0x557d045f7eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x557d045f59f0 0x557d045f7eb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8e8034300 msgr2=0x7fb8e80367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8e8034300 0x7fb8e80367c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba0801b380 tx=0x7fba080c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba1eea4640 1 -- 172.21.15.170:0/1468315873 reap_dead start 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 msgr2=0x557d045e1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045e1580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fba1800ffd0 tx=0x7fba18028f00 comp rx=0 tx=0).stop 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 shutdown_connections 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/1468315873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045da6e0 0x557d045e1580 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 >> 172.21.15.170:0/1468315873 conn(0x557d045d6a20 msgr2=0x557d045d4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 shutdown_connections 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/1468315873 wait complete. 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8f80342b0 msgr2=0x7fb8f8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8f80342b0 0x7fb8f8036770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x557d045b8370 tx=0x7fba08005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045cd780 msgr2=0x557d045cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045cd780 0x557d045cdba0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fba18000f70 tx=0x7fba180187d0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 shutdown_connections 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb8f80342b0 0x7fb8f8036770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/2521567567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d045cd780 0x557d045cdba0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 >> 172.21.15.170:0/2521567567 conn(0x557d045c2590 msgr2=0x557d045c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 shutdown_connections 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/2521567567 wait complete. 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045b9230 msgr2=0x557d045bb620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x557d045b9230 0x557d045bb620 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d044c1360 msgr2=0x557d045b6c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x557d044c1360 0x557d045b6c70 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x557d045bced0 msgr2=0x557d045bf2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x557d045bced0 0x557d045bf2c0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb9040342b0 msgr2=0x7fb904036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb9040342b0 0x7fb904036770 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba080ae000 tx=0x7fba080a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 msgr2=0x557d045afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 --2- 172.21.15.170:0/539458598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557d044318c0 0x557d045afff0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x557d044bcbd0 tx=0x7fba18000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba1eea4640 1 -- 172.21.15.170:0/539458598 reap_dead start 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 shutdown_connections 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 >> 172.21.15.170:0/539458598 conn(0x557d044b7720 msgr2=0x557d044bbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 shutdown_connections 2024-03-28T16:49:39.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.310+0000 7fba20b88980 1 -- 172.21.15.170:0/539458598 wait complete. 2024-03-28T16:49:39.322 INFO:teuthology.orchestra.run.smithi170.stdout:fooFoofooFooFULL_CONTROL 2024-03-28T16:49:39.322 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-03-28T16:49:39.325 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-03-28T16:49:39.330 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:39.330 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-03-28T16:49:39.330 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-03-28T16:49:39.331 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-03-28T16:49:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:39.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f3066d8c980 1 Processor -- start 2024-03-28T16:49:39.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f3066d8c980 1 -- start start 2024-03-28T16:49:39.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f3066d8c980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c450f422e0 0x55c450f446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f3066d8c980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c450e8a4c0 con 0x55c450eb20c0 2024-03-28T16:49:39.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f3066d8c980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c450e510f0 con 0x55c450f422e0 2024-03-28T16:49:39.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f30640a6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c450f422e0 0x55c450f446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f30640a6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c450f422e0 0x55c450f446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34872/0 (socket says 172.21.15.170:34872) 2024-03-28T16:49:39.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f30640a6640 1 -- 172.21.15.170:0/2811752991 learned_addr learned my addr 172.21.15.170:0/2811752991 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f30648a7640 1 --1- 172.21.15.170:0/2811752991 >> v1:172.21.15.119:6789/0 conn(0x55c450eb20c0 0x55c450f40bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:41210/0 (socket says 172.21.15.170:41210) 2024-03-28T16:49:39.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f30640a6640 1 -- 172.21.15.170:0/2811752991 >> v1:172.21.15.119:6789/0 conn(0x55c450eb20c0 legacy=0x55c450f40bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.402+0000 7f30640a6640 1 -- 172.21.15.170:0/2811752991 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c450f45ce0 con 0x55c450f422e0 2024-03-28T16:49:39.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f30640a6640 1 --2- 172.21.15.170:0/2811752991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450f422e0 0x55c450f446d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f304c00a5c0 tx=0x7f304c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd6f0e7516adafa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f30638a5640 1 -- 172.21.15.170:0/2811752991 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f304c0aa470 con 0x55c450f422e0 2024-03-28T16:49:39.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f30638a5640 1 -- 172.21.15.170:0/2811752991 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c0aa5d0 con 0x55c450f422e0 2024-03-28T16:49:39.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f30638a5640 1 -- 172.21.15.170:0/2811752991 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f304c0aa890 con 0x55c450f422e0 2024-03-28T16:49:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f3066d8c980 1 -- 172.21.15.170:0/2811752991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450f422e0 msgr2=0x55c450f446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2811752991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450f422e0 0x55c450f446d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f304c00a5c0 tx=0x7f304c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f3066d8c980 1 -- 172.21.15.170:0/2811752991 shutdown_connections 2024-03-28T16:49:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2811752991 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450f422e0 0x55c450f446d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f3066d8c980 1 -- 172.21.15.170:0/2811752991 >> 172.21.15.170:0/2811752991 conn(0x55c450f386e0 msgr2=0x55c450f3ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f3066d8c980 1 -- 172.21.15.170:0/2811752991 shutdown_connections 2024-03-28T16:49:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.406+0000 7f3066d8c980 1 -- 172.21.15.170:0/2811752991 wait complete. 2024-03-28T16:49:39.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f3066d8c980 1 Processor -- start 2024-03-28T16:49:39.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f3066d8c980 1 -- start start 2024-03-28T16:49:39.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f3066d8c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c451027f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f3066d8c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c450e4e620 con 0x55c450eb20c0 2024-03-28T16:49:39.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c451027f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c451027f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34882/0 (socket says 172.21.15.170:34882) 2024-03-28T16:49:39.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f30648a7640 1 -- 172.21.15.170:0/3024013701 learned_addr learned my addr 172.21.15.170:0/3024013701 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f30648a7640 1 -- 172.21.15.170:0/3024013701 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c005df0 con 0x55c450eb20c0 2024-03-28T16:49:39.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.422+0000 7f30648a7640 1 --2- 172.21.15.170:0/3024013701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c451027f40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3054001cc0 tx=0x7f305400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb7fb1b92b16d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f2f52e0d640 1 -- 172.21.15.170:0/3024013701 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054014070 con 0x55c450eb20c0 2024-03-28T16:49:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f2f52e0d640 1 -- 172.21.15.170:0/3024013701 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305400f590 con 0x55c450eb20c0 2024-03-28T16:49:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f2f52e0d640 1 -- 172.21.15.170:0/3024013701 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054007040 con 0x55c450eb20c0 2024-03-28T16:49:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- 172.21.15.170:0/3024013701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 msgr2=0x55c451027f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 --2- 172.21.15.170:0/3024013701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c451027f40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3054001cc0 tx=0x7f305400b610 comp rx=0 tx=0).stop 2024-03-28T16:49:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- 172.21.15.170:0/3024013701 shutdown_connections 2024-03-28T16:49:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 --2- 172.21.15.170:0/3024013701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c451027f40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- 172.21.15.170:0/3024013701 >> 172.21.15.170:0/3024013701 conn(0x55c450f386e0 msgr2=0x55c451023ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- 172.21.15.170:0/3024013701 shutdown_connections 2024-03-28T16:49:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- 172.21.15.170:0/3024013701 wait complete. 2024-03-28T16:49:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 Processor -- start 2024-03-28T16:49:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- start start 2024-03-28T16:49:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c450f3e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c450e49e00 con 0x55c450eb20c0 2024-03-28T16:49:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c450f3e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c450f3e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34886/0 (socket says 172.21.15.170:34886) 2024-03-28T16:49:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f30648a7640 1 -- 172.21.15.170:0/830670187 learned_addr learned my addr 172.21.15.170:0/830670187 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f30648a7640 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c450e8a4c0 con 0x55c450eb20c0 2024-03-28T16:49:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f30648a7640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c450f3e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f305400b650 tx=0x7f3054000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f2f5160a640 1 -- 172.21.15.170:0/830670187 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054014030 con 0x55c450eb20c0 2024-03-28T16:49:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f2f5160a640 1 -- 172.21.15.170:0/830670187 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305401c650 con 0x55c450eb20c0 2024-03-28T16:49:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c450f3e860 con 0x55c450eb20c0 2024-03-28T16:49:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f2f5160a640 1 -- 172.21.15.170:0/830670187 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305401c970 con 0x55c450eb20c0 2024-03-28T16:49:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.426+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c450f3cd00 con 0x55c450eb20c0 2024-03-28T16:49:39.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f2f5160a640 1 -- 172.21.15.170:0/830670187 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f305401cbe0 con 0x55c450eb20c0 2024-03-28T16:49:39.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f2f5160a640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f4c034300 0x7f2f4c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f2f5160a640 1 -- 172.21.15.170:0/830670187 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f3054006070 con 0x55c450eb20c0 2024-03-28T16:49:39.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30640a6640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f4c034300 0x7f2f4c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c450f422e0 0x55c451037cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451038210 con 0x55c450f422e0 2024-03-28T16:49:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30650a8640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c450f422e0 0x55c451037cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30640a6640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f4c034300 0x7f2f4c0367c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f304c0ae000 tx=0x7f304c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30650a8640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c450f422e0 0x55c451037cd0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30650a8640 1 -- 172.21.15.170:0/830670187 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c450f422e0 2024-03-28T16:49:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451038570 con 0x55c450f422e0 2024-03-28T16:49:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30650a8640 1 -- 172.21.15.170:0/830670187 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c450f422e0 2024-03-28T16:49:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451039ec0 0x55c45103c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45103ca70 con 0x55c451039ec0 2024-03-28T16:49:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30648a7640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451039ec0 0x55c45103c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30648a7640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451039ec0 0x55c45103c2e0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f30648a7640 1 -- 172.21.15.170:0/830670187 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f3054016070 con 0x55c451039ec0 2024-03-28T16:49:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.430+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451038d70 con 0x55c450f422e0 2024-03-28T16:49:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30650a8640 1 -- 172.21.15.170:0/830670187 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c450f422e0 2024-03-28T16:49:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c45103de90 0x55c451040280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451040970 con 0x55c45103de90 2024-03-28T16:49:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30640a6640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c45103de90 0x55c451040280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30640a6640 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c45103de90 0x55c451040280 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30640a6640 1 -- 172.21.15.170:0/830670187 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f304c0b0020 con 0x55c45103de90 2024-03-28T16:49:39.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 Processor -- start 2024-03-28T16:49:39.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 -- start start 2024-03-28T16:49:39.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45104e8a0 0x55c45104ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c450de2e30 con 0x55c45104e8a0 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45104e8a0 0x55c45104ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45104e8a0 0x55c45104ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34894/0 (socket says 172.21.15.170:34894) 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30648a7640 1 -- 172.21.15.170:0/1615576804 learned_addr learned my addr 172.21.15.170:0/1615576804 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30648a7640 1 -- 172.21.15.170:0/1615576804 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45104f200 con 0x55c45104e8a0 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f30648a7640 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45104e8a0 0x55c45104ecc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3054049010 tx=0x7f30540187d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f2f4a7fc640 1 -- 172.21.15.170:0/1615576804 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30540258f0 con 0x55c45104e8a0 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f2f4a7fc640 1 -- 172.21.15.170:0/1615576804 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30540112f0 con 0x55c45104e8a0 2024-03-28T16:49:39.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c45104f450 con 0x55c45104e8a0 2024-03-28T16:49:39.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f2f4a7fc640 1 -- 172.21.15.170:0/1615576804 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054011590 con 0x55c45104e8a0 2024-03-28T16:49:39.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.434+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45104ffa0 con 0x55c45104e8a0 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f2f4a7fc640 1 -- 172.21.15.170:0/1615576804 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3054041350 con 0x55c45104e8a0 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f2f4a7fc640 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f3c0342b0 0x7f2f3c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f2f4a7fc640 1 -- 172.21.15.170:0/1615576804 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f305403bbb0 con 0x55c45104e8a0 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f30640a6640 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f3c0342b0 0x7f2f3c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 Processor -- start 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- start start 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c45105bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f30640a6640 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f3c0342b0 0x7f2f3c036770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55c45103ce00 tx=0x7f304c005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c450dd6790 con 0x55c45105b770 2024-03-28T16:49:39.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c45105bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c45105bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34900/0 (socket says 172.21.15.170:34900) 2024-03-28T16:49:39.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f30648a7640 1 -- 172.21.15.170:0/4163471593 learned_addr learned my addr 172.21.15.170:0/4163471593 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f30648a7640 1 -- 172.21.15.170:0/4163471593 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45105c140 con 0x55c45105b770 2024-03-28T16:49:39.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f30648a7640 1 --2- 172.21.15.170:0/4163471593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c45105bb70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f30540119a0 tx=0x7f305402f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e2bd40b86352f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f2f3bfff640 1 -- 172.21.15.170:0/4163471593 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054041c70 con 0x55c45105b770 2024-03-28T16:49:39.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f2f3bfff640 1 -- 172.21.15.170:0/4163471593 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3054041750 con 0x55c45105b770 2024-03-28T16:49:39.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f2f3bfff640 1 -- 172.21.15.170:0/4163471593 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054019c80 con 0x55c45105b770 2024-03-28T16:49:39.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- 172.21.15.170:0/4163471593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 msgr2=0x55c45105bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 --2- 172.21.15.170:0/4163471593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c45105bb70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f30540119a0 tx=0x7f305402f4c0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- 172.21.15.170:0/4163471593 shutdown_connections 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 --2- 172.21.15.170:0/4163471593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c45105bb70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- 172.21.15.170:0/4163471593 >> 172.21.15.170:0/4163471593 conn(0x55c451057b00 msgr2=0x55c451055b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- 172.21.15.170:0/4163471593 shutdown_connections 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- 172.21.15.170:0/4163471593 wait complete. 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 Processor -- start 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.438+0000 7f3066d8c980 1 -- start start 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f3066d8c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c4510625f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f3066d8c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c451062b30 con 0x55c45105b770 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c4510625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30648a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c4510625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34902/0 (socket says 172.21.15.170:34902) 2024-03-28T16:49:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 learned_addr learned my addr 172.21.15.170:0/2410287509 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45104ffa0 con 0x55c45105b770 2024-03-28T16:49:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30648a7640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c4510625f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f305402ffd0 tx=0x7f3054009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f2f3a7fc640 1 -- 172.21.15.170:0/2410287509 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054041c70 con 0x55c45105b770 2024-03-28T16:49:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c451062d30 con 0x55c45105b770 2024-03-28T16:49:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f2f3a7fc640 1 -- 172.21.15.170:0/2410287509 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3054032a50 con 0x55c45105b770 2024-03-28T16:49:39.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f2f3a7fc640 1 -- 172.21.15.170:0/2410287509 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054032cf0 con 0x55c45105b770 2024-03-28T16:49:39.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c451062d30 con 0x55c45105b770 2024-03-28T16:49:39.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f2f3a7fc640 1 -- 172.21.15.170:0/2410287509 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3054067020 con 0x55c45105b770 2024-03-28T16:49:39.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f2f3a7fc640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f300342b0 0x7f2f30036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f2f3a7fc640 1 -- 172.21.15.170:0/2410287509 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f305406bbd0 con 0x55c45105b770 2024-03-28T16:49:39.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30640a6640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f300342b0 0x7f2f30036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c45106da40 0x55c45106ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451070650 con 0x55c45106da40 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30650a8640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c45106da40 0x55c45106ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.442+0000 7f30640a6640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f300342b0 0x7f2f30036770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f304c018f70 tx=0x7f304c0c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30650a8640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c45106da40 0x55c45106ff00 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 realm 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451070a20 con 0x55c45106da40 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:39.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451071600 con 0x55c45106da40 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451071fb0 0x55c4510743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510749b0 con 0x55c451071fb0 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30648a7640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451071fb0 0x55c4510743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30648a7640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451071fb0 0x55c4510743a0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 20 searching for the correct realm 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c451076b90 0x55c451079050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451079720 con 0x55c451076b90 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30640a6640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c451076b90 0x55c451079050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.446+0000 7f30640a6640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c451076b90 0x55c451079050 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30640a6640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f304c0e0040 0x7f304c0d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451079720 con 0x7f304c0e0040 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f304c0e0040 0x7f304c0d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f304c0e0040 0x7f304c0d3400 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x7f304c0e0040 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.450+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x7f304c0e0040 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.454+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x7f304c0e0040 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x7f304c0e0040 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451076b90 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.458+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c451071fb0 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x55c45106da40 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510712a0 con 0x7f304c0e0040 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451074f90 con 0x55c45106da40 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451074f90 con 0x55c45106da40 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451074f60 con 0x7f304c0e0040 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 zone default found 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 4 Realm: () 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:39.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:39.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 zonegroup default 2024-03-28T16:49:39.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510757e0 con 0x55c451071fb0 2024-03-28T16:49:39.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451075c10 con 0x55c451076b90 2024-03-28T16:49:39.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107b640 con 0x55c451076b90 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 started sync module instance, tier type = 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.462+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107ae20 con 0x7f304c0e0040 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.466+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.466+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94301661407616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107b1d0 con 0x7f304c0e0040 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.466+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107c3a0 con 0x55c45106da40 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.466+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94301661407616] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.466+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.466+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94301661411936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107c320 con 0x55c45106da40 2024-03-28T16:49:39.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.466+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107cda0 con 0x55c451076b90 2024-03-28T16:49:39.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94301661415584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107d620 con 0x55c451076b90 2024-03-28T16:49:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107dff0 con 0x7f304c0e0040 2024-03-28T16:49:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94301661411936] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94301661420272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107e3a0 con 0x7f304c0e0040 2024-03-28T16:49:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107ed10 con 0x55c45106da40 2024-03-28T16:49:39.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.470+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94301661415584] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94301661420272] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94301661412992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107e870 con 0x55c45106da40 2024-03-28T16:49:39.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107f070 con 0x55c451071fb0 2024-03-28T16:49:39.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94301661412992] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94301661423728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107ec90 con 0x55c451071fb0 2024-03-28T16:49:39.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.474+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107fd90 con 0x55c451071fb0 2024-03-28T16:49:39.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94301661423728] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94301661427168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107ffe0 con 0x55c451071fb0 2024-03-28T16:49:39.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451080ab0 con 0x55c451071fb0 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94301661427168] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94301661431184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c451080f50 con 0x55c451071fb0 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 20 add_watcher() i=0 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 20 add_watcher() i=1 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 20 add_watcher() i=2 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 20 add_watcher() i=3 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 20 add_watcher() i=4 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 20 add_watcher() i=5 2024-03-28T16:49:39.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.478+0000 7f3066d8c980 20 add_watcher() i=6 2024-03-28T16:49:39.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94301661431184] v31'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 20 add_watcher() i=7 2024-03-28T16:49:39.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:39.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c45107fb70 con 0x55c45106da40 2024-03-28T16:49:39.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94301661432432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510817a0 con 0x55c45106da40 2024-03-28T16:49:39.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94301661432432] v31'180 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f2f1afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:39.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:39.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:39.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 5 note: GC not initialized 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f2f19fe3640 20 reqs_thread_entry: start 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.482+0000 7f3066d8c980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 20 Filter name: none 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f2e8b7fe640 20 reqs_thread_entry: start 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510a6450 con 0x7f304c0e0040 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:39.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 : miss 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:63 8.11 8:8b360716:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510a7d20 con 0x55c451076b90 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(63 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 info.flags=0x17 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 to cache LRU end 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 20 get_obj_state: octx=0x7f305c00aac0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55c4510aa228 s->prefetch_data=0 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 3.39 3:9e9db662:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c451076b90 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(64 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+926 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 10 manifest: total_size = 5 2024-03-28T16:49:39.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 20 get_obj_state: setting s->obj_tag to _pneSKGzoJOQaZUPFWhlkVZJCpyWpdr- 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f2f1afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94301661432432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c45106da40 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94301661432432] v31'181 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.486+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94301661407616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x7f304c0e0040 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.490+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94301661407616] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.490+0000 7f3066d8c980 20 remove_watcher() i=0 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.490+0000 7f3066d8c980 2 removed watcher, disabling cache 2024-03-28T16:49:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.490+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94301661411936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c45106da40 2024-03-28T16:49:39.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.490+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94301661411936] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.490+0000 7f3066d8c980 20 remove_watcher() i=1 2024-03-28T16:49:39.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.490+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94301661415584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c451076b90 2024-03-28T16:49:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.494+0000 7f30640a6640 1 -- 172.21.15.170:0/2410287509 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94301661415584] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f304c0b0020 con 0x55c451076b90 2024-03-28T16:49:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.494+0000 7f3066d8c980 20 remove_watcher() i=2 2024-03-28T16:49:39.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.494+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94301661420272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x7f304c0e0040 2024-03-28T16:49:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.494+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94301661420272] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x7f304c0e0040 2024-03-28T16:49:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.494+0000 7f3066d8c980 20 remove_watcher() i=3 2024-03-28T16:49:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.494+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94301661412992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c45106da40 2024-03-28T16:49:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.498+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94301661412992] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f305c00a4d0 con 0x55c45106da40 2024-03-28T16:49:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.498+0000 7f3066d8c980 20 remove_watcher() i=4 2024-03-28T16:49:39.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.498+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94301661423728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c451071fb0 2024-03-28T16:49:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.498+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94301661423728] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.498+0000 7f3066d8c980 20 remove_watcher() i=5 2024-03-28T16:49:39.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.498+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94301661427168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c451071fb0 2024-03-28T16:49:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94301661427168] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 20 remove_watcher() i=6 2024-03-28T16:49:39.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94301661431184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c4510ab360 con 0x55c451071fb0 2024-03-28T16:49:39.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f30648a7640 1 -- 172.21.15.170:0/2410287509 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94301661431184] v31'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f3054040070 con 0x55c451071fb0 2024-03-28T16:49:39.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 20 remove_watcher() i=7 2024-03-28T16:49:39.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451071fb0 msgr2=0x55c4510743a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451071fb0 0x55c4510743a0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c45106da40 msgr2=0x55c45106ff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c45106da40 0x55c45106ff00 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f304c0e0040 msgr2=0x7f304c0d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f304c0e0040 0x7f304c0d3400 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c451076b90 msgr2=0x55c451079050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c451076b90 0x55c451079050 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f300342b0 msgr2=0x7f2f30036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f300342b0 0x7f2f30036770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f304c018f70 tx=0x7f304c0c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 msgr2=0x55c4510625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f30650a8640 1 -- 172.21.15.170:0/2410287509 reap_dead start 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c4510625f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f305402ffd0 tx=0x7f3054009f80 comp rx=0 tx=0).stop 2024-03-28T16:49:39.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 shutdown_connections 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/2410287509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45105b770 0x55c4510625f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 >> 172.21.15.170:0/2410287509 conn(0x55c451057b00 msgr2=0x55c451055160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 shutdown_connections 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/2410287509 wait complete. 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f3c0342b0 msgr2=0x7f2f3c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f3c0342b0 0x7f2f3c036770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55c45103ce00 tx=0x7f304c005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45104e8a0 msgr2=0x55c45104ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45104e8a0 0x55c45104ecc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3054049010 tx=0x7f30540187d0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 shutdown_connections 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f3c0342b0 0x7f2f3c036770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 --2- 172.21.15.170:0/1615576804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c45104e8a0 0x55c45104ecc0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 >> 172.21.15.170:0/1615576804 conn(0x55c4510436b0 msgr2=0x55c4510441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 shutdown_connections 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.502+0000 7f3066d8c980 1 -- 172.21.15.170:0/1615576804 wait complete. 2024-03-28T16:49:39.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451039ec0 msgr2=0x55c45103c2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c451039ec0 0x55c45103c2e0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c450f422e0 msgr2=0x55c451037cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c450f422e0 0x55c451037cd0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c45103de90 msgr2=0x55c451040280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c45103de90 0x55c451040280 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f4c034300 msgr2=0x7f2f4c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2f4c034300 0x7f2f4c0367c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f304c0ae000 tx=0x7f304c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 msgr2=0x55c450f3e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 --2- 172.21.15.170:0/830670187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c450eb20c0 0x55c450f3e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f305400b650 tx=0x7f3054000bd0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f30650a8640 1 -- 172.21.15.170:0/830670187 reap_dead start 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 shutdown_connections 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 >> 172.21.15.170:0/830670187 conn(0x55c450f386e0 msgr2=0x55c4510225e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 shutdown_connections 2024-03-28T16:49:39.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.506+0000 7f3066d8c980 1 -- 172.21.15.170:0/830670187 wait complete. 2024-03-28T16:49:39.516 INFO:teuthology.orchestra.run.smithi170.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-03-28T16:49:39.517 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-03-28T16:49:39.519 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-03-28T16:49:39.540 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:39.654 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-03-28T16:49:39.655 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-03-28T16:49:39.655 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-03-28T16:49:39.655 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-03-28T16:49:39.655 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-03-28T16:49:39.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-03-28T16:49:39.655 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-03-28T16:49:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:39.695 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73d09d980 1 Processor -- start 2024-03-28T16:49:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73d09d980 1 -- start start 2024-03-28T16:49:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73d09d980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5565d4ebb140 0x5565d4ebb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73d09d980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5565d4e054c0 con 0x5565d4ebba60 2024-03-28T16:49:39.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73d09d980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5565d4dcc0f0 con 0x5565d4ebb140 2024-03-28T16:49:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73abb8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5565d4ebb140 0x5565d4ebb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73abb8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5565d4ebb140 0x5565d4ebb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34916/0 (socket says 172.21.15.170:34916) 2024-03-28T16:49:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73abb8640 1 -- 172.21.15.170:0/1264604800 learned_addr learned my addr 172.21.15.170:0/1264604800 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73abb8640 1 -- 172.21.15.170:0/1264604800 >> v1:172.21.15.119:6789/0 conn(0x5565d4ebba60 legacy=0x5565d4ebc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.694+0000 7fa73abb8640 1 -- 172.21.15.170:0/1264604800 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d4ebcc40 con 0x5565d4ebb140 2024-03-28T16:49:39.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73abb8640 1 --2- 172.21.15.170:0/1264604800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4ebb520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00a5c0 tx=0x7fa72c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb25f9d467ef5e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa739bb6640 1 -- 172.21.15.170:0/1264604800 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0aa470 con 0x5565d4ebb140 2024-03-28T16:49:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa739bb6640 1 -- 172.21.15.170:0/1264604800 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0aa5d0 con 0x5565d4ebb140 2024-03-28T16:49:39.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa739bb6640 1 -- 172.21.15.170:0/1264604800 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0aa890 con 0x5565d4ebb140 2024-03-28T16:49:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73d09d980 1 -- 172.21.15.170:0/1264604800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 msgr2=0x5565d4ebb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73d09d980 1 --2- 172.21.15.170:0/1264604800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4ebb520 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00a5c0 tx=0x7fa72c01b340 comp rx=0 tx=0).stop 2024-03-28T16:49:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73d09d980 1 -- 172.21.15.170:0/1264604800 shutdown_connections 2024-03-28T16:49:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73d09d980 1 --2- 172.21.15.170:0/1264604800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4ebb520 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73d09d980 1 -- 172.21.15.170:0/1264604800 >> 172.21.15.170:0/1264604800 conn(0x5565d4e2e860 msgr2=0x5565d4e2ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73d09d980 1 -- 172.21.15.170:0/1264604800 shutdown_connections 2024-03-28T16:49:39.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.698+0000 7fa73d09d980 1 -- 172.21.15.170:0/1264604800 wait complete. 2024-03-28T16:49:39.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.714+0000 7fa73d09d980 1 Processor -- start 2024-03-28T16:49:39.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- start start 2024-03-28T16:49:39.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fa5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d4dc9620 con 0x5565d4ebb140 2024-03-28T16:49:39.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fa5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fa5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34926/0 (socket says 172.21.15.170:34926) 2024-03-28T16:49:39.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 -- 172.21.15.170:0/1808556851 learned_addr learned my addr 172.21.15.170:0/1808556851 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 -- 172.21.15.170:0/1808556851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c005df0 con 0x5565d4ebb140 2024-03-28T16:49:39.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 --2- 172.21.15.170:0/1808556851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fa5610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa72c018f70 tx=0x7fa72c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236c5e76b1aa9963 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa628e09640 1 -- 172.21.15.170:0/1808556851 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0aa9c0 con 0x5565d4ebb140 2024-03-28T16:49:39.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa628e09640 1 -- 172.21.15.170:0/1808556851 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0aab20 con 0x5565d4ebb140 2024-03-28T16:49:39.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa628e09640 1 -- 172.21.15.170:0/1808556851 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0aadc0 con 0x5565d4ebb140 2024-03-28T16:49:39.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- 172.21.15.170:0/1808556851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 msgr2=0x5565d4fa5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 --2- 172.21.15.170:0/1808556851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fa5610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa72c018f70 tx=0x7fa72c003030 comp rx=0 tx=0).stop 2024-03-28T16:49:39.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- 172.21.15.170:0/1808556851 shutdown_connections 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 --2- 172.21.15.170:0/1808556851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fa5610 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- 172.21.15.170:0/1808556851 >> 172.21.15.170:0/1808556851 conn(0x5565d4e2e860 msgr2=0x5565d4fa11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- 172.21.15.170:0/1808556851 shutdown_connections 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- 172.21.15.170:0/1808556851 wait complete. 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 Processor -- start 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- start start 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73d09d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d4dc4e00 con 0x5565d4ebb140 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fac380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34928/0 (socket says 172.21.15.170:34928) 2024-03-28T16:49:39.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.718+0000 7fa73abb8640 1 -- 172.21.15.170:0/3428842175 learned_addr learned my addr 172.21.15.170:0/3428842175 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73abb8640 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d4ebcc40 con 0x5565d4ebb140 2024-03-28T16:49:39.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73abb8640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fac380 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c10 tx=0x7fa72c0180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa627606640 1 -- 172.21.15.170:0/3428842175 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0c6650 con 0x5565d4ebb140 2024-03-28T16:49:39.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5565d4fac8c0 con 0x5565d4ebb140 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa627606640 1 -- 172.21.15.170:0/3428842175 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0c67b0 con 0x5565d4ebb140 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa627606640 1 -- 172.21.15.170:0/3428842175 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0c6a50 con 0x5565d4ebb140 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565d4faccf0 con 0x5565d4ebb140 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa627606640 1 -- 172.21.15.170:0/3428842175 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa72c0c6c80 con 0x5565d4ebb140 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa627606640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6200342b0 0x7fa620036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa627606640 1 -- 172.21.15.170:0/3428842175 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa72c0e4ad0 con 0x5565d4ebb140 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6200342b0 0x7fa620036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4ebba60 0x5565d4fb3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fb3540 con 0x5565d4ebba60 2024-03-28T16:49:39.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73b3b9640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4ebba60 0x5565d4fb3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6200342b0 0x7fa620036770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa730001e20 tx=0x7fa7300013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73b3b9640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4ebba60 0x5565d4fb3000 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73b3b9640 1 -- 172.21.15.170:0/3428842175 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4ebba60 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fb38a0 con 0x5565d4ebba60 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73b3b9640 1 -- 172.21.15.170:0/3428842175 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4ebba60 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fb55c0 0x5565d4fb79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.722+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fb7ef0 con 0x5565d4fb55c0 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73abb8640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fb55c0 0x5565d4fb79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73abb8640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fb55c0 0x5565d4fb79b0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73abb8640 1 -- 172.21.15.170:0/3428842175 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fb55c0 2024-03-28T16:49:39.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fb4400 con 0x5565d4ebba60 2024-03-28T16:49:39.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73b3b9640 1 -- 172.21.15.170:0/3428842175 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4ebba60 2024-03-28T16:49:39.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5565d4fb9280 0x5565d4fbb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fbbd10 con 0x5565d4fb9280 2024-03-28T16:49:39.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5565d4fb9280 0x5565d4fbb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5565d4fb9280 0x5565d4fbb670 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73a3b7640 1 -- 172.21.15.170:0/3428842175 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fa730010630 con 0x5565d4fb9280 2024-03-28T16:49:39.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73d09d980 1 Processor -- start 2024-03-28T16:49:39.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73d09d980 1 -- start start 2024-03-28T16:49:39.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73d09d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fc9c40 0x5565d4fca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73d09d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d4d5de30 con 0x5565d4fc9c40 2024-03-28T16:49:39.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.726+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fc9c40 0x5565d4fca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fc9c40 0x5565d4fca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34944/0 (socket says 172.21.15.170:34944) 2024-03-28T16:49:39.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73abb8640 1 -- 172.21.15.170:0/2330309585 learned_addr learned my addr 172.21.15.170:0/2330309585 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73abb8640 1 -- 172.21.15.170:0/2330309585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d4fca5a0 con 0x5565d4fc9c40 2024-03-28T16:49:39.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73abb8640 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fc9c40 0x5565d4fca060 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0cfc40 tx=0x7fa72c0c7f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa624e01640 1 -- 172.21.15.170:0/2330309585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0e9070 con 0x5565d4fc9c40 2024-03-28T16:49:39.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa624e01640 1 -- 172.21.15.170:0/2330309585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0a4c90 con 0x5565d4fc9c40 2024-03-28T16:49:39.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5565d4fca7f0 con 0x5565d4fc9c40 2024-03-28T16:49:39.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa624e01640 1 -- 172.21.15.170:0/2330309585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0bc3a0 con 0x5565d4fc9c40 2024-03-28T16:49:39.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565d4fcb340 con 0x5565d4fc9c40 2024-03-28T16:49:39.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa624e01640 1 -- 172.21.15.170:0/2330309585 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa72c0bc500 con 0x5565d4fc9c40 2024-03-28T16:49:39.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa624e01640 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa614034300 0x7fa6140367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa624e01640 1 -- 172.21.15.170:0/2330309585 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa72c0efbd0 con 0x5565d4fc9c40 2024-03-28T16:49:39.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa614034300 0x7fa6140367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73d09d980 1 Processor -- start 2024-03-28T16:49:39.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73d09d980 1 -- start start 2024-03-28T16:49:39.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa614034300 0x7fa6140367c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5565d4fb0a50 tx=0x7fa730006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73d09d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fd6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.730+0000 7fa73d09d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d4d51790 con 0x5565d4fd6ba0 2024-03-28T16:49:39.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fd6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fd6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34960/0 (socket says 172.21.15.170:34960) 2024-03-28T16:49:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 -- 172.21.15.170:0/594613592 learned_addr learned my addr 172.21.15.170:0/594613592 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 -- 172.21.15.170:0/594613592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d4fd7590 con 0x5565d4fd6ba0 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 --2- 172.21.15.170:0/594613592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fd6fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0c6c50 tx=0x7fa72c0eff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc67a20a9de398b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa61e7fc640 1 -- 172.21.15.170:0/594613592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0e1d10 con 0x5565d4fd6ba0 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa61e7fc640 1 -- 172.21.15.170:0/594613592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0e1e70 con 0x5565d4fd6ba0 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa61e7fc640 1 -- 172.21.15.170:0/594613592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0d1c20 con 0x5565d4fd6ba0 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- 172.21.15.170:0/594613592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 msgr2=0x5565d4fd6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 --2- 172.21.15.170:0/594613592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fd6fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0c6c50 tx=0x7fa72c0eff30 comp rx=0 tx=0).stop 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- 172.21.15.170:0/594613592 shutdown_connections 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 --2- 172.21.15.170:0/594613592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fd6fc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- 172.21.15.170:0/594613592 >> 172.21.15.170:0/594613592 conn(0x5565d4fd2ee0 msgr2=0x5565d4fd0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- 172.21.15.170:0/594613592 shutdown_connections 2024-03-28T16:49:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- 172.21.15.170:0/594613592 wait complete. 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 Processor -- start 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- start start 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fddab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d4fddff0 con 0x5565d4fd6ba0 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fddab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34976/0 (socket says 172.21.15.170:34976) 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 learned_addr learned my addr 172.21.15.170:0/2547412797 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d4fcb340 con 0x5565d4fd6ba0 2024-03-28T16:49:39.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73abb8640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fddab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0db9b0 tx=0x7fa72c0d1e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa61cff9640 1 -- 172.21.15.170:0/2547412797 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0dba20 con 0x5565d4fd6ba0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.734+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5565d4fcb340 con 0x5565d4fd6ba0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa61cff9640 1 -- 172.21.15.170:0/2547412797 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0dbc50 con 0x5565d4fd6ba0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa61cff9640 1 -- 172.21.15.170:0/2547412797 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa72c0de490 con 0x5565d4fd6ba0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565d4fde3d0 con 0x5565d4fd6ba0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa61cff9640 1 -- 172.21.15.170:0/2547412797 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa72c109050 con 0x5565d4fd6ba0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa61cff9640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa604034300 0x7fa6040367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa61cff9640 1 -- 172.21.15.170:0/2547412797 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa72c0fd040 con 0x5565d4fd6ba0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa604034300 0x7fa6040367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4fe8b60 0x5565d4feb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4feb750 con 0x5565d4fe8b60 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73b3b9640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4fe8b60 0x5565d4feb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa604034300 0x7fa6040367c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa730010fd0 tx=0x7fa730011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73b3b9640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4fe8b60 0x5565d4feb000 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 realm 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4febb20 con 0x5565d4fe8b60 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec6d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fed080 0x5565d4fef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.738+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fefac0 con 0x5565d4fed080 2024-03-28T16:49:39.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73abb8640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fed080 0x5565d4fef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73abb8640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fed080 0x5565d4fef470 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73d09d980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73d09d980 20 searching for the correct realm 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5565d4ff1c80 0x5565d4ff4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff47c0 con 0x5565d4ff1c80 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5565d4ff1c80 0x5565d4ff4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5565d4ff1c80 0x5565d4ff4140 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73a3b7640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa73000f3d0 0x7fa73000f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff47c0 con 0x7fa73000f3d0 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73b3b9640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa73000f3d0 0x7fa73000f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73b3b9640 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa73000f3d0 0x7fa73000f7f0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.742+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x7fa73000f3d0 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x7fa73000f3d0 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.746+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x7fa73000f3d0 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.750+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x7fa73000f3d0 2024-03-28T16:49:39.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4ff1c80 2024-03-28T16:49:39.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fed080 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x5565d4fe8b60 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fec370 con 0x7fa73000f3d0 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:39.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff00a0 con 0x5565d4fe8b60 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff00a0 con 0x5565d4fe8b60 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.754+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff00a0 con 0x7fa73000f3d0 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:39.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 zone default found 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 4 Realm: () 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 zonegroup default 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff08f0 con 0x5565d4fed080 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff65e0 con 0x5565d4ff1c80 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff6940 con 0x5565d4ff1c80 2024-03-28T16:49:39.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 started sync module instance, tier type = 2024-03-28T16:49:39.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:39.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff5ec0 con 0x7fa73000f3d0 2024-03-28T16:49:39.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93895853562400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff5e40 con 0x7fa73000f3d0 2024-03-28T16:49:39.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.758+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff75e0 con 0x5565d4fe8b60 2024-03-28T16:49:39.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.762+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93895853562400] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.762+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.762+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93895853567136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff5e40 con 0x5565d4fe8b60 2024-03-28T16:49:39.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.762+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff7ca0 con 0x5565d4ff1c80 2024-03-28T16:49:39.765 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.762+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93895853567136] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93895853569920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff8050 con 0x5565d4ff1c80 2024-03-28T16:49:39.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff89c0 con 0x7fa73000f3d0 2024-03-28T16:49:39.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93895853573680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff90b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff9b80 con 0x5565d4fe8b60 2024-03-28T16:49:39.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93895853569920] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93895853573680] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.769 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93895853577824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff9b40 con 0x5565d4fe8b60 2024-03-28T16:49:39.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.766+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff7230 con 0x5565d4fed080 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.770+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93895853577824] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.770+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.770+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93895853574928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ffa3d0 con 0x5565d4fed080 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.770+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ffabd0 con 0x5565d4fed080 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93895853574928] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93895853582128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ffab50 con 0x5565d4fed080 2024-03-28T16:49:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ffba30 con 0x5565d4fed080 2024-03-28T16:49:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93895853582128] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93895853584176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ffba30 con 0x5565d4fed080 2024-03-28T16:49:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 20 add_watcher() i=0 2024-03-28T16:49:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 20 add_watcher() i=1 2024-03-28T16:49:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 20 add_watcher() i=2 2024-03-28T16:49:39.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 20 add_watcher() i=3 2024-03-28T16:49:39.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 20 add_watcher() i=4 2024-03-28T16:49:39.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 20 add_watcher() i=5 2024-03-28T16:49:39.777 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.774+0000 7fa73d09d980 20 add_watcher() i=6 2024-03-28T16:49:39.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93895853584176] v31'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73d09d980 20 add_watcher() i=7 2024-03-28T16:49:39.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73d09d980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:39.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ffeed0 con 0x5565d4fe8b60 2024-03-28T16:49:39.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93895853586480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ffc260 con 0x5565d4fe8b60 2024-03-28T16:49:39.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93895853586480] v31'182 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa5f17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:39.780 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73d09d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:39.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73d09d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:39.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.778+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5007420 con 0x5565d4fe8b60 2024-03-28T16:49:39.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.782+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.782+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5007430 con 0x5565d4fed080 2024-03-28T16:49:39.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.782+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.782+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5007920 con 0x5565d4ff1c80 2024-03-28T16:49:39.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.782+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.782+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50082f0 con 0x5565d4fe8b60 2024-03-28T16:49:39.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.786+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'183 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.786+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5005590 con 0x7fa73000f3d0 2024-03-28T16:49:39.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.786+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'62 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.786+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5005590 con 0x5565d4ff1c80 2024-03-28T16:49:39.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.786+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'41 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.789 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.786+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009030 con 0x5565d4fe8b60 2024-03-28T16:49:39.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.790+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.790+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009030 con 0x5565d4fed080 2024-03-28T16:49:39.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.790+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'41 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.793 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.790+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009560 con 0x7fa73000f3d0 2024-03-28T16:49:39.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.794+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.794+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009560 con 0x5565d4ff1c80 2024-03-28T16:49:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.794+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.794+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009a90 con 0x7fa73000f3d0 2024-03-28T16:49:39.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.794+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'63 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.794+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009a90 con 0x5565d4fed080 2024-03-28T16:49:39.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.798+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'27 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.799 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.798+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009fc0 con 0x7fa73000f3d0 2024-03-28T16:49:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.798+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.798+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009fc0 con 0x5565d4fed080 2024-03-28T16:49:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.802+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'48 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.802+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4fe8b60 2024-03-28T16:49:39.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.802+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'184 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.802+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5009a90 con 0x5565d4ff1c80 2024-03-28T16:49:39.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.802+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'56 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.806+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x7fa73000f3d0 2024-03-28T16:49:39.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.806+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'64 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.806+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.806+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.806+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4fe8b60 2024-03-28T16:49:39.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.810+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.810+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4fe8b60 2024-03-28T16:49:39.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.810+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'185 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.810+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.810+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'57 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.810+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x7fa73000f3d0 2024-03-28T16:49:39.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'34 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'27 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'20 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.814+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.818+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'42 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.818+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4fed080 2024-03-28T16:49:39.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.818+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'28 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.818+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.822+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.822 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.822+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4fed080 2024-03-28T16:49:39.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.822+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'42 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.822+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4ff1c80 2024-03-28T16:49:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.826+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'58 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.826+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x5565d4fed080 2024-03-28T16:49:39.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.826+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'49 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.826+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d500a460 con 0x7fa73000f3d0 2024-03-28T16:49:39.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa56bfff640 20 reqs_thread_entry: start 2024-03-28T16:49:39.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:39.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 20 Filter name: none 2024-03-28T16:49:39.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa569ffb640 20 reqs_thread_entry: start 2024-03-28T16:49:39.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:49:39.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4fb02a0 con 0x7fa73000f3d0 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 : miss 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:95 8.11 8:8b360716:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5004420 con 0x5565d4ff1c80 2024-03-28T16:49:39.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(95 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 info.flags=0x17 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 to cache LRU end 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 4.2b 4:d67411c7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a31c0 con 0x5565d4fe8b60 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(96 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.830+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:97 4.33 4:cc08504f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a31c0 con 0x5565d4fed080 2024-03-28T16:49:39.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(97 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:98 4.c 4:335b8573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a31c0 con 0x5565d4fe8b60 2024-03-28T16:49:39.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(98 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2 [call out=425b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-03-28T16:49:39.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:99 4.34 4:2e87d6d7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(99 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:100 4.2c 4:359f3f8a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x5565d4fe8b60 2024-03-28T16:49:39.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(100 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:101 4.18 4:1adcac47:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(101 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:102 4.34 4:2fc1a47d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(102 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6 [call out=426b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+426 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:103 4.c 4:33557cc2:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x5565d4fe8b60 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(103 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:104 4.28 4:174dfcb4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x5565d4fe8b60 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(104 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8 [call out=426b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+426 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-03-28T16:49:39.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.834+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:105 4.b 4:d025d7e3:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(105 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:106 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a36b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(106 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [call out=698b] v0'0 uv9 ondisk = 0) v8 ==== 195+0+698 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:107 3.33 3:cd7a5ddb:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a2e00 con 0x5565d4fed080 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(107 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 10 manifest: total_size = 6 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.4440572608939307232 2024-03-28T16:49:39.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 20 prepare_atomic_modification: state is not atomic. state=0x5565d5026bd8 2024-03-28T16:49:39.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2 2024-03-28T16:49:39.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:108 4.c 4:335b8573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502bdd0 con 0x5565d4fe8b60 2024-03-28T16:49:39.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(108 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2 [stat,call,call] v31'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.838+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:109 3.33 3:cd7a5ddb:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502bdd0 con 0x5565d4fed080 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(109 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_9c8454ddf7aa50116496bac348d7550d [call] v31'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:110 4.c 4:335b8573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502c370 con 0x5565d4fe8b60 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:111 3.15 3:a9cc339f:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502cfc0 con 0x7fa73000f3d0 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 25 ==== osd_op_reply(111 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 10 manifest: total_size = 3 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.16334160116063301560 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 20 prepare_atomic_modification: state is not atomic. state=0x5565d5026bd8 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6 2024-03-28T16:49:39.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:112 4.34 4:2fc1a47d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x7fa73000f3d0 2024-03-28T16:49:39.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(110 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2 [stat,call,call] v31'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa61e7fc640 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-03-28T16:49:39.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 26 ==== osd_op_reply(112 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6 [stat,call,call] v31'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.842+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:113 3.15 3:a9cc339f:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x7fa73000f3d0 2024-03-28T16:49:39.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 27 ==== osd_op_reply(113 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_b1b9a972ccd8c962a473909b97007eb4 [call] v31'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:114 4.34 4:2fc1a47d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x7fa73000f3d0 2024-03-28T16:49:39.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:115 3.7 3:e2d1a47f:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502dc00 con 0x5565d4ff1c80 2024-03-28T16:49:39.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(115 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 10 manifest: total_size = 4 2024-03-28T16:49:39.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.15141240429277292868 2024-03-28T16:49:39.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 20 prepare_atomic_modification: state is not atomic. state=0x5565d5026bd8 2024-03-28T16:49:39.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8 2024-03-28T16:49:39.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:116 4.28 4:174dfcb4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x5565d4fe8b60 2024-03-28T16:49:39.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 28 ==== osd_op_reply(114 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6 [stat,call,call] v31'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa61dffb640 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-03-28T16:49:39.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(116 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8 [stat,call,call] v31'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.846+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:117 3.7 3:e2d1a47f:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x5565d4ff1c80 2024-03-28T16:49:39.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(117 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v31'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:118 4.28 4:174dfcb4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x5565d4fe8b60 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:119 3.6 3:6082721c:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502dc00 con 0x5565d4fe8b60 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(119 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 10 manifest: total_size = 5 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.12081297362759053268 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 20 prepare_atomic_modification: state is not atomic. state=0x5565d5026bd8 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 2024-03-28T16:49:39.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.850+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:120 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x7fa73000f3d0 2024-03-28T16:49:39.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 29 ==== osd_op_reply(120 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [stat,call,call] v31'10 uv10 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:121 3.6 3:6082721c:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502d000 con 0x5565d4fe8b60 2024-03-28T16:49:39.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 36 ==== osd_op_reply(118 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8 [stat,call,call] v31'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa61e7fc640 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-03-28T16:49:39.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 37 ==== osd_op_reply(121 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_24d27c169c2c881eb09a065116f2aa5c [call] v31'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:122 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502def0 con 0x7fa73000f3d0 2024-03-28T16:49:39.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:123 3.39 3:9e9db662:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e950 con 0x5565d4ff1c80 2024-03-28T16:49:39.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(123 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+926 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 10 manifest: total_size = 5 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 20 get_obj_state: setting s->obj_tag to _pneSKGzoJOQaZUPFWhlkVZJCpyWpdr- 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 20 get_obj_state: octx=0x7fa73400aac0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5565d5026bd8 s->prefetch_data=0 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 20 prepare_atomic_modification: state is not atomic. state=0x5565d5026bd8 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:124 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032990 con 0x7fa73000f3d0 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 30 ==== osd_op_reply(122 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [stat,call,call] v31'11 uv11 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.854+0000 7fa61dffb640 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-03-28T16:49:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 31 ==== osd_op_reply(124 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [stat,call,call] v31'12 uv12 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:125 3.39 3:9e9db662:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50276a0 con 0x5565d4ff1c80 2024-03-28T16:49:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(125 b88c2392-6ef8-4660-b080-19480b748fc0.4221.5_bb3aec0fdcdbc2974890f805c585d432 [call] v31'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:126 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e950 con 0x7fa73000f3d0 2024-03-28T16:49:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 20 list_objects_ordered: starting attempt 1 2024-03-28T16:49:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-28T16:49:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:127 4.2b 4:d67411c7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e0a0 con 0x5565d4fe8b60 2024-03-28T16:49:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:128 4.33 4:cc08504f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50269f0 con 0x5565d4fed080 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:129 4.c 4:335b8573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032060 con 0x5565d4fe8b60 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:130 4.34 4:2e87d6d7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032a60 con 0x7fa73000f3d0 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:131 4.2c 4:359f3f8a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d503a440 con 0x5565d4fe8b60 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:132 4.18 4:1adcac47:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d503cd50 con 0x7fa73000f3d0 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:133 4.34 4:2fc1a47d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d503f5f0 con 0x7fa73000f3d0 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:134 4.c 4:33557cc2:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5042310 con 0x5565d4fe8b60 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:135 4.28 4:174dfcb4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d503f5f0 con 0x5565d4fe8b60 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:136 4.b 4:d025d7e3:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5047780 con 0x7fa73000f3d0 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:137 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d504a0c0 con 0x7fa73000f3d0 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 38 ==== osd_op_reply(127 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 32 ==== osd_op_reply(132 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 39 ==== osd_op_reply(129 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 40 ==== osd_op_reply(135 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 41 ==== osd_op_reply(131 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 33 ==== osd_op_reply(136 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 34 ==== osd_op_reply(130 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 35 ==== osd_op_reply(133 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 42 ==== osd_op_reply(134 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 36 ==== osd_op_reply(126 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [stat,call,call] v31'13 uv13 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa61dffb640 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-03-28T16:49:39.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 37 ==== osd_op_reply(137 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [call out=124b] v0'0 uv13 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(128 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.5])"; uploads.size()=0; is_truncated=0 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:138 4.2b 4:d67411c7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d503f630 con 0x5565d4fe8b60 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:139 4.33 4:cc08504f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032270 con 0x5565d4fed080 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:140 4.c 4:335b8573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032610 con 0x5565d4fe8b60 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:141 4.34 4:2e87d6d7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e1b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:142 4.2c 4:359f3f8a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d503f630 con 0x5565d4fe8b60 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:143 4.18 4:1adcac47:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e5b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:144 4.34 4:2fc1a47d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502ea00 con 0x7fa73000f3d0 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:145 4.c 4:33557cc2:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032610 con 0x5565d4fe8b60 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:146 4.28 4:174dfcb4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d503f630 con 0x5565d4fe8b60 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:147 4.b 4:d025d7e3:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033f40 con 0x7fa73000f3d0 2024-03-28T16:49:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.858+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:148 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50363e0 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 43 ==== osd_op_reply(138 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 44 ==== osd_op_reply(140 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 45 ==== osd_op_reply(142 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 46 ==== osd_op_reply(146 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(139 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 38 ==== osd_op_reply(141 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 39 ==== osd_op_reply(143 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 40 ==== osd_op_reply(147 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 41 ==== osd_op_reply(148 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [call out=124b] v0'0 uv13 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 42 ==== osd_op_reply(144 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 47 ==== osd_op_reply(145 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:149 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033f40 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 43 ==== osd_op_reply(149 foo.buckets [call] v31'39 uv39 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73d09d980 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x11) 2024-03-28T16:49:39.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73d09d980 10 removing default.rgw.meta+root+myfoo from cache 2024-03-28T16:49:39.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73d09d980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:49:39.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:150 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93895854270176 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033f40 con 0x5565d4fed080 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== watch-notify(notify (1) cookie 93895853582128 notify 133143986202 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x5565d4fd7590 con 0x5565d4fed080 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa61e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986202 cookie 93895853582128 notifier 4641 bl.length()=149 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa61e7fc640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:151 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fa600003d60 con 0x5565d4fed080 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.862+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(150 notify.6 [notify cookie 93895854270176 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(151 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== watch-notify(notify_complete (2) cookie 93895854270176 notify 133143986202 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5565d4fca5a0 con 0x5565d4fed080 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:152 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033f40 con 0x7fa73000f3d0 2024-03-28T16:49:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 44 ==== osd_op_reply(152 myfoo [call,call,delete] v31'40 uv40 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73d09d980 10 removing default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 from cache 2024-03-28T16:49:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73d09d980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5, ofs0, ns] 2024-03-28T16:49:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:153 7.9 7:93e5b521:::notify.7:head [notify cookie 93895853793376 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033f40 con 0x5565d4fed080 2024-03-28T16:49:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== watch-notify(notify (1) cookie 93895853584176 notify 133143986203 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x7fa72c133040 con 0x5565d4fed080 2024-03-28T16:49:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa61dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986203 cookie 93895853584176 notifier 4641 bl.length()=206 2024-03-28T16:49:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa61dffb640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:154 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fa5700032c0 con 0x5565d4fed080 2024-03-28T16:49:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.866+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(153 notify.7 [notify cookie 93895853793376 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== osd_op_reply(154 notify.7 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== watch-notify(notify_complete (2) cookie 93895853793376 notify 133143986203 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5565d4fca5a0 con 0x5565d4fed080 2024-03-28T16:49:39.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:155 8.11 8:8b360716:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033f40 con 0x5565d4ff1c80 2024-03-28T16:49:39.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 31 ==== osd_op_reply(155 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 [call,call,delete] v31'2 uv2 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:156 4.2b 4:d67411c7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50a26e0 con 0x5565d4fe8b60 2024-03-28T16:49:39.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:157 4.33 4:cc08504f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e550 con 0x5565d4fed080 2024-03-28T16:49:39.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:158 4.c 4:335b8573:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032f10 con 0x5565d4fe8b60 2024-03-28T16:49:39.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:159 4.34 4:2e87d6d7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50332b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:160 4.2c 4:359f3f8a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032f10 con 0x5565d4fe8b60 2024-03-28T16:49:39.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:161 4.18 4:1adcac47:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e550 con 0x7fa73000f3d0 2024-03-28T16:49:39.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:162 4.34 4:2fc1a47d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50332b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:163 4.c 4:33557cc2:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032f10 con 0x5565d4fe8b60 2024-03-28T16:49:39.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:164 4.28 4:174dfcb4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e550 con 0x5565d4fe8b60 2024-03-28T16:49:39.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:165 4.b 4:d025d7e3:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d50332b0 con 0x7fa73000f3d0 2024-03-28T16:49:39.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.870+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:166 4.3e 4:7c6fd791:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5032f10 con 0x7fa73000f3d0 2024-03-28T16:49:39.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 45 ==== osd_op_reply(159 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.3 [delete] v31'7 uv7 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 48 ==== osd_op_reply(156 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.0 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 46 ==== osd_op_reply(166 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.10 [delete] v31'14 uv14 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 49 ==== osd_op_reply(158 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.2 [delete] v31'7 uv7 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 47 ==== osd_op_reply(162 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.6 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(157 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.1 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 50 ==== osd_op_reply(163 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.7 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 48 ==== osd_op_reply(161 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.5 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 51 ==== osd_op_reply(164 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.8 [delete] v31'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 52 ==== osd_op_reply(160 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.4 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 49 ==== osd_op_reply(165 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.5.9 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 : miss 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 20 rados->read ofs=0 len=0 2024-03-28T16:49:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:167 6.1d 6:be09aab5:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e8f0 con 0x5565d4fed080 2024-03-28T16:49:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(167 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 info.flags=0x0 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 10 adding default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 to cache LRU end 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 10 removing default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 from cache 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.log:pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.5, ofs0, ns] 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:168 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93895854270176 in=215b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033ea0 con 0x5565d4fed080 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== watch-notify(notify (1) cookie 93895853582128 notify 133143986204 ret 0) v3 ==== 245+0+0 (crc 0 0 0) 0x5565d4fca5a0 con 0x5565d4fed080 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa61e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986204 cookie 93895853582128 notifier 4641 bl.length()=203 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa61e7fc640 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:169 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fa600003d60 con 0x5565d4fed080 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.874+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 41 ==== osd_op_reply(168 notify.6 [notify cookie 93895854270176 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.878+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 42 ==== osd_op_reply(169 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.878+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 43 ==== watch-notify(notify_complete (2) cookie 93895854270176 notify 133143986204 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5565d4faccf0 con 0x5565d4fed080 2024-03-28T16:49:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.878+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:170 6.1d 6:be09aab5:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d502e060 con 0x5565d4fed080 2024-03-28T16:49:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.878+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 44 ==== osd_op_reply(170 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.5 [delete] v31'43 uv36 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.878+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:171 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5033ea0 con 0x7fa73000f3d0 2024-03-28T16:49:39.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.882+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 50 ==== osd_op_reply(171 foo.buckets [call] v31'41 uv41 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.882+0000 7fa5f17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:39.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.882+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:172 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93895853586480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5008120 con 0x5565d4fe8b60 2024-03-28T16:49:39.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.882+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 53 ==== osd_op_reply(172 data_loggenerations_metadata [watch unwatch cookie 93895853586480] v31'186 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.882+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:173 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93895853562400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5008120 con 0x7fa73000f3d0 2024-03-28T16:49:39.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.886+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 51 ==== osd_op_reply(173 notify.0 [watch unwatch cookie 93895853562400] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.886+0000 7fa73d09d980 20 remove_watcher() i=0 2024-03-28T16:49:39.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.886+0000 7fa73d09d980 2 removed watcher, disabling cache 2024-03-28T16:49:39.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.886+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:174 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93895853567136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5008120 con 0x5565d4fe8b60 2024-03-28T16:49:39.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.886+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 54 ==== osd_op_reply(174 notify.1 [watch unwatch cookie 93895853567136] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.886+0000 7fa73d09d980 20 remove_watcher() i=1 2024-03-28T16:49:39.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.886+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:175 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93895853569920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff6ca0 con 0x5565d4ff1c80 2024-03-28T16:49:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.890+0000 7fa73a3b7640 1 -- 172.21.15.170:0/2547412797 <== osd.3 v2:172.21.15.170:6816/2102412426 32 ==== osd_op_reply(175 notify.2 [watch unwatch cookie 93895853569920] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa730032040 con 0x5565d4ff1c80 2024-03-28T16:49:39.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.890+0000 7fa73d09d980 20 remove_watcher() i=2 2024-03-28T16:49:39.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.890+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:176 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93895853573680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5008120 con 0x7fa73000f3d0 2024-03-28T16:49:39.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.890+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.2 v2:172.21.15.170:6800/447751618 52 ==== osd_op_reply(176 notify.3 [watch unwatch cookie 93895853573680] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x7fa73000f3d0 2024-03-28T16:49:39.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.890+0000 7fa73d09d980 20 remove_watcher() i=3 2024-03-28T16:49:39.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.890+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:177 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93895853577824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff7780 con 0x5565d4fe8b60 2024-03-28T16:49:39.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.894+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 <== osd.1 v2:172.21.15.170:6805/11349688 55 ==== osd_op_reply(177 notify.4 [watch unwatch cookie 93895853577824] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa73400a4d0 con 0x5565d4fe8b60 2024-03-28T16:49:39.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.894+0000 7fa73d09d980 20 remove_watcher() i=4 2024-03-28T16:49:39.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.894+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:178 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93895853574928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff6ca0 con 0x5565d4fed080 2024-03-28T16:49:39.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.894+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 45 ==== osd_op_reply(178 notify.5 [watch unwatch cookie 93895853574928] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.894+0000 7fa73d09d980 20 remove_watcher() i=5 2024-03-28T16:49:39.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.894+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:179 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93895853582128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d4ff8630 con 0x5565d4fed080 2024-03-28T16:49:39.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 46 ==== osd_op_reply(179 notify.6 [watch unwatch cookie 93895853582128] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 20 remove_watcher() i=6 2024-03-28T16:49:39.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:180 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93895853584176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5565d5008120 con 0x5565d4fed080 2024-03-28T16:49:39.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73abb8640 1 -- 172.21.15.170:0/2547412797 <== osd.0 v2:172.21.15.119:6800/2028815515 47 ==== osd_op_reply(180 notify.7 [watch unwatch cookie 93895853584176] v31'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa72c0cd070 con 0x5565d4fed080 2024-03-28T16:49:39.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 20 remove_watcher() i=7 2024-03-28T16:49:39.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fed080 msgr2=0x5565d4fef470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fed080 0x5565d4fef470 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4fe8b60 msgr2=0x5565d4feb000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4fe8b60 0x5565d4feb000 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa73000f3d0 msgr2=0x7fa73000f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa73000f3d0 0x7fa73000f7f0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5565d4ff1c80 msgr2=0x5565d4ff4140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5565d4ff1c80 0x5565d4ff4140 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa604034300 msgr2=0x7fa6040367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa604034300 0x7fa6040367c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa730010fd0 tx=0x7fa730011000 comp rx=0 tx=0).stop 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73b3b9640 1 -- 172.21.15.170:0/2547412797 reap_dead start 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 msgr2=0x5565d4fddab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fddab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0db9b0 tx=0x7fa72c0d1e20 comp rx=0 tx=0).stop 2024-03-28T16:49:39.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 shutdown_connections 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2547412797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fd6ba0 0x5565d4fddab0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 >> 172.21.15.170:0/2547412797 conn(0x5565d4fd2ee0 msgr2=0x5565d4fd0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 shutdown_connections 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2547412797 wait complete. 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa614034300 msgr2=0x7fa6140367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa614034300 0x7fa6140367c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5565d4fb0a50 tx=0x7fa730006000 comp rx=0 tx=0).stop 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fc9c40 msgr2=0x5565d4fca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fc9c40 0x5565d4fca060 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0cfc40 tx=0x7fa72c0c7f70 comp rx=0 tx=0).stop 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 shutdown_connections 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa614034300 0x7fa6140367c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 --2- 172.21.15.170:0/2330309585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4fc9c40 0x5565d4fca060 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 >> 172.21.15.170:0/2330309585 conn(0x5565d4fbea50 msgr2=0x5565d4fbf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 shutdown_connections 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.898+0000 7fa73d09d980 1 -- 172.21.15.170:0/2330309585 wait complete. 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fb55c0 msgr2=0x5565d4fb79b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5565d4fb55c0 0x5565d4fb79b0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4ebba60 msgr2=0x5565d4fb3000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5565d4ebba60 0x5565d4fb3000 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5565d4fb9280 msgr2=0x5565d4fbb670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5565d4fb9280 0x5565d4fbb670 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6200342b0 msgr2=0x7fa620036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa6200342b0 0x7fa620036770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa730001e20 tx=0x7fa7300013a0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 msgr2=0x5565d4fac380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 --2- 172.21.15.170:0/3428842175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5565d4ebb140 0x5565d4fac380 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa72c000c10 tx=0x7fa72c0180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73b3b9640 1 -- 172.21.15.170:0/3428842175 reap_dead start 2024-03-28T16:49:39.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 shutdown_connections 2024-03-28T16:49:39.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 >> 172.21.15.170:0/3428842175 conn(0x5565d4e2e860 msgr2=0x5565d4f9fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:39.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 shutdown_connections 2024-03-28T16:49:39.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.902+0000 7fa73d09d980 1 -- 172.21.15.170:0/3428842175 wait complete. 2024-03-28T16:49:39.914 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-03-28T16:49:39.914 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-03-28T16:49:39.914 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-03-28T16:49:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.994+0000 7f394b8b2980 1 Processor -- start 2024-03-28T16:49:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.994+0000 7f394b8b2980 1 -- start start 2024-03-28T16:49:39.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.994+0000 7f394b8b2980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d165c956e0 0x55d165c97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:39.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.994+0000 7f394b8b2980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d165bdb4c0 con 0x55d165c030c0 2024-03-28T16:49:39.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.994+0000 7f394b8b2980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d165ba20f0 con 0x55d165c956e0 2024-03-28T16:49:39.998 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3948bcc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d165c956e0 0x55d165c97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:39.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3948bcc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d165c956e0 0x55d165c97ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34988/0 (socket says 172.21.15.170:34988) 2024-03-28T16:49:39.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f39493cd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55d165c030c0 0x55d165c93fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:41232/0 (socket says 172.21.15.170:41232) 2024-03-28T16:49:39.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3948bcc640 1 -- 172.21.15.170:0/2282477022 learned_addr learned my addr 172.21.15.170:0/2282477022 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:39.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3948bcc640 1 -- 172.21.15.170:0/2282477022 >> v1:172.21.15.119:6789/0 conn(0x55d165c030c0 legacy=0x55d165c93fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:39.999 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3948bcc640 1 -- 172.21.15.170:0/2282477022 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d165c99020 con 0x55d165c956e0 2024-03-28T16:49:40.001 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3948bcc640 1 --2- 172.21.15.170:0/2282477022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c956e0 0x55d165c97ad0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f393400a5c0 tx=0x7f393401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5f878a9ffb1807 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3943fff640 1 -- 172.21.15.170:0/2282477022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39340aa470 con 0x55d165c956e0 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3943fff640 1 -- 172.21.15.170:0/2282477022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39340aa5d0 con 0x55d165c956e0 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:39.998+0000 7f3943fff640 1 -- 172.21.15.170:0/2282477022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39340aa890 con 0x55d165c956e0 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.002+0000 7f394b8b2980 1 -- 172.21.15.170:0/2282477022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c956e0 msgr2=0x55d165c97ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.002+0000 7f394b8b2980 1 --2- 172.21.15.170:0/2282477022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c956e0 0x55d165c97ad0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f393400a5c0 tx=0x7f393401b340 comp rx=0 tx=0).stop 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.002+0000 7f394b8b2980 1 -- 172.21.15.170:0/2282477022 shutdown_connections 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.002+0000 7f394b8b2980 1 --2- 172.21.15.170:0/2282477022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c956e0 0x55d165c97ad0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.002+0000 7f394b8b2980 1 -- 172.21.15.170:0/2282477022 >> 172.21.15.170:0/2282477022 conn(0x55d165c896e0 msgr2=0x55d165c8bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.002+0000 7f394b8b2980 1 -- 172.21.15.170:0/2282477022 shutdown_connections 2024-03-28T16:49:40.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.002+0000 7f394b8b2980 1 -- 172.21.15.170:0/2282477022 wait complete. 2024-03-28T16:49:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 Processor -- start 2024-03-28T16:49:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- start start 2024-03-28T16:49:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d7b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165b9f620 con 0x55d165c030c0 2024-03-28T16:49:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d7b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d7b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:34996/0 (socket says 172.21.15.170:34996) 2024-03-28T16:49:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 -- 172.21.15.170:0/2886763449 learned_addr learned my addr 172.21.15.170:0/2886763449 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 -- 172.21.15.170:0/2886763449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3934005df0 con 0x55d165c030c0 2024-03-28T16:49:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 --2- 172.21.15.170:0/2886763449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d7b120 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f393c006fc0 tx=0x7f393c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f028b80444a8b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f3837606640 1 -- 172.21.15.170:0/2886763449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c013420 con 0x55d165c030c0 2024-03-28T16:49:40.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f3837606640 1 -- 172.21.15.170:0/2886763449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c013580 con 0x55d165c030c0 2024-03-28T16:49:40.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f3837606640 1 -- 172.21.15.170:0/2886763449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c013880 con 0x55d165c030c0 2024-03-28T16:49:40.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- 172.21.15.170:0/2886763449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 msgr2=0x55d165d7b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 --2- 172.21.15.170:0/2886763449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d7b120 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f393c006fc0 tx=0x7f393c0019b0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- 172.21.15.170:0/2886763449 shutdown_connections 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 --2- 172.21.15.170:0/2886763449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d7b120 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- 172.21.15.170:0/2886763449 >> 172.21.15.170:0/2886763449 conn(0x55d165c896e0 msgr2=0x55d165d76cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- 172.21.15.170:0/2886763449 shutdown_connections 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- 172.21.15.170:0/2886763449 wait complete. 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 Processor -- start 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- start start 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d81f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f394b8b2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165b9ae00 con 0x55d165c030c0 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d81f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d81f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35006/0 (socket says 172.21.15.170:35006) 2024-03-28T16:49:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.018+0000 7f39493cd640 1 -- 172.21.15.170:0/4064768257 learned_addr learned my addr 172.21.15.170:0/4064768257 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f39493cd640 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d165bdb4c0 con 0x55d165c030c0 2024-03-28T16:49:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f39493cd640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d81f70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f393c0012f0 tx=0x7f393c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3835e03640 1 -- 172.21.15.170:0/4064768257 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c006320 con 0x55d165c030c0 2024-03-28T16:49:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d165d824b0 con 0x55d165c030c0 2024-03-28T16:49:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3835e03640 1 -- 172.21.15.170:0/4064768257 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c006840 con 0x55d165c030c0 2024-03-28T16:49:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d165d828e0 con 0x55d165c030c0 2024-03-28T16:49:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3835e03640 1 -- 172.21.15.170:0/4064768257 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c004c50 con 0x55d165c030c0 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3835e03640 1 -- 172.21.15.170:0/4064768257 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f393c027400 con 0x55d165c030c0 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3835e03640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38300342b0 0x7f3830036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3835e03640 1 -- 172.21.15.170:0/4064768257 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f393c049740 con 0x55d165c030c0 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3948bcc640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38300342b0 0x7f3830036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165c956e0 0x55d165d88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165d89130 con 0x55d165c956e0 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3949bce640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165c956e0 0x55d165d88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3948bcc640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38300342b0 0x7f3830036770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f39340ae000 tx=0x7f39340a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3949bce640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165c956e0 0x55d165d88bf0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3949bce640 1 -- 172.21.15.170:0/4064768257 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165c956e0 2024-03-28T16:49:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165d89490 con 0x55d165c956e0 2024-03-28T16:49:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f3949bce640 1 -- 172.21.15.170:0/4064768257 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f39440083f0 con 0x55d165c956e0 2024-03-28T16:49:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165d8b200 0x55d165d8d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.022+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165d8db30 con 0x55d165d8b200 2024-03-28T16:49:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f39493cd640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165d8b200 0x55d165d8d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f39493cd640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165d8b200 0x55d165d8d5f0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f39493cd640 1 -- 172.21.15.170:0/4064768257 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f393c024070 con 0x55d165d8b200 2024-03-28T16:49:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165d8a040 con 0x55d165c956e0 2024-03-28T16:49:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f3949bce640 1 -- 172.21.15.170:0/4064768257 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f39440083f0 con 0x55d165c956e0 2024-03-28T16:49:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d165d8eea0 0x55d165d91290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165d918e0 con 0x55d165d8eea0 2024-03-28T16:49:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f3948bcc640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d165d8eea0 0x55d165d91290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f3948bcc640 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d165d8eea0 0x55d165d91290 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f3948bcc640 1 -- 172.21.15.170:0/4064768257 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f39340b0020 con 0x55d165d8eea0 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f394b8b2980 1 Processor -- start 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f394b8b2980 1 -- start start 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f394b8b2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165d9f7f0 0x55d165d9fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f394b8b2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165b33e30 con 0x55d165d9f7f0 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165d9f7f0 0x55d165d9fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165d9f7f0 0x55d165d9fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35018/0 (socket says 172.21.15.170:35018) 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.026+0000 7f39493cd640 1 -- 172.21.15.170:0/3456077356 learned_addr learned my addr 172.21.15.170:0/3456077356 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f39493cd640 1 -- 172.21.15.170:0/3456077356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d165da0150 con 0x55d165d9f7f0 2024-03-28T16:49:40.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f39493cd640 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165d9f7f0 0x55d165d9fc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f393c0064a0 tx=0x7f393c041880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f382effd640 1 -- 172.21.15.170:0/3456077356 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c037070 con 0x55d165d9f7f0 2024-03-28T16:49:40.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f382effd640 1 -- 172.21.15.170:0/3456077356 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c0109b0 con 0x55d165d9f7f0 2024-03-28T16:49:40.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d165da03a0 con 0x55d165d9f7f0 2024-03-28T16:49:40.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f382effd640 1 -- 172.21.15.170:0/3456077356 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c046040 con 0x55d165d9f7f0 2024-03-28T16:49:40.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d165da0ef0 con 0x55d165d9f7f0 2024-03-28T16:49:40.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f382effd640 1 -- 172.21.15.170:0/3456077356 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f393c049ae0 con 0x55d165d9f7f0 2024-03-28T16:49:40.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f382effd640 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38200342b0 0x7f3820036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f382effd640 1 -- 172.21.15.170:0/3456077356 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f393c03dbd0 con 0x55d165d9f7f0 2024-03-28T16:49:40.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f3948bcc640 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38200342b0 0x7f3820036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f394b8b2980 1 Processor -- start 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f394b8b2980 1 -- start start 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f3948bcc640 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38200342b0 0x7f3820036770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55d165d8a3a0 tx=0x7f3934005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f394b8b2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165dac980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f394b8b2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165b27790 con 0x55d165dac560 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165dac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165dac980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35022/0 (socket says 172.21.15.170:35022) 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f39493cd640 1 -- 172.21.15.170:0/3816634044 learned_addr learned my addr 172.21.15.170:0/3816634044 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.030+0000 7f39493cd640 1 -- 172.21.15.170:0/3816634044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d165dacf50 con 0x55d165dac560 2024-03-28T16:49:40.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f39493cd640 1 --2- 172.21.15.170:0/3816634044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165dac980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f393c0115a0 tx=0x7f393c041320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0cd1cc3f10ab17 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f382cff9640 1 -- 172.21.15.170:0/3816634044 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c0242c0 con 0x55d165dac560 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f382cff9640 1 -- 172.21.15.170:0/3816634044 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c0461a0 con 0x55d165dac560 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f382cff9640 1 -- 172.21.15.170:0/3816634044 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c00a0d0 con 0x55d165dac560 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- 172.21.15.170:0/3816634044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 msgr2=0x55d165dac980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3816634044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165dac980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f393c0115a0 tx=0x7f393c041320 comp rx=0 tx=0).stop 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- 172.21.15.170:0/3816634044 shutdown_connections 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3816634044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165dac980 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- 172.21.15.170:0/3816634044 >> 172.21.15.170:0/3816634044 conn(0x55d165da88f0 msgr2=0x55d165da6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- 172.21.15.170:0/3816634044 shutdown_connections 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- 172.21.15.170:0/3816634044 wait complete. 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 Processor -- start 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- start start 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165db33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d165db3920 con 0x55d165dac560 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165db33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f39493cd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165db33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35036/0 (socket says 172.21.15.170:35036) 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 learned_addr learned my addr 172.21.15.170:0/3161761752 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c040030 con 0x55d165dac560 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f39493cd640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165db33e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f393c0115a0 tx=0x7f393c029f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f381effd640 1 -- 172.21.15.170:0/3161761752 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c046310 con 0x55d165dac560 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d165db3b20 con 0x55d165dac560 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f381effd640 1 -- 172.21.15.170:0/3161761752 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c0242c0 con 0x55d165dac560 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f381effd640 1 -- 172.21.15.170:0/3161761752 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f393c011e40 con 0x55d165dac560 2024-03-28T16:49:40.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.034+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d165db3f50 con 0x55d165dac560 2024-03-28T16:49:40.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f381effd640 1 -- 172.21.15.170:0/3161761752 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f393c0244e0 con 0x55d165dac560 2024-03-28T16:49:40.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f381effd640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38140342b0 0x7f3814036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f381effd640 1 -- 172.21.15.170:0/3161761752 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f393c062070 con 0x55d165dac560 2024-03-28T16:49:40.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3948bcc640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38140342b0 0x7f3814036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165dbe7c0 0x55d165dc0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc13d0 con 0x55d165dbe7c0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3949bce640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165dbe7c0 0x55d165dc0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3948bcc640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38140342b0 0x7f3814036770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c10 tx=0x7f39340c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3949bce640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165dbe7c0 0x55d165dc0c80 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 realm 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc17a0 con 0x55d165dbe7c0 2024-03-28T16:49:40.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2020 con 0x55d165dbe7c0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165dc3090 0x55d165dc5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc5a90 con 0x55d165dc3090 2024-03-28T16:49:40.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f39493cd640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165dc3090 0x55d165dc5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.038+0000 7f39493cd640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165dc3090 0x55d165dc5480 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f393c024070 con 0x55d165dc3090 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f394b8b2980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f394b8b2980 20 searching for the correct realm 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d165dc7c90 0x55d165dca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dca820 con 0x55d165dc7c90 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3948bcc640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d165dc7c90 0x55d165dca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3948bcc640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d165dc7c90 0x55d165dca150 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3948bcc640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39340e0040 0x7f39340d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dca820 con 0x7f39340e0040 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39340e0040 0x7f39340d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39340e0040 0x7f39340d3400 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c024070 con 0x55d165dc3090 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.042+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x7f39340e0040 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x7f39340e0040 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.046+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x7f39340e0040 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.050+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x7f39340e0040 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc7c90 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dc3090 2024-03-28T16:49:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x55d165dbe7c0 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc2380 con 0x7f39340e0040 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc6090 con 0x55d165dbe7c0 2024-03-28T16:49:40.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.054+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc6090 con 0x55d165dbe7c0 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc6040 con 0x7f39340e0040 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 zone default found 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 4 Realm: () 2024-03-28T16:49:40.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 zonegroup default 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc68e0 con 0x55d165dc3090 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dc6ce0 con 0x55d165dc7c90 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcc6e0 con 0x55d165dc7c90 2024-03-28T16:49:40.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 started sync module instance, tier type = 2024-03-28T16:49:40.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:40.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcbf20 con 0x7f39340e0040 2024-03-28T16:49:40.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94357845490304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcc2d0 con 0x7f39340e0040 2024-03-28T16:49:40.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.058+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcd440 con 0x55d165dbe7c0 2024-03-28T16:49:40.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.062+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94357845490304] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.062+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.062+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94357845494528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcd3c0 con 0x55d165dbe7c0 2024-03-28T16:49:40.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.062+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcde70 con 0x55d165dc7c90 2024-03-28T16:49:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94357845494528] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94357845495584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcd340 con 0x55d165dc7c90 2024-03-28T16:49:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dceb90 con 0x7f39340e0040 2024-03-28T16:49:40.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94357845501984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcf3e0 con 0x7f39340e0040 2024-03-28T16:49:40.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcfdb0 con 0x55d165dbe7c0 2024-03-28T16:49:40.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94357845495584] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94357845501984] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94357845506224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dceef0 con 0x55d165dbe7c0 2024-03-28T16:49:40.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.066+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd0630 con 0x55d165dc3090 2024-03-28T16:49:40.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.070+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94357845506224] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.070+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.070+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94357845508496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dceef0 con 0x55d165dc3090 2024-03-28T16:49:40.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.070+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd0e30 con 0x55d165dc3090 2024-03-28T16:49:40.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.070+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.070+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94357845512672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd1aa0 con 0x55d165dc3090 2024-03-28T16:49:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.070+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd2470 con 0x55d165dc3090 2024-03-28T16:49:40.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94357845508496] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94357845512672] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94357845513728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dce250 con 0x55d165dc3090 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 20 add_watcher() i=0 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 20 add_watcher() i=1 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 20 add_watcher() i=2 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 20 add_watcher() i=3 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 20 add_watcher() i=4 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 20 add_watcher() i=5 2024-03-28T16:49:40.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.074+0000 7f394b8b2980 20 add_watcher() i=6 2024-03-28T16:49:40.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94357845513728] v31'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 20 add_watcher() i=7 2024-03-28T16:49:40.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:40.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd5850 con 0x55d165dbe7c0 2024-03-28T16:49:40.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94357845516144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd2cc0 con 0x55d165dbe7c0 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94357845516144] v31'187 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f37fffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 5 note: GC not initialized 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f37fefe5640 20 reqs_thread_entry: start 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 20 Filter name: none 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f37fcfe1640 20 reqs_thread_entry: start 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165df7670 con 0x55d165dbe7c0 2024-03-28T16:49:40.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:40.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:40.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:40.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:40.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.078+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~369 in=369b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dfd310 con 0x55d165dbe7c0 2024-03-28T16:49:40.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~369] v31'16 uv16 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f394b8b2980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:40.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f394b8b2980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:40.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f394b8b2980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:40.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94357845692336 in=557b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dfa030 con 0x55d165dbe7c0 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94357845506224 notify 133143986195 ret 0) v3 ==== 587+0+0 (crc 0 0 0) 0x7f394400aab0 con 0x55d165dbe7c0 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f382cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986195 cookie 94357845506224 notifier 4648 bl.length()=545 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f382cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f382cff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(64 notify.4 [notify cookie 94357845692336 out=8b] v0'0 uv180 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f382cff9640 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f380c003ea0 con 0x55d165dbe7c0 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== watch-notify(notify_complete (2) cookie 94357845692336 notify 133143986195 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f394400ac40 con 0x55d165dbe7c0 2024-03-28T16:49:40.085 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[{"type":"user","perm":"read"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:40.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f37fffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:40.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.082+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94357845516144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165de9770 con 0x55d165dbe7c0 2024-03-28T16:49:40.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94357845516144] v31'188 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94357845490304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd1f70 con 0x7f39340e0040 2024-03-28T16:49:40.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94357845490304] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f394b8b2980 20 remove_watcher() i=0 2024-03-28T16:49:40.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f394b8b2980 2 removed watcher, disabling cache 2024-03-28T16:49:40.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94357845494528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd1f70 con 0x55d165dbe7c0 2024-03-28T16:49:40.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94357845494528] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f394b8b2980 20 remove_watcher() i=1 2024-03-28T16:49:40.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.086+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94357845495584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dccb00 con 0x55d165dc7c90 2024-03-28T16:49:40.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.090+0000 7f3948bcc640 1 -- 172.21.15.170:0/3161761752 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94357845495584] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39340b0020 con 0x55d165dc7c90 2024-03-28T16:49:40.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.090+0000 7f394b8b2980 20 remove_watcher() i=2 2024-03-28T16:49:40.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.090+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94357845501984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd1f70 con 0x7f39340e0040 2024-03-28T16:49:40.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.090+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94357845501984] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x7f39340e0040 2024-03-28T16:49:40.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.090+0000 7f394b8b2980 20 remove_watcher() i=3 2024-03-28T16:49:40.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.090+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94357845506224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd1f70 con 0x55d165dbe7c0 2024-03-28T16:49:40.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.094+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94357845506224] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f39440083f0 con 0x55d165dbe7c0 2024-03-28T16:49:40.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.094+0000 7f394b8b2980 20 remove_watcher() i=4 2024-03-28T16:49:40.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.094+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94357845508496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcf8b0 con 0x55d165dc3090 2024-03-28T16:49:40.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.094+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94357845508496] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.094+0000 7f394b8b2980 20 remove_watcher() i=5 2024-03-28T16:49:40.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.094+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94357845512672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dd1f70 con 0x55d165dc3090 2024-03-28T16:49:40.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94357845512672] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 20 remove_watcher() i=6 2024-03-28T16:49:40.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94357845513728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d165dcf8b0 con 0x55d165dc3090 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f39493cd640 1 -- 172.21.15.170:0/3161761752 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94357845513728] v31'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f393c075040 con 0x55d165dc3090 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 20 remove_watcher() i=7 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165dc3090 msgr2=0x55d165dc5480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165dc3090 0x55d165dc5480 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165dbe7c0 msgr2=0x55d165dc0c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165dbe7c0 0x55d165dc0c80 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39340e0040 msgr2=0x7f39340d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f39340e0040 0x7f39340d3400 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d165dc7c90 msgr2=0x55d165dca150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d165dc7c90 0x55d165dca150 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38140342b0 msgr2=0x7f3814036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38140342b0 0x7f3814036770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c10 tx=0x7f39340c1040 comp rx=0 tx=0).stop 2024-03-28T16:49:40.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f3949bce640 1 -- 172.21.15.170:0/3161761752 reap_dead start 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 msgr2=0x55d165db33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165db33e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f393c0115a0 tx=0x7f393c029f50 comp rx=0 tx=0).stop 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 shutdown_connections 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3161761752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165dac560 0x55d165db33e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.098+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 >> 172.21.15.170:0/3161761752 conn(0x55d165da88f0 msgr2=0x55d165da5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 shutdown_connections 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3161761752 wait complete. 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38200342b0 msgr2=0x7f3820036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38200342b0 0x7f3820036770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55d165d8a3a0 tx=0x7f3934005ad0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165d9f7f0 msgr2=0x55d165d9fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165d9f7f0 0x55d165d9fc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f393c0064a0 tx=0x7f393c041880 comp rx=0 tx=0).stop 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 shutdown_connections 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38200342b0 0x7f3820036770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/3456077356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165d9f7f0 0x55d165d9fc10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 >> 172.21.15.170:0/3456077356 conn(0x55d165d94600 msgr2=0x55d165d95110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 shutdown_connections 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/3456077356 wait complete. 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165d8b200 msgr2=0x55d165d8d5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d165d8b200 0x55d165d8d5f0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165c956e0 msgr2=0x55d165d88bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d165c956e0 0x55d165d88bf0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d165d8eea0 msgr2=0x55d165d91290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d165d8eea0 0x55d165d91290 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38300342b0 msgr2=0x7f3830036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f38300342b0 0x7f3830036770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f39340ae000 tx=0x7f39340a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 msgr2=0x55d165d81f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 --2- 172.21.15.170:0/4064768257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d165c030c0 0x55d165d81f70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f393c0012f0 tx=0x7f393c001260 comp rx=0 tx=0).stop 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f3949bce640 1 -- 172.21.15.170:0/4064768257 reap_dead start 2024-03-28T16:49:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 shutdown_connections 2024-03-28T16:49:40.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 >> 172.21.15.170:0/4064768257 conn(0x55d165c896e0 msgr2=0x55d165d757c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 shutdown_connections 2024-03-28T16:49:40.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.102+0000 7f394b8b2980 1 -- 172.21.15.170:0/4064768257 wait complete. 2024-03-28T16:49:40.114 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:40.114 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-03-28T16:49:40.114 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-03-28T16:49:40.115 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-03-28T16:49:40.195 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:40.195 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd0584e4980 1 Processor -- start 2024-03-28T16:49:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd0584e4980 1 -- start start 2024-03-28T16:49:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd0584e4980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5591e9198140 0x5591e9198520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd0584e4980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5591e90e24c0 con 0x5591e9198a60 2024-03-28T16:49:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd0584e4980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5591e90a90f0 con 0x5591e9198140 2024-03-28T16:49:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd055fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5591e9198140 0x5591e9198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd055fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5591e9198140 0x5591e9198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35038/0 (socket says 172.21.15.170:35038) 2024-03-28T16:49:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd055fff640 1 -- 172.21.15.170:0/1003536176 learned_addr learned my addr 172.21.15.170:0/1003536176 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd0557fe640 1 --1- 172.21.15.170:0/1003536176 >> v1:172.21.15.119:6789/0 conn(0x5591e9198a60 0x5591e9199410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:41238/0 (socket says 172.21.15.170:41238) 2024-03-28T16:49:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd055fff640 1 -- 172.21.15.170:0/1003536176 >> v1:172.21.15.119:6789/0 conn(0x5591e9198a60 legacy=0x5591e9199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.194+0000 7fd055fff640 1 -- 172.21.15.170:0/1003536176 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e9199c40 con 0x5591e9198140 2024-03-28T16:49:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd055fff640 1 --2- 172.21.15.170:0/1003536176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9198520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd04000a5c0 tx=0x7fd04001b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f541592f59af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd054ffd640 1 -- 172.21.15.170:0/1003536176 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400aa470 con 0x5591e9198140 2024-03-28T16:49:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd054ffd640 1 -- 172.21.15.170:0/1003536176 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400aa5d0 con 0x5591e9198140 2024-03-28T16:49:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd054ffd640 1 -- 172.21.15.170:0/1003536176 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400aa890 con 0x5591e9198140 2024-03-28T16:49:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd0584e4980 1 -- 172.21.15.170:0/1003536176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 msgr2=0x5591e9198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1003536176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9198520 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd04000a5c0 tx=0x7fd04001b340 comp rx=0 tx=0).stop 2024-03-28T16:49:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd0584e4980 1 -- 172.21.15.170:0/1003536176 shutdown_connections 2024-03-28T16:49:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1003536176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9198520 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd0584e4980 1 -- 172.21.15.170:0/1003536176 >> 172.21.15.170:0/1003536176 conn(0x5591e910b860 msgr2=0x5591e910bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd0584e4980 1 -- 172.21.15.170:0/1003536176 shutdown_connections 2024-03-28T16:49:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.198+0000 7fd0584e4980 1 -- 172.21.15.170:0/1003536176 wait complete. 2024-03-28T16:49:40.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd0584e4980 1 Processor -- start 2024-03-28T16:49:40.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd0584e4980 1 -- start start 2024-03-28T16:49:40.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd0584e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9282610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd0584e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e90a6620 con 0x5591e9198140 2024-03-28T16:49:40.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9282610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9282610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35040/0 (socket says 172.21.15.170:35040) 2024-03-28T16:49:40.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd055fff640 1 -- 172.21.15.170:0/1718853809 learned_addr learned my addr 172.21.15.170:0/1718853809 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd055fff640 1 -- 172.21.15.170:0/1718853809 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd040005df0 con 0x5591e9198140 2024-03-28T16:49:40.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.214+0000 7fd055fff640 1 --2- 172.21.15.170:0/1718853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9282610 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd040018f70 tx=0x7fd040003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5a54e43f914e97 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fcf43e07640 1 -- 172.21.15.170:0/1718853809 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400aa9c0 con 0x5591e9198140 2024-03-28T16:49:40.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fcf43e07640 1 -- 172.21.15.170:0/1718853809 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400aab20 con 0x5591e9198140 2024-03-28T16:49:40.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fcf43e07640 1 -- 172.21.15.170:0/1718853809 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400aadc0 con 0x5591e9198140 2024-03-28T16:49:40.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- 172.21.15.170:0/1718853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 msgr2=0x5591e9282610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1718853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9282610 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd040018f70 tx=0x7fd040003030 comp rx=0 tx=0).stop 2024-03-28T16:49:40.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- 172.21.15.170:0/1718853809 shutdown_connections 2024-03-28T16:49:40.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1718853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9282610 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- 172.21.15.170:0/1718853809 >> 172.21.15.170:0/1718853809 conn(0x5591e910b860 msgr2=0x5591e927e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- 172.21.15.170:0/1718853809 shutdown_connections 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- 172.21.15.170:0/1718853809 wait complete. 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 Processor -- start 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- start start 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9289380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e90a1e00 con 0x5591e9198140 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9289380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9289380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35052/0 (socket says 172.21.15.170:35052) 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd055fff640 1 -- 172.21.15.170:0/1884573770 learned_addr learned my addr 172.21.15.170:0/1884573770 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd055fff640 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e9199c40 con 0x5591e9198140 2024-03-28T16:49:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd055fff640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9289380 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd040000c10 tx=0x7fd0400180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fcf42604640 1 -- 172.21.15.170:0/1884573770 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400c6650 con 0x5591e9198140 2024-03-28T16:49:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5591e92898c0 con 0x5591e9198140 2024-03-28T16:49:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fcf42604640 1 -- 172.21.15.170:0/1884573770 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400c67b0 con 0x5591e9198140 2024-03-28T16:49:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591e9289cf0 con 0x5591e9198140 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.218+0000 7fcf42604640 1 -- 172.21.15.170:0/1884573770 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400c6a50 con 0x5591e9198140 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fcf42604640 1 -- 172.21.15.170:0/1884573770 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd0400c6bb0 con 0x5591e9198140 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fcf42604640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf3c0342b0 0x7fcf3c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fcf42604640 1 -- 172.21.15.170:0/1884573770 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd0400e4b00 con 0x5591e9198140 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0557fe640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf3c0342b0 0x7fcf3c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e9198a60 0x5591e9290000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e9290540 con 0x5591e9198a60 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd056800640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e9198a60 0x5591e9290000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0557fe640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf3c0342b0 0x7fcf3c036770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd0480012f0 tx=0x7fd048006700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd056800640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e9198a60 0x5591e9290000 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd056800640 1 -- 172.21.15.170:0/1884573770 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e9198a60 2024-03-28T16:49:40.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92908a0 con 0x5591e9198a60 2024-03-28T16:49:40.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd056800640 1 -- 172.21.15.170:0/1884573770 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e9198a60 2024-03-28T16:49:40.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92925c0 0x5591e92949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e9294ef0 con 0x5591e92925c0 2024-03-28T16:49:40.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd055fff640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92925c0 0x5591e92949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd055fff640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92925c0 0x5591e92949b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd055fff640 1 -- 172.21.15.170:0/1884573770 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fd0400cd070 con 0x5591e92925c0 2024-03-28T16:49:40.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.222+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e9291400 con 0x5591e9198a60 2024-03-28T16:49:40.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd056800640 1 -- 172.21.15.170:0/1884573770 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e9198a60 2024-03-28T16:49:40.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5591e9296260 0x5591e9298650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e9298ca0 con 0x5591e9296260 2024-03-28T16:49:40.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0557fe640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5591e9296260 0x5591e9298650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0557fe640 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5591e9296260 0x5591e9298650 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0557fe640 1 -- 172.21.15.170:0/1884573770 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e9296260 2024-03-28T16:49:40.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 Processor -- start 2024-03-28T16:49:40.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 -- start start 2024-03-28T16:49:40.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92a6bd0 0x5591e92a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e903ae30 con 0x5591e92a6bd0 2024-03-28T16:49:40.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92a6bd0 0x5591e92a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92a6bd0 0x5591e92a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35066/0 (socket says 172.21.15.170:35066) 2024-03-28T16:49:40.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd055fff640 1 -- 172.21.15.170:0/4038795146 learned_addr learned my addr 172.21.15.170:0/4038795146 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd055fff640 1 -- 172.21.15.170:0/4038795146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e92a7530 con 0x5591e92a6bd0 2024-03-28T16:49:40.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd055fff640 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92a6bd0 0x5591e92a6ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd0400c5fd0 tx=0x7fd0400bdb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fcf3b7fe640 1 -- 172.21.15.170:0/4038795146 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400c5a60 con 0x5591e92a6bd0 2024-03-28T16:49:40.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fcf3b7fe640 1 -- 172.21.15.170:0/4038795146 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400e4040 con 0x5591e92a6bd0 2024-03-28T16:49:40.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5591e92a7780 con 0x5591e92a6bd0 2024-03-28T16:49:40.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fcf3b7fe640 1 -- 172.21.15.170:0/4038795146 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400bc6d0 con 0x5591e92a6bd0 2024-03-28T16:49:40.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.226+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591e92a82d0 con 0x5591e92a6bd0 2024-03-28T16:49:40.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fcf3b7fe640 1 -- 172.21.15.170:0/4038795146 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd0400bc830 con 0x5591e92a6bd0 2024-03-28T16:49:40.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fcf3b7fe640 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf300342b0 0x7fcf30036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fcf3b7fe640 1 -- 172.21.15.170:0/4038795146 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd0400f8bf0 con 0x5591e92a6bd0 2024-03-28T16:49:40.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0557fe640 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf300342b0 0x7fcf30036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 Processor -- start 2024-03-28T16:49:40.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- start start 2024-03-28T16:49:40.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0557fe640 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf300342b0 0x7fcf30036770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd048006300 tx=0x7fd04800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e902e790 con 0x5591e92b3bf0 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35072/0 (socket says 172.21.15.170:35072) 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd055fff640 1 -- 172.21.15.170:0/609518155 learned_addr learned my addr 172.21.15.170:0/609518155 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd055fff640 1 -- 172.21.15.170:0/609518155 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e92b45c0 con 0x5591e92b3bf0 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd055fff640 1 --2- 172.21.15.170:0/609518155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92b3ff0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d1ec0 tx=0x7fd0400e4a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995a7cdc220f5616 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fcf397fa640 1 -- 172.21.15.170:0/609518155 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400bcb30 con 0x5591e92b3bf0 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fcf397fa640 1 -- 172.21.15.170:0/609518155 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400ff070 con 0x5591e92b3bf0 2024-03-28T16:49:40.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fcf397fa640 1 -- 172.21.15.170:0/609518155 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400f0cb0 con 0x5591e92b3bf0 2024-03-28T16:49:40.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- 172.21.15.170:0/609518155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 msgr2=0x5591e92b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 --2- 172.21.15.170:0/609518155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92b3ff0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd0400d1ec0 tx=0x7fd0400e4a60 comp rx=0 tx=0).stop 2024-03-28T16:49:40.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- 172.21.15.170:0/609518155 shutdown_connections 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 --2- 172.21.15.170:0/609518155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92b3ff0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- 172.21.15.170:0/609518155 >> 172.21.15.170:0/609518155 conn(0x5591e92aff30 msgr2=0x5591e92adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- 172.21.15.170:0/609518155 shutdown_connections 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- 172.21.15.170:0/609518155 wait complete. 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 Processor -- start 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- start start 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92baa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.230+0000 7fd0584e4980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e92bafd0 con 0x5591e92b3bf0 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92baa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd055fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92baa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35084/0 (socket says 172.21.15.170:35084) 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 learned_addr learned my addr 172.21.15.170:0/2966290310 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e92a82d0 con 0x5591e92b3bf0 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd055fff640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92baa90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd0400c5e60 tx=0x7fd04000a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fcf277fe640 1 -- 172.21.15.170:0/2966290310 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400b33f0 con 0x5591e92b3bf0 2024-03-28T16:49:40.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fcf277fe640 1 -- 172.21.15.170:0/2966290310 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400b3550 con 0x5591e92b3bf0 2024-03-28T16:49:40.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fcf277fe640 1 -- 172.21.15.170:0/2966290310 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400b37f0 con 0x5591e92b3bf0 2024-03-28T16:49:40.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5591e92a82d0 con 0x5591e92b3bf0 2024-03-28T16:49:40.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591e92a82d0 con 0x5591e92b3bf0 2024-03-28T16:49:40.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fcf277fe640 1 -- 172.21.15.170:0/2966290310 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd0400cf7f0 con 0x5591e92b3bf0 2024-03-28T16:49:40.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fcf277fe640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf20034210 0x7fcf200366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fcf277fe640 1 -- 172.21.15.170:0/2966290310 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd04011abb0 con 0x5591e92b3bf0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0557fe640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf20034210 0x7fcf200366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e92c5cb0 0x5591e92c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c88c0 con 0x5591e92c5cb0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd056800640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e92c5cb0 0x5591e92c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0557fe640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf20034210 0x7fcf200366d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5591e9291760 tx=0x7fd048010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd056800640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e92c5cb0 0x5591e92c8170 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 20 realm 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.234+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c8c90 con 0x5591e92c5cb0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9870 con 0x5591e92c5cb0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92ca220 0x5591e92cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92ccc20 con 0x5591e92ca220 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd055fff640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92ca220 0x5591e92cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd055fff640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92ca220 0x5591e92cc610 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 20 searching for the correct realm 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5591e92cee00 0x5591e92d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d1990 con 0x5591e92cee00 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0557fe640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5591e92cee00 0x5591e92d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0557fe640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5591e92cee00 0x5591e92d12c0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0557fe640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd048024070 0x7fd048031970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d1990 con 0x7fd048024070 2024-03-28T16:49:40.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd056800640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd048024070 0x7fd048031970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.238+0000 7fd056800640 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd048024070 0x7fd048031970 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x7fd048024070 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.242+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x7fd048024070 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.246+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x7fd048024070 2024-03-28T16:49:40.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x7fd048024070 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92cee00 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92ca220 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.250+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x5591e92c5cb0 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92c9510 con 0x7fd048024070 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92cd200 con 0x5591e92c5cb0 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92cd200 con 0x5591e92c5cb0 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92cd1d0 con 0x7fd048024070 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 zone default found 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 4 Realm: () 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 zonegroup default 2024-03-28T16:49:40.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92cda50 con 0x5591e92ca220 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92cde30 con 0x5591e92cee00 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d3870 con 0x5591e92cee00 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 started sync module instance, tier type = 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.254+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d3090 con 0x7fd048024070 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.258+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.258+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94085170670576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d3440 con 0x7fd048024070 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.258+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d45d0 con 0x5591e92c5cb0 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.258+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94085170670576] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.258+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.258+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94085170674832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d45d0 con 0x5591e92c5cb0 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.258+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d50a0 con 0x5591e92cee00 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94085170674832] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94085170678656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d5450 con 0x5591e92cee00 2024-03-28T16:49:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d5dc0 con 0x7fd048024070 2024-03-28T16:49:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94085170682416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d6610 con 0x7fd048024070 2024-03-28T16:49:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d6fe0 con 0x5591e92c5cb0 2024-03-28T16:49:40.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94085170678656] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.262+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94085170682416] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.266+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.266+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94085170686688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d60f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.266+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d7830 con 0x5591e92ca220 2024-03-28T16:49:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.266+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94085170686688] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.266+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.266+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94085170688912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d60f0 con 0x5591e92ca220 2024-03-28T16:49:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.266+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d8030 con 0x5591e92ca220 2024-03-28T16:49:40.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94085170688912] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94085170690960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d7fb0 con 0x5591e92ca220 2024-03-28T16:49:40.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d8e90 con 0x5591e92ca220 2024-03-28T16:49:40.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94085170694640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d92b0 con 0x5591e92ca220 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 20 add_watcher() i=0 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 20 add_watcher() i=1 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 20 add_watcher() i=2 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 20 add_watcher() i=3 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 20 add_watcher() i=4 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 20 add_watcher() i=5 2024-03-28T16:49:40.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94085170690960] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.270+0000 7fd0584e4980 20 add_watcher() i=6 2024-03-28T16:49:40.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94085170694640] v31'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 20 add_watcher() i=7 2024-03-28T16:49:40.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:40.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d83e0 con 0x5591e92c5cb0 2024-03-28T16:49:40.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94085170695696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d83e0 con 0x5591e92c5cb0 2024-03-28T16:49:40.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94085170695696] v31'189 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fcf0bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:40.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:40.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:40.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 5 note: GC not initialized 2024-03-28T16:49:40.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fcf0afe5640 20 reqs_thread_entry: start 2024-03-28T16:49:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 20 Filter name: none 2024-03-28T16:49:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fcf08fe1640 20 reqs_thread_entry: start 2024-03-28T16:49:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.274+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92fe220 con 0x5591e92c5cb0 2024-03-28T16:49:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~369 out=369b,getxattrs out=75b] v0'0 uv16 ondisk = 0) v8 ==== 231+0+492 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 20 rados_obj.operate() r=0 bl.length=369 2024-03-28T16:49:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e9303ec0 con 0x5591e92c5cb0 2024-03-28T16:49:40.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v31'17 uv17 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:40.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94085170871136 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e9300be0 con 0x5591e92c5cb0 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== watch-notify(notify (1) cookie 94085170686688 notify 133143986196 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x7fd05000a930 con 0x5591e92c5cb0 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(64 notify.4 [notify cookie 94085170871136 out=8b] v0'0 uv183 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fcf397fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986196 cookie 94085170686688 notifier 4655 bl.length()=533 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fcf397fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fcf397fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fcf397fa640 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fcf18003ea0 con 0x5591e92c5cb0 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== watch-notify(notify_complete (2) cookie 94085170871136 notify 133143986196 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fd05000aac0 con 0x5591e92c5cb0 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.281 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:49:40.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fcf0bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:40.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.278+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94085170695696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92fcce0 con 0x5591e92c5cb0 2024-03-28T16:49:40.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94085170695696] v31'190 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94085170670576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d8e10 con 0x7fd048024070 2024-03-28T16:49:40.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94085170670576] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd0584e4980 20 remove_watcher() i=0 2024-03-28T16:49:40.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd0584e4980 2 removed watcher, disabling cache 2024-03-28T16:49:40.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94085170674832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d8e10 con 0x5591e92c5cb0 2024-03-28T16:49:40.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94085170674832] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd0584e4980 20 remove_watcher() i=1 2024-03-28T16:49:40.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.282+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94085170678656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d3c90 con 0x5591e92cee00 2024-03-28T16:49:40.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.286+0000 7fd0557fe640 1 -- 172.21.15.170:0/2966290310 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94085170678656] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0480044f0 con 0x5591e92cee00 2024-03-28T16:49:40.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.286+0000 7fd0584e4980 20 remove_watcher() i=2 2024-03-28T16:49:40.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.286+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94085170682416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d8e10 con 0x7fd048024070 2024-03-28T16:49:40.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.286+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94085170682416] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x7fd048024070 2024-03-28T16:49:40.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.286+0000 7fd0584e4980 20 remove_watcher() i=3 2024-03-28T16:49:40.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.286+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94085170686688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d5a30 con 0x5591e92c5cb0 2024-03-28T16:49:40.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.290+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94085170686688] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd0500083f0 con 0x5591e92c5cb0 2024-03-28T16:49:40.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.290+0000 7fd0584e4980 20 remove_watcher() i=4 2024-03-28T16:49:40.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.290+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94085170688912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d8e10 con 0x5591e92ca220 2024-03-28T16:49:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.290+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94085170688912] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.290+0000 7fd0584e4980 20 remove_watcher() i=5 2024-03-28T16:49:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.290+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94085170690960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d6ae0 con 0x5591e92ca220 2024-03-28T16:49:40.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94085170690960] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 20 remove_watcher() i=6 2024-03-28T16:49:40.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94085170694640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5591e92d5a30 con 0x5591e92ca220 2024-03-28T16:49:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd055fff640 1 -- 172.21.15.170:0/2966290310 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94085170694640] v31'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fd040116070 con 0x5591e92ca220 2024-03-28T16:49:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 20 remove_watcher() i=7 2024-03-28T16:49:40.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92ca220 msgr2=0x5591e92cc610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92ca220 0x5591e92cc610 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e92c5cb0 msgr2=0x5591e92c8170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e92c5cb0 0x5591e92c8170 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd048024070 msgr2=0x7fd048031970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fd048024070 0x7fd048031970 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5591e92cee00 msgr2=0x5591e92d12c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5591e92cee00 0x5591e92d12c0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf20034210 msgr2=0x7fcf200366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf20034210 0x7fcf200366d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5591e9291760 tx=0x7fd048010000 comp rx=0 tx=0).stop 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd056800640 1 -- 172.21.15.170:0/2966290310 reap_dead start 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 msgr2=0x5591e92baa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.294+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92baa90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd0400c5e60 tx=0x7fd04000a1d0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 shutdown_connections 2024-03-28T16:49:40.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/2966290310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92b3bf0 0x5591e92baa90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 >> 172.21.15.170:0/2966290310 conn(0x5591e92aff30 msgr2=0x5591e92ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 shutdown_connections 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/2966290310 wait complete. 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf300342b0 msgr2=0x7fcf30036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf300342b0 0x7fcf30036770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd048006300 tx=0x7fd04800f000 comp rx=0 tx=0).stop 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92a6bd0 msgr2=0x5591e92a6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92a6bd0 0x5591e92a6ff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd0400c5fd0 tx=0x7fd0400bdb50 comp rx=0 tx=0).stop 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 shutdown_connections 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf300342b0 0x7fcf30036770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/4038795146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e92a6bd0 0x5591e92a6ff0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 >> 172.21.15.170:0/4038795146 conn(0x5591e929b9e0 msgr2=0x5591e929c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 shutdown_connections 2024-03-28T16:49:40.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/4038795146 wait complete. 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92925c0 msgr2=0x5591e92949b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5591e92925c0 0x5591e92949b0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e9198a60 msgr2=0x5591e9290000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5591e9198a60 0x5591e9290000 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5591e9296260 msgr2=0x5591e9298650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5591e9296260 0x5591e9298650 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf3c0342b0 msgr2=0x7fcf3c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcf3c0342b0 0x7fcf3c036770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd0480012f0 tx=0x7fd048006700 comp rx=0 tx=0).stop 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 msgr2=0x5591e9289380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 --2- 172.21.15.170:0/1884573770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591e9198140 0x5591e9289380 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd040000c10 tx=0x7fd0400180e0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd056800640 1 -- 172.21.15.170:0/1884573770 reap_dead start 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 shutdown_connections 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 >> 172.21.15.170:0/1884573770 conn(0x5591e910b860 msgr2=0x5591e927ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 shutdown_connections 2024-03-28T16:49:40.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.298+0000 7fd0584e4980 1 -- 172.21.15.170:0/1884573770 wait complete. 2024-03-28T16:49:40.310 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:49:40.326 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:40.326 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-03-28T16:49:40.326 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-03-28T16:49:40.327 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-03-28T16:49:40.388 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:49:40.388 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:49:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.386+0000 7f8a93b69980 1 Processor -- start 2024-03-28T16:49:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.386+0000 7f8a93b69980 1 -- start start 2024-03-28T16:49:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.386+0000 7f8a93b69980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c20d1e4750 0x55c20d1e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.386+0000 7f8a93b69980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c20d12a520 con 0x55c20d1528b0 2024-03-28T16:49:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.386+0000 7f8a93b69980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c20d0f10f0 con 0x55c20d1e4750 2024-03-28T16:49:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a90e83640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c20d1e4750 0x55c20d1e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a91684640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55c20d1528b0 0x55c20d1e3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:41240/0 (socket says 172.21.15.170:41240) 2024-03-28T16:49:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a91684640 1 -- 172.21.15.170:0/26139567 learned_addr learned my addr 172.21.15.170:0/26139567 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:49:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a90e83640 1 -- 172.21.15.170:0/26139567 >> v1:172.21.15.119:6789/0 conn(0x55c20d1528b0 legacy=0x55c20d1e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a90e83640 1 -- 172.21.15.170:0/26139567 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20d1dea50 con 0x55c20d1e4750 2024-03-28T16:49:40.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a90e83640 1 --2- 172.21.15.170:0/26139567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1e4750 0x55c20d1e6b40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00a480 tx=0x7f8a7c01b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab59d7db817074e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a8bfff640 1 -- 172.21.15.170:0/26139567 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c0ab540 con 0x55c20d1e4750 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a8bfff640 1 -- 172.21.15.170:0/26139567 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c0ab6a0 con 0x55c20d1e4750 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a8bfff640 1 -- 172.21.15.170:0/26139567 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a7c0ab960 con 0x55c20d1e4750 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a93b69980 1 -- 172.21.15.170:0/26139567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1e4750 msgr2=0x55c20d1e6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.390+0000 7f8a93b69980 1 --2- 172.21.15.170:0/26139567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1e4750 0x55c20d1e6b40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00a480 tx=0x7f8a7c01b280 comp rx=0 tx=0).stop 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.394+0000 7f8a93b69980 1 -- 172.21.15.170:0/26139567 shutdown_connections 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.394+0000 7f8a93b69980 1 --2- 172.21.15.170:0/26139567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1e4750 0x55c20d1e6b40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.394+0000 7f8a93b69980 1 -- 172.21.15.170:0/26139567 >> 172.21.15.170:0/26139567 conn(0x55c20d1d8710 msgr2=0x55c20d1dab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.394+0000 7f8a93b69980 1 -- 172.21.15.170:0/26139567 shutdown_connections 2024-03-28T16:49:40.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.394+0000 7f8a93b69980 1 -- 172.21.15.170:0/26139567 wait complete. 2024-03-28T16:49:40.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.406+0000 7f8a93b69980 1 Processor -- start 2024-03-28T16:49:40.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.406+0000 7f8a93b69980 1 -- start start 2024-03-28T16:49:40.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.406+0000 7f8a93b69980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.406+0000 7f8a93b69980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20d0ee620 con 0x55c20d1528b0 2024-03-28T16:49:40.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.406+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.406+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35092/0 (socket says 172.21.15.170:35092) 2024-03-28T16:49:40.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.406+0000 7f8a91684640 1 -- 172.21.15.170:0/2554363811 learned_addr learned my addr 172.21.15.170:0/2554363811 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a91684640 1 -- 172.21.15.170:0/2554363811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c005d30 con 0x55c20d1528b0 2024-03-28T16:49:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a91684640 1 --2- 172.21.15.170:0/2554363811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2ca0b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c001590 tx=0x7f8a8c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835dcc87f8c9099b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f897f606640 1 -- 172.21.15.170:0/2554363811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c004530 con 0x55c20d1528b0 2024-03-28T16:49:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f897f606640 1 -- 172.21.15.170:0/2554363811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c004690 con 0x55c20d1528b0 2024-03-28T16:49:40.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f897f606640 1 -- 172.21.15.170:0/2554363811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c004950 con 0x55c20d1528b0 2024-03-28T16:49:40.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 -- 172.21.15.170:0/2554363811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 msgr2=0x55c20d2ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 --2- 172.21.15.170:0/2554363811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2ca0b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c001590 tx=0x7f8a8c001a50 comp rx=0 tx=0).stop 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 -- 172.21.15.170:0/2554363811 shutdown_connections 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 --2- 172.21.15.170:0/2554363811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2ca0b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 -- 172.21.15.170:0/2554363811 >> 172.21.15.170:0/2554363811 conn(0x55c20d1d8710 msgr2=0x55c20d2c5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 -- 172.21.15.170:0/2554363811 shutdown_connections 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 -- 172.21.15.170:0/2554363811 wait complete. 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 Processor -- start 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 -- start start 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a93b69980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20d0e9e00 con 0x55c20d1528b0 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35098/0 (socket says 172.21.15.170:35098) 2024-03-28T16:49:40.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a91684640 1 -- 172.21.15.170:0/219750755 learned_addr learned my addr 172.21.15.170:0/219750755 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a91684640 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20d12a520 con 0x55c20d1528b0 2024-03-28T16:49:40.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.410+0000 7f8a91684640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2d0f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c001360 tx=0x7f8a8c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f897de03640 1 -- 172.21.15.170:0/219750755 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c022070 con 0x55c20d1528b0 2024-03-28T16:49:40.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c20d2d1440 con 0x55c20d1528b0 2024-03-28T16:49:40.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f897de03640 1 -- 172.21.15.170:0/219750755 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c013780 con 0x55c20d1528b0 2024-03-28T16:49:40.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f897de03640 1 -- 172.21.15.170:0/219750755 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c013a60 con 0x55c20d1528b0 2024-03-28T16:49:40.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c20d2d1810 con 0x55c20d1528b0 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f897de03640 1 -- 172.21.15.170:0/219750755 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8a8c013cd0 con 0x55c20d1528b0 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f897de03640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89780342b0 0x7f8978036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f897de03640 1 -- 172.21.15.170:0/219750755 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8a8c048f90 con 0x55c20d1528b0 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a90e83640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89780342b0 0x7f8978036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d1e4750 0x55c20d2d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d2d80c0 con 0x55c20d1e4750 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a91e85640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d1e4750 0x55c20d2d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a90e83640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89780342b0 0x7f8978036770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0ad000 tx=0x7f8a7c0a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a91e85640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d1e4750 0x55c20d2d7b80 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a91e85640 1 -- 172.21.15.170:0/219750755 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d1e4750 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d2d8420 con 0x55c20d1e4750 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a91e85640 1 -- 172.21.15.170:0/219750755 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d1e4750 2024-03-28T16:49:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d2d9db0 0x55c20d2dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d2dc940 con 0x55c20d2d9db0 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.414+0000 7f8a91684640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d2d9db0 0x55c20d2dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91684640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d2d9db0 0x55c20d2dc1d0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91684640 1 -- 172.21.15.170:0/219750755 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d2d9db0 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d2d8c20 con 0x55c20d1e4750 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91e85640 1 -- 172.21.15.170:0/219750755 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d1e4750 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c20d2ddd60 0x55c20d2e0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d2e07f0 con 0x55c20d2ddd60 2024-03-28T16:49:40.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a90e83640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c20d2ddd60 0x55c20d2e0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a90e83640 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c20d2ddd60 0x55c20d2e0150 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a90e83640 1 -- 172.21.15.170:0/219750755 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d2ddd60 2024-03-28T16:49:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a93b69980 1 Processor -- start 2024-03-28T16:49:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a93b69980 1 -- start start 2024-03-28T16:49:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a93b69980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2ee700 0x55c20d2eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a93b69980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20d082e30 con 0x55c20d2ee700 2024-03-28T16:49:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2ee700 0x55c20d2eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2ee700 0x55c20d2eeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35114/0 (socket says 172.21.15.170:35114) 2024-03-28T16:49:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91684640 1 -- 172.21.15.170:0/4006524372 learned_addr learned my addr 172.21.15.170:0/4006524372 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91684640 1 -- 172.21.15.170:0/4006524372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20d2ef060 con 0x55c20d2ee700 2024-03-28T16:49:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.418+0000 7f8a91684640 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2ee700 0x55c20d2eeb20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0473d0 tx=0x7f8a8c026d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8976ffd640 1 -- 172.21.15.170:0/4006524372 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c0279f0 con 0x55c20d2ee700 2024-03-28T16:49:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c20d2ef2b0 con 0x55c20d2ee700 2024-03-28T16:49:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8976ffd640 1 -- 172.21.15.170:0/4006524372 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c026370 con 0x55c20d2ee700 2024-03-28T16:49:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8976ffd640 1 -- 172.21.15.170:0/4006524372 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c026610 con 0x55c20d2ee700 2024-03-28T16:49:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c20d2efe00 con 0x55c20d2ee700 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8976ffd640 1 -- 172.21.15.170:0/4006524372 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8a8c026770 con 0x55c20d2ee700 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8976ffd640 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89680342b0 0x7f8968036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8976ffd640 1 -- 172.21.15.170:0/4006524372 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8a8c032070 con 0x55c20d2ee700 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a90e83640 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89680342b0 0x7f8968036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 Processor -- start 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 -- start start 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a90e83640 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89680342b0 0x7f8968036770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55c20d2dccd0 tx=0x7f8a7c005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d2fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20d076790 con 0x55c20d2fb5f0 2024-03-28T16:49:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d2fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d2fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35120/0 (socket says 172.21.15.170:35120) 2024-03-28T16:49:40.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a91684640 1 -- 172.21.15.170:0/2072089529 learned_addr learned my addr 172.21.15.170:0/2072089529 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a91684640 1 -- 172.21.15.170:0/2072089529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20d2fbfe0 con 0x55c20d2fb5f0 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a91684640 1 --2- 172.21.15.170:0/2072089529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d2fba10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c047f90 tx=0x7f8a8c01e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42e90d2e995a299 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8974ff9640 1 -- 172.21.15.170:0/2072089529 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c05d070 con 0x55c20d2fb5f0 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8974ff9640 1 -- 172.21.15.170:0/2072089529 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c040c40 con 0x55c20d2fb5f0 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8974ff9640 1 -- 172.21.15.170:0/2072089529 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c024d10 con 0x55c20d2fb5f0 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 -- 172.21.15.170:0/2072089529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 msgr2=0x55c20d2fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 --2- 172.21.15.170:0/2072089529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d2fba10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c047f90 tx=0x7f8a8c01e8a0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 -- 172.21.15.170:0/2072089529 shutdown_connections 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 --2- 172.21.15.170:0/2072089529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d2fba10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.422+0000 7f8a93b69980 1 -- 172.21.15.170:0/2072089529 >> 172.21.15.170:0/2072089529 conn(0x55c20d2f7980 msgr2=0x55c20d2f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 -- 172.21.15.170:0/2072089529 shutdown_connections 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 -- 172.21.15.170:0/2072089529 wait complete. 2024-03-28T16:49:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 Processor -- start 2024-03-28T16:49:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 -- start start 2024-03-28T16:49:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d3024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20d302a00 con 0x55c20d2fb5f0 2024-03-28T16:49:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d3024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a91684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d3024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:35122/0 (socket says 172.21.15.170:35122) 2024-03-28T16:49:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 learned_addr learned my addr 172.21.15.170:0/1333472305 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:49:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20d2efe00 con 0x55c20d2fb5f0 2024-03-28T16:49:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a91684640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d3024c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0461a0 tx=0x7f8a8c031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8966ffd640 1 -- 172.21.15.170:0/1333472305 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c029e20 con 0x55c20d2fb5f0 2024-03-28T16:49:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8966ffd640 1 -- 172.21.15.170:0/1333472305 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c040fc0 con 0x55c20d2fb5f0 2024-03-28T16:49:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c20d2efe00 con 0x55c20d2fb5f0 2024-03-28T16:49:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8966ffd640 1 -- 172.21.15.170:0/1333472305 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a8c01e420 con 0x55c20d2fb5f0 2024-03-28T16:49:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c20d2efe00 con 0x55c20d2fb5f0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8966ffd640 1 -- 172.21.15.170:0/1333472305 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8a8c0242c0 con 0x55c20d2fb5f0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8966ffd640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f895c0342b0 0x7f895c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8966ffd640 1 -- 172.21.15.170:0/1333472305 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8a8c071bd0 con 0x55c20d2fb5f0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a90e83640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f895c0342b0 0x7f895c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d30d760 0x55c20d30fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310370 con 0x55c20d30d760 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a91e85640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d30d760 0x55c20d30fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.426+0000 7f8a90e83640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f895c0342b0 0x7f895c036770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0ab670 tx=0x7f8a7c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91e85640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d30d760 0x55c20d30fc20 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 realm 2024-03-28T16:49:40.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310740 con 0x55c20d30d760 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d311320 con 0x55c20d30d760 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d311cd0 0x55c20d3140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d314680 con 0x55c20d311cd0 2024-03-28T16:49:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91684640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d311cd0 0x55c20d3140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91684640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d311cd0 0x55c20d3140c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:49:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 20 searching for the correct realm 2024-03-28T16:49:40.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c20d316880 0x55c20d318d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d319410 con 0x55c20d316880 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a90e83640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c20d316880 0x55c20d318d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.430+0000 7f8a90e83640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c20d316880 0x55c20d318d40 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a90e83640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8a7c0d9040 0x7f8a7c0d8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d319410 con 0x7f8a7c0d9040 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8a7c0d9040 0x7f8a7c0d8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8a7c0d9040 0x7f8a7c0d8920 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.439 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.434+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.438+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d316880 2024-03-28T16:49:40.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.442+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d311cd0 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x55c20d30d760 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d310fc0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:49:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d314c80 con 0x55c20d30d760 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d314c80 con 0x55c20d30d760 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d314c30 con 0x7f8a7c0d9040 2024-03-28T16:49:40.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 zone default found 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 4 Realm: () 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 zonegroup default 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d3154d0 con 0x55c20d311cd0 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d3158d0 con 0x55c20d316880 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31b2f0 con 0x55c20d316880 2024-03-28T16:49:40.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:49:40.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 started sync module instance, tier type = 2024-03-28T16:49:40.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:49:40.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.446+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31ab10 con 0x7f8a7c0d9040 2024-03-28T16:49:40.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.450+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.450+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94291933374064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31aa90 con 0x7f8a7c0d9040 2024-03-28T16:49:40.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.450+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31c050 con 0x55c20d30d760 2024-03-28T16:49:40.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.450+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94291933374064] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.450+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.450+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94291933378320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31aa90 con 0x55c20d30d760 2024-03-28T16:49:40.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.450+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31c710 con 0x55c20d316880 2024-03-28T16:49:40.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94291933378320] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94291933381104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31cac0 con 0x55c20d316880 2024-03-28T16:49:40.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31d430 con 0x7f8a7c0d9040 2024-03-28T16:49:40.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94291933384864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31db40 con 0x7f8a7c0d9040 2024-03-28T16:49:40.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31e610 con 0x55c20d30d760 2024-03-28T16:49:40.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94291933381104] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.454+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94291933384864] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.458+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.458+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94291933389040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31e5d0 con 0x55c20d30d760 2024-03-28T16:49:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.458+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31bca0 con 0x55c20d311cd0 2024-03-28T16:49:40.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.458+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94291933389040] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.458+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.458+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94291933386144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31ee60 con 0x55c20d311cd0 2024-03-28T16:49:40.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.458+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31f660 con 0x55c20d311cd0 2024-03-28T16:49:40.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94291933395392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d320280 con 0x55c20d311cd0 2024-03-28T16:49:40.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d320c90 con 0x55c20d311cd0 2024-03-28T16:49:40.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94291933386144] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94291933395392] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94291933391296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31fde0 con 0x55c20d311cd0 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 20 add_watcher() i=0 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 20 add_watcher() i=1 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 20 add_watcher() i=2 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 20 add_watcher() i=3 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 20 add_watcher() i=4 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 20 add_watcher() i=5 2024-03-28T16:49:40.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.462+0000 7f8a93b69980 20 add_watcher() i=6 2024-03-28T16:49:40.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94291933391296] v31'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a93b69980 20 add_watcher() i=7 2024-03-28T16:49:40.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a93b69980 2 all 8 watchers are set, enabling cache 2024-03-28T16:49:40.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d323ea0 con 0x55c20d30d760 2024-03-28T16:49:40.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94291933398912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d321310 con 0x55c20d30d760 2024-03-28T16:49:40.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94291933398912] v31'191 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f89477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:40.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a93b69980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:49:40.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a93b69980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:49:40.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.466+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32c1d0 con 0x55c20d30d760 2024-03-28T16:49:40.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.470+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.470+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32c610 con 0x55c20d311cd0 2024-03-28T16:49:40.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.470+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.470+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32c610 con 0x55c20d316880 2024-03-28T16:49:40.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.470+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.470+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32cfe0 con 0x55c20d30d760 2024-03-28T16:49:40.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.474+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'192 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.474+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32a2b0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.474+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'65 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.474+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32a2b0 con 0x55c20d316880 2024-03-28T16:49:40.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.474+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'43 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.474+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32dd20 con 0x55c20d30d760 2024-03-28T16:49:40.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.478+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.478+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32dd20 con 0x55c20d311cd0 2024-03-28T16:49:40.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.478+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'44 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.478+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e250 con 0x7f8a7c0d9040 2024-03-28T16:49:40.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.482+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.482+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e250 con 0x55c20d316880 2024-03-28T16:49:40.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.482+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.482+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e780 con 0x7f8a7c0d9040 2024-03-28T16:49:40.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.482+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'66 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.482+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e780 con 0x55c20d311cd0 2024-03-28T16:49:40.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.486+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'28 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.486+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32ec20 con 0x7f8a7c0d9040 2024-03-28T16:49:40.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.486+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.486+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32ec20 con 0x55c20d311cd0 2024-03-28T16:49:40.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.490+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'50 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.490+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32ec20 con 0x55c20d30d760 2024-03-28T16:49:40.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.490+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'193 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.490+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32ec20 con 0x55c20d316880 2024-03-28T16:49:40.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.490+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'59 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.490+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32ea70 con 0x7f8a7c0d9040 2024-03-28T16:49:40.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.494+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'67 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.494+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32ea70 con 0x55c20d316880 2024-03-28T16:49:40.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.494+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.494+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32eb00 con 0x55c20d30d760 2024-03-28T16:49:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.498+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'33 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.498+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32eb00 con 0x55c20d30d760 2024-03-28T16:49:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.498+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'194 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.498+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32eb00 con 0x55c20d316880 2024-03-28T16:49:40.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.498+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'60 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.498+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32eb90 con 0x7f8a7c0d9040 2024-03-28T16:49:40.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'35 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32eb90 con 0x55c20d316880 2024-03-28T16:49:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'28 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32eb90 con 0x55c20d316880 2024-03-28T16:49:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32eb00 con 0x55c20d316880 2024-03-28T16:49:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'21 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.502+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e5d0 con 0x55c20d316880 2024-03-28T16:49:40.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.506+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'44 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.506+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e5d0 con 0x55c20d311cd0 2024-03-28T16:49:40.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.506+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'29 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.506+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e5d0 con 0x55c20d316880 2024-03-28T16:49:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.506+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.506+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e540 con 0x55c20d311cd0 2024-03-28T16:49:40.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.510+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'45 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.510+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e540 con 0x55c20d316880 2024-03-28T16:49:40.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.510+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'61 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.510+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e540 con 0x55c20d311cd0 2024-03-28T16:49:40.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.514+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'51 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.514+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d32e540 con 0x7f8a7c0d9040 2024-03-28T16:49:40.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.514+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f89467e4640 20 reqs_thread_entry: start 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 20 init_complete bucket index max shards: 11 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 20 Filter name: none 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f88b7fff640 20 reqs_thread_entry: start 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 20 rados->read ofs=0 len=0 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d3284e0 con 0x55c20d30d760 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:49:40.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:49:40.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:49:40.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:49:40.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d3284e0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv43 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.520 INFO:teuthology.orchestra.run.smithi170.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-03-28T16:49:40.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f89477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:49:40.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94291933398912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d34a490 con 0x55c20d30d760 2024-03-28T16:49:40.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94291933398912] v31'195 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.518+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94291933374064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d320780 con 0x7f8a7c0d9040 2024-03-28T16:49:40.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.522+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94291933374064] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.522+0000 7f8a93b69980 20 remove_watcher() i=0 2024-03-28T16:49:40.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.522+0000 7f8a93b69980 2 removed watcher, disabling cache 2024-03-28T16:49:40.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.522+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94291933378320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d320780 con 0x55c20d30d760 2024-03-28T16:49:40.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.522+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94291933378320] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.522+0000 7f8a93b69980 20 remove_watcher() i=1 2024-03-28T16:49:40.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.522+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94291933381104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31b710 con 0x55c20d316880 2024-03-28T16:49:40.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a90e83640 1 -- 172.21.15.170:0/1333472305 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94291933381104] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a7c0abc60 con 0x55c20d316880 2024-03-28T16:49:40.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a93b69980 20 remove_watcher() i=2 2024-03-28T16:49:40.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94291933384864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d320780 con 0x7f8a7c0d9040 2024-03-28T16:49:40.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94291933384864] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x7f8a7c0d9040 2024-03-28T16:49:40.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a93b69980 20 remove_watcher() i=3 2024-03-28T16:49:40.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94291933389040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31c1f0 con 0x55c20d30d760 2024-03-28T16:49:40.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94291933389040] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a840083f0 con 0x55c20d30d760 2024-03-28T16:49:40.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a93b69980 20 remove_watcher() i=4 2024-03-28T16:49:40.530 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.526+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94291933386144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31b710 con 0x55c20d311cd0 2024-03-28T16:49:40.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.530+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94291933386144] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.530+0000 7f8a93b69980 20 remove_watcher() i=5 2024-03-28T16:49:40.532 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.530+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94291933395392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d320780 con 0x55c20d311cd0 2024-03-28T16:49:40.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.530+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94291933395392] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.530+0000 7f8a93b69980 20 remove_watcher() i=6 2024-03-28T16:49:40.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.530+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94291933391296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c20d31c1f0 con 0x55c20d311cd0 2024-03-28T16:49:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a91684640 1 -- 172.21.15.170:0/1333472305 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94291933391296] v31'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8a8c024070 con 0x55c20d311cd0 2024-03-28T16:49:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 20 remove_watcher() i=7 2024-03-28T16:49:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d311cd0 msgr2=0x55c20d3140c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d311cd0 0x55c20d3140c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d30d760 msgr2=0x55c20d30fc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d30d760 0x55c20d30fc20 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8a7c0d9040 msgr2=0x7f8a7c0d8920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8a7c0d9040 0x7f8a7c0d8920 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c20d316880 msgr2=0x55c20d318d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c20d316880 0x55c20d318d40 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f895c0342b0 msgr2=0x7f895c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f895c0342b0 0x7f895c036770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0ab670 tx=0x7f8a7c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a91e85640 1 -- 172.21.15.170:0/1333472305 reap_dead start 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 msgr2=0x55c20d3024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d3024c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0461a0 tx=0x7f8a8c031f50 comp rx=0 tx=0).stop 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 shutdown_connections 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/1333472305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2fb5f0 0x55c20d3024c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 >> 172.21.15.170:0/1333472305 conn(0x55c20d2f7980 msgr2=0x55c20d2f4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 shutdown_connections 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/1333472305 wait complete. 2024-03-28T16:49:40.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89680342b0 msgr2=0x7f8968036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89680342b0 0x7f8968036770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55c20d2dccd0 tx=0x7f8a7c005a10 comp rx=0 tx=0).stop 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2ee700 msgr2=0x55c20d2eeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2ee700 0x55c20d2eeb20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0473d0 tx=0x7f8a8c026d90 comp rx=0 tx=0).stop 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 shutdown_connections 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89680342b0 0x7f8968036770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/4006524372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d2ee700 0x55c20d2eeb20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 >> 172.21.15.170:0/4006524372 conn(0x55c20d2e3510 msgr2=0x55c20d2e4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 shutdown_connections 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/4006524372 wait complete. 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d2d9db0 msgr2=0x55c20d2dc1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c20d2d9db0 0x55c20d2dc1d0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d1e4750 msgr2=0x55c20d2d7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c20d1e4750 0x55c20d2d7b80 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c20d2ddd60 msgr2=0x55c20d2e0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c20d2ddd60 0x55c20d2e0150 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89780342b0 msgr2=0x7f8978036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f89780342b0 0x7f8978036770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0ad000 tx=0x7f8a7c0a5040 comp rx=0 tx=0).stop 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 msgr2=0x55c20d2d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 --2- 172.21.15.170:0/219750755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c20d1528b0 0x55c20d2d0f00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c001360 tx=0x7f8a8c001260 comp rx=0 tx=0).stop 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a91e85640 1 -- 172.21.15.170:0/219750755 reap_dead start 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 shutdown_connections 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 >> 172.21.15.170:0/219750755 conn(0x55c20d1d8710 msgr2=0x55c20d2c4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 shutdown_connections 2024-03-28T16:49:40.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:49:40.534+0000 7f8a93b69980 1 -- 172.21.15.170:0/219750755 wait complete. 2024-03-28T16:49:40.547 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-03-28T16:49:40.566 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:49:40.571 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-03-28T16:50:15.572 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-03-28T16:50:15.573 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-03-28T16:50:15.573 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-03-28T16:50:15.612 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:15.612 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaef2cc980 1 Processor -- start 2024-03-28T16:50:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaef2cc980 1 -- start start 2024-03-28T16:50:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaef2cc980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d5944c0630 0x55d5944c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaef2cc980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d594406520 con 0x55d5944bc130 2024-03-28T16:50:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaef2cc980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d5943cd0f0 con 0x55d5944c0630 2024-03-28T16:50:15.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaec5e6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d5944c0630 0x55d5944c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaec5e6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55d5944c0630 0x55d5944c2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55222/0 (socket says 172.21.15.170:55222) 2024-03-28T16:50:15.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaec5e6640 1 -- 172.21.15.170:0/342942571 learned_addr learned my addr 172.21.15.170:0/342942571 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaec5e6640 1 -- 172.21.15.170:0/342942571 >> v1:172.21.15.119:6789/0 conn(0x55d5944bc130 legacy=0x55d5944bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaebde5640 1 -- 172.21.15.170:0/342942571 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538662542 0 0) 0x55d594406520 con 0x55d5944bc130 2024-03-28T16:50:15.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.612+0000 7fdaec5e6640 1 -- 172.21.15.170:0/342942571 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5944c3f70 con 0x55d5944c0630 2024-03-28T16:50:15.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/342942571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944c0630 0x55d5944c2a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdad400a5c0 tx=0x7fdad401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b3a0f81b4f2e77 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaebde5640 1 -- 172.21.15.170:0/342942571 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad40aa470 con 0x55d5944c0630 2024-03-28T16:50:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaebde5640 1 -- 172.21.15.170:0/342942571 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad40aa5d0 con 0x55d5944c0630 2024-03-28T16:50:15.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaebde5640 1 -- 172.21.15.170:0/342942571 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdad40aa890 con 0x55d5944c0630 2024-03-28T16:50:15.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaef2cc980 1 -- 172.21.15.170:0/342942571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944c0630 msgr2=0x55d5944c2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/342942571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944c0630 0x55d5944c2a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdad400a5c0 tx=0x7fdad401b340 comp rx=0 tx=0).stop 2024-03-28T16:50:15.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaef2cc980 1 -- 172.21.15.170:0/342942571 shutdown_connections 2024-03-28T16:50:15.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/342942571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944c0630 0x55d5944c2a20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaef2cc980 1 -- 172.21.15.170:0/342942571 >> 172.21.15.170:0/342942571 conn(0x55d59442f850 msgr2=0x55d59442fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaef2cc980 1 -- 172.21.15.170:0/342942571 shutdown_connections 2024-03-28T16:50:15.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.616+0000 7fdaef2cc980 1 -- 172.21.15.170:0/342942571 wait complete. 2024-03-28T16:50:15.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.632+0000 7fdaef2cc980 1 Processor -- start 2024-03-28T16:50:15.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.632+0000 7fdaef2cc980 1 -- start start 2024-03-28T16:50:15.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.632+0000 7fdaef2cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.632+0000 7fdaef2cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5943ca620 con 0x55d5944bc130 2024-03-28T16:50:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.632+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55230/0 (socket says 172.21.15.170:55230) 2024-03-28T16:50:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 -- 172.21.15.170:0/164972804 learned_addr learned my addr 172.21.15.170:0/164972804 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 -- 172.21.15.170:0/164972804 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4005df0 con 0x55d5944bc130 2024-03-28T16:50:15.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 --2- 172.21.15.170:0/164972804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945a5e50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdadc001670 tx=0x7fdadc001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c17e270d803a7e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fd9dae0d640 1 -- 172.21.15.170:0/164972804 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc0025d0 con 0x55d5944bc130 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fd9dae0d640 1 -- 172.21.15.170:0/164972804 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc002730 con 0x55d5944bc130 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fd9dae0d640 1 -- 172.21.15.170:0/164972804 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc002a30 con 0x55d5944bc130 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- 172.21.15.170:0/164972804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 msgr2=0x55d5945a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/164972804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945a5e50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdadc001670 tx=0x7fdadc001b30 comp rx=0 tx=0).stop 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- 172.21.15.170:0/164972804 shutdown_connections 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/164972804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945a5e50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- 172.21.15.170:0/164972804 >> 172.21.15.170:0/164972804 conn(0x55d59442f850 msgr2=0x55d5945a19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- 172.21.15.170:0/164972804 shutdown_connections 2024-03-28T16:50:15.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- 172.21.15.170:0/164972804 wait complete. 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 Processor -- start 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- start start 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5943c5e00 con 0x55d5944bc130 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55242/0 (socket says 172.21.15.170:55242) 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 -- 172.21.15.170:0/2595507627 learned_addr learned my addr 172.21.15.170:0/2595507627 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc006cc0 con 0x55d5944bc130 2024-03-28T16:50:15.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaecde7640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945acd00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdadc006fb0 tx=0x7fdadc001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fd9d960a640 1 -- 172.21.15.170:0/2595507627 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc012860 con 0x55d5944bc130 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fd9d960a640 1 -- 172.21.15.170:0/2595507627 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc0129c0 con 0x55d5944bc130 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.636+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5945ad240 con 0x55d5944bc130 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fd9d960a640 1 -- 172.21.15.170:0/2595507627 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc012c60 con 0x55d5944bc130 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5945ad670 con 0x55d5944bc130 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fd9d960a640 1 -- 172.21.15.170:0/2595507627 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fdadc019070 con 0x55d5944bc130 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fd9d960a640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9d40342b0 0x7fd9d4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fd9d960a640 1 -- 172.21.15.170:0/2595507627 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fdadc0492a0 con 0x55d5944bc130 2024-03-28T16:50:15.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9d40342b0 0x7fd9d4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5944c0630 0x55d5945b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945b3e70 con 0x55d5944c0630 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaed5e8640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5944c0630 0x55d5945b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9d40342b0 0x7fd9d4036770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdad40ae000 tx=0x7fdad40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaed5e8640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5944c0630 0x55d5945b3930 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaed5e8640 1 -- 172.21.15.170:0/2595507627 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5944c0630 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945b41d0 con 0x55d5944c0630 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaed5e8640 1 -- 172.21.15.170:0/2595507627 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5944c0630 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945b5b60 0x55d5945b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945b8770 con 0x55d5945b5b60 2024-03-28T16:50:15.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.640+0000 7fdaecde7640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945b5b60 0x55d5945b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaecde7640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945b5b60 0x55d5945b7fe0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaecde7640 1 -- 172.21.15.170:0/2595507627 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945b5b60 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945b49d0 con 0x55d5944c0630 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaed5e8640 1 -- 172.21.15.170:0/2595507627 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5944c0630 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d5945b9b90 0x55d5945bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945bc620 con 0x55d5945b9b90 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d5945b9b90 0x55d5945bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d5945b9b90 0x55d5945bbf80 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaec5e6640 1 -- 172.21.15.170:0/2595507627 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945b9b90 2024-03-28T16:50:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaef2cc980 1 Processor -- start 2024-03-28T16:50:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaef2cc980 1 -- start start 2024-03-28T16:50:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaef2cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945ca540 0x55d5945ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaef2cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d59435ee30 con 0x55d5945ca540 2024-03-28T16:50:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945ca540 0x55d5945ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945ca540 0x55d5945ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55248/0 (socket says 172.21.15.170:55248) 2024-03-28T16:50:15.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaecde7640 1 -- 172.21.15.170:0/1056959587 learned_addr learned my addr 172.21.15.170:0/1056959587 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.644+0000 7fdaecde7640 1 -- 172.21.15.170:0/1056959587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5945caea0 con 0x55d5945ca540 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaecde7640 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945ca540 0x55d5945ca960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdadc01cc20 tx=0x7fdadc02f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fd9d27fc640 1 -- 172.21.15.170:0/1056959587 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc036070 con 0x55d5945ca540 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5945cb0f0 con 0x55d5945ca540 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fd9d27fc640 1 -- 172.21.15.170:0/1056959587 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc01c390 con 0x55d5945ca540 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5945cbc40 con 0x55d5945ca540 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fd9d27fc640 1 -- 172.21.15.170:0/1056959587 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc01c630 con 0x55d5945ca540 2024-03-28T16:50:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fd9d27fc640 1 -- 172.21.15.170:0/1056959587 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fdadc01d390 con 0x55d5945ca540 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fd9d27fc640 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9c40342b0 0x7fd9c4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fd9d27fc640 1 -- 172.21.15.170:0/1056959587 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fdadc04ad70 con 0x55d5945ca540 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9c40342b0 0x7fd9c4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaef2cc980 1 Processor -- start 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaef2cc980 1 -- start start 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaef2cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945d7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9c40342b0 0x7fd9c4036770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55d5945b4d30 tx=0x7fdad4005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaef2cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d594352790 con 0x55d5945d7430 2024-03-28T16:50:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945d7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945d7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55256/0 (socket says 172.21.15.170:55256) 2024-03-28T16:50:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaecde7640 1 -- 172.21.15.170:0/2783832975 learned_addr learned my addr 172.21.15.170:0/2783832975 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaecde7640 1 -- 172.21.15.170:0/2783832975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5945d7de0 con 0x55d5945d7430 2024-03-28T16:50:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.648+0000 7fdaecde7640 1 --2- 172.21.15.170:0/2783832975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945d7810 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0478d0 tx=0x7fdadc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9059f1dfc0ca394 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fd9c3fff640 1 -- 172.21.15.170:0/2783832975 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc042df0 con 0x55d5945d7430 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fd9c3fff640 1 -- 172.21.15.170:0/2783832975 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc018280 con 0x55d5945d7430 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fd9c3fff640 1 -- 172.21.15.170:0/2783832975 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc018600 con 0x55d5945d7430 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2783832975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 msgr2=0x55d5945d7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2783832975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945d7810 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0478d0 tx=0x7fdadc01c8a0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2783832975 shutdown_connections 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2783832975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945d7810 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2783832975 >> 172.21.15.170:0/2783832975 conn(0x55d5945d37c0 msgr2=0x55d5945d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2783832975 shutdown_connections 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2783832975 wait complete. 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 Processor -- start 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- start start 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945de330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5945de870 con 0x55d5945d7430 2024-03-28T16:50:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945de330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaecde7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945de330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55262/0 (socket says 172.21.15.170:55262) 2024-03-28T16:50:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 learned_addr learned my addr 172.21.15.170:0/4176359668 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5945cbc40 con 0x55d5945d7430 2024-03-28T16:50:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaecde7640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945de330 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdadc03afd0 tx=0x7fdadc03ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fd9c27fc640 1 -- 172.21.15.170:0/4176359668 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc042df0 con 0x55d5945d7430 2024-03-28T16:50:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fd9c27fc640 1 -- 172.21.15.170:0/4176359668 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc031650 con 0x55d5945d7430 2024-03-28T16:50:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d5945cbc40 con 0x55d5945d7430 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fd9c27fc640 1 -- 172.21.15.170:0/4176359668 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdadc0318f0 con 0x55d5945d7430 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.652+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5945cbc40 con 0x55d5945d7430 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fd9c27fc640 1 -- 172.21.15.170:0/4176359668 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fdadc031a50 con 0x55d5945d7430 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fd9c27fc640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9b80342b0 0x7fd9b8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fd9c27fc640 1 -- 172.21.15.170:0/4176359668 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fdadc071bf0 con 0x55d5945d7430 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9b80342b0 0x7fd9b8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5945e94d0 0x55d5945eb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ec0c0 con 0x55d5945e94d0 2024-03-28T16:50:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaed5e8640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5945e94d0 0x55d5945eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9b80342b0 0x7fd9b8036770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdad4000c10 tx=0x7fdad40c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaed5e8640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5945e94d0 0x55d5945eb970 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 realm 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ec490 con 0x55d5945e94d0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ed040 con 0x55d5945e94d0 2024-03-28T16:50:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945ed9f0 0x55d5945efde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f0360 con 0x55d5945ed9f0 2024-03-28T16:50:15.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaecde7640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945ed9f0 0x55d5945efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.656+0000 7fdaecde7640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945ed9f0 0x55d5945efde0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaef2cc980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaef2cc980 20 searching for the correct realm 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5945f2260 0x55d5945f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f4d80 con 0x55d5945f2260 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5945f2260 0x55d5945f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5945f2260 0x55d5945f46b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaec5e6640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdad40e0040 0x7fdad40d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f4d80 con 0x7fdad40e0040 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdad40e0040 0x7fdad40d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdad40e0040 0x7fdad40d3400 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.660+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x7fdad40e0040 2024-03-28T16:50:15.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x7fdad40e0040 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.664+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x7fdad40e0040 2024-03-28T16:50:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.668+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x7fdad40e0040 2024-03-28T16:50:15.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945f2260 2024-03-28T16:50:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945ed9f0 2024-03-28T16:50:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x55d5945e94d0 2024-03-28T16:50:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ecce0 con 0x7fdad40e0040 2024-03-28T16:50:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f0bd0 con 0x55d5945e94d0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.672+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f0bd0 con 0x55d5945e94d0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f0ba0 con 0x7fdad40e0040 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 zone default found 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 4 Realm: () 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 zonegroup default 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f1420 con 0x55d5945ed9f0 2024-03-28T16:50:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f1820 con 0x55d5945f2260 2024-03-28T16:50:15.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f6ba0 con 0x55d5945f2260 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 started sync module instance, tier type = 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f6480 con 0x7fdad40e0040 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94375805673440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f6830 con 0x7fdad40e0040 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.676+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f7930 con 0x55d5945e94d0 2024-03-28T16:50:15.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.680+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94375805673440] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.680+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.680+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94375805677552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f7970 con 0x55d5945e94d0 2024-03-28T16:50:15.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.680+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f83e0 con 0x55d5945f2260 2024-03-28T16:50:15.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.680+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.680+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94375805681344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f8c30 con 0x55d5945f2260 2024-03-28T16:50:15.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.680+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f95e0 con 0x7fdad40e0040 2024-03-28T16:50:15.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94375805677552] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94375805678608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f99c0 con 0x7fdad40e0040 2024-03-28T16:50:15.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fa330 con 0x55d5945e94d0 2024-03-28T16:50:15.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94375805681344] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94375805678608] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94375805686016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f9e90 con 0x55d5945e94d0 2024-03-28T16:50:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.684+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fa690 con 0x55d5945ed9f0 2024-03-28T16:50:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94375805686016] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94375805689488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fa2b0 con 0x55d5945ed9f0 2024-03-28T16:50:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fb330 con 0x55d5945ed9f0 2024-03-28T16:50:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94375805695632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fbf50 con 0x55d5945ed9f0 2024-03-28T16:50:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fc900 con 0x55d5945ed9f0 2024-03-28T16:50:15.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.688+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94375805689488] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94375805695632] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94375805696688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fc480 con 0x55d5945ed9f0 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 20 add_watcher() i=0 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 20 add_watcher() i=1 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 20 add_watcher() i=2 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 20 add_watcher() i=3 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 20 add_watcher() i=4 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 20 add_watcher() i=5 2024-03-28T16:50:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.692+0000 7fdaef2cc980 20 add_watcher() i=6 2024-03-28T16:50:15.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94375805696688] v31'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 20 add_watcher() i=7 2024-03-28T16:50:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945ffb70 con 0x55d5945e94d0 2024-03-28T16:50:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94375805692400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fd080 con 0x55d5945e94d0 2024-03-28T16:50:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94375805692400] v31'196 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fd9a2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 5 note: GC not initialized 2024-03-28T16:50:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fd9a1fe3640 20 reqs_thread_entry: start 2024-03-28T16:50:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:15.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 20 Filter name: none 2024-03-28T16:50:15.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fd9137fe640 20 reqs_thread_entry: start 2024-03-28T16:50:15.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594622280 con 0x55d5945e94d0 2024-03-28T16:50:15.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594622280 con 0x7fdad40e0040 2024-03-28T16:50:15.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594622280 con 0x55d5945e94d0 2024-03-28T16:50:15.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594622280 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.696+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5946225e0 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5946225e0 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5946225e0 con 0x55d5945e94d0 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5946225e0 con 0x55d5945e94d0 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5946225e0 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5946225e0 con 0x55d5945f2260 2024-03-28T16:50:15.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5946225e0 con 0x55d5945ed9f0 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623fe0 con 0x55d5945e94d0 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623fe0 con 0x7fdad40e0040 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623fe0 con 0x55d5945f2260 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.700+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623fe0 con 0x55d5945ed9f0 2024-03-28T16:50:15.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623fe0 con 0x55d5945ed9f0 2024-03-28T16:50:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623f90 con 0x55d5945f2260 2024-03-28T16:50:15.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945ed9f0 2024-03-28T16:50:15.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945e94d0 2024-03-28T16:50:15.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.704+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945f2260 2024-03-28T16:50:15.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945ed9f0 2024-03-28T16:50:15.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x7fdad40e0040 2024-03-28T16:50:15.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945ed9f0 2024-03-28T16:50:15.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945e94d0 2024-03-28T16:50:15.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945e94d0 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623ea0 con 0x55d5945f2260 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623e50 con 0x55d5945e94d0 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623e20 con 0x55d5945e94d0 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623e00 con 0x55d5945f2260 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623e00 con 0x55d5945f2260 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.708+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623e00 con 0x55d5945f2260 2024-03-28T16:50:15.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.712+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.712+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d594623e00 con 0x55d5945ed9f0 2024-03-28T16:50:15.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.712+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.712+0000 7fd9a2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:15.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.712+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94375805692400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d59461ffd0 con 0x55d5945e94d0 2024-03-28T16:50:15.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.712+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94375805692400] v31'197 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.712+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94375805673440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fa9f0 con 0x7fdad40e0040 2024-03-28T16:50:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.716+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94375805673440] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.716+0000 7fdaef2cc980 20 remove_watcher() i=0 2024-03-28T16:50:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.716+0000 7fdaef2cc980 2 removed watcher, disabling cache 2024-03-28T16:50:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.716+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94375805677552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fa9f0 con 0x55d5945e94d0 2024-03-28T16:50:15.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.716+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94375805677552] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.716+0000 7fdaef2cc980 20 remove_watcher() i=1 2024-03-28T16:50:15.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.716+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94375805681344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f6ff0 con 0x55d5945f2260 2024-03-28T16:50:15.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.720+0000 7fdaec5e6640 1 -- 172.21.15.170:0/4176359668 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94375805681344] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdad40b0020 con 0x55d5945f2260 2024-03-28T16:50:15.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.720+0000 7fdaef2cc980 20 remove_watcher() i=2 2024-03-28T16:50:15.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.720+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94375805678608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fa9f0 con 0x7fdad40e0040 2024-03-28T16:50:15.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.720+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94375805678608] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x7fdad40e0040 2024-03-28T16:50:15.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.720+0000 7fdaef2cc980 20 remove_watcher() i=3 2024-03-28T16:50:15.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.720+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94375805686016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f7ec0 con 0x55d5945e94d0 2024-03-28T16:50:15.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.724+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94375805686016] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdae40083f0 con 0x55d5945e94d0 2024-03-28T16:50:15.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.724+0000 7fdaef2cc980 20 remove_watcher() i=4 2024-03-28T16:50:15.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.724+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94375805689488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945fa9f0 con 0x55d5945ed9f0 2024-03-28T16:50:15.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.724+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94375805689488] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.724+0000 7fdaef2cc980 20 remove_watcher() i=5 2024-03-28T16:50:15.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.724+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94375805695632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f9100 con 0x55d5945ed9f0 2024-03-28T16:50:15.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94375805695632] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 20 remove_watcher() i=6 2024-03-28T16:50:15.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94375805696688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55d5945f7ec0 con 0x55d5945ed9f0 2024-03-28T16:50:15.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaecde7640 1 -- 172.21.15.170:0/4176359668 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94375805696688] v31'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fdadc01c9d0 con 0x55d5945ed9f0 2024-03-28T16:50:15.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 20 remove_watcher() i=7 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945ed9f0 msgr2=0x55d5945efde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945ed9f0 0x55d5945efde0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5945e94d0 msgr2=0x55d5945eb970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5945e94d0 0x55d5945eb970 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdad40e0040 msgr2=0x7fdad40d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fdad40e0040 0x7fdad40d3400 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5945f2260 msgr2=0x55d5945f46b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55d5945f2260 0x55d5945f46b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9b80342b0 msgr2=0x7fd9b8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9b80342b0 0x7fd9b8036770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdad4000c10 tx=0x7fdad40c1040 comp rx=0 tx=0).stop 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 msgr2=0x55d5945de330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/4176359668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945d7430 0x55d5945de330 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdadc03afd0 tx=0x7fdadc03ced0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 reap_dead start 2024-03-28T16:50:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaed5e8640 1 -- 172.21.15.170:0/4176359668 reap_dead start 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 shutdown_connections 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 >> 172.21.15.170:0/4176359668 conn(0x55d5945d37c0 msgr2=0x55d5945d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 shutdown_connections 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/4176359668 wait complete. 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9c40342b0 msgr2=0x7fd9c4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9c40342b0 0x7fd9c4036770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55d5945b4d30 tx=0x7fdad4005ad0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945ca540 msgr2=0x55d5945ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945ca540 0x55d5945ca960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdadc01cc20 tx=0x7fdadc02f720 comp rx=0 tx=0).stop 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 shutdown_connections 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9c40342b0 0x7fd9c4036770 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/1056959587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5945ca540 0x55d5945ca960 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 >> 172.21.15.170:0/1056959587 conn(0x55d5945bf350 msgr2=0x55d5945bfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 shutdown_connections 2024-03-28T16:50:15.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.728+0000 7fdaef2cc980 1 -- 172.21.15.170:0/1056959587 wait complete. 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945b5b60 msgr2=0x55d5945b7fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55d5945b5b60 0x55d5945b7fe0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5944c0630 msgr2=0x55d5945b3930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55d5944c0630 0x55d5945b3930 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d5945b9b90 msgr2=0x55d5945bbf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55d5945b9b90 0x55d5945bbf80 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9d40342b0 msgr2=0x7fd9d4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd9d40342b0 0x7fd9d4036770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdad40ae000 tx=0x7fdad40a6000 comp rx=0 tx=0).stop 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 msgr2=0x55d5945acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 --2- 172.21.15.170:0/2595507627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5944bc130 0x55d5945acd00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdadc006fb0 tx=0x7fdadc001260 comp rx=0 tx=0).stop 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaed5e8640 1 -- 172.21.15.170:0/2595507627 reap_dead start 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 shutdown_connections 2024-03-28T16:50:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 >> 172.21.15.170:0/2595507627 conn(0x55d59442f850 msgr2=0x55d5945a04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 shutdown_connections 2024-03-28T16:50:15.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.732+0000 7fdaef2cc980 1 -- 172.21.15.170:0/2595507627 wait complete. 2024-03-28T16:50:15.742 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"anonymous","categories":[]}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}]},{"bucket":"myfoo2","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo3","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo4","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]}]}],"summary":[{"user":"anonymous","categories":[],"total":{"bytes_sent":0,"bytes_received":0,"ops":0,"successful_ops":0}},{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}}]} 2024-03-28T16:50:15.742 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'anonymous', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-03-28T16:50:15.742 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-03-28T16:50:15.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-03-28T16:50:15.743 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-03-28T16:50:15.818 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a948c980 1 Processor -- start 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a948c980 1 -- start start 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a948c980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e38f3c08e0 0x55e38f450fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a948c980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e38f398510 con 0x55e38f451520 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a948c980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e38f35f0f0 con 0x55e38f3c08e0 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a6fa7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e38f3c08e0 0x55e38f450fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a6fa7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55e38f3c08e0 0x55e38f450fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55268/0 (socket says 172.21.15.170:55268) 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3951378794 learned_addr learned my addr 172.21.15.170:0/3951378794 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a67a6640 1 --1- 172.21.15.170:0/3951378794 >> v1:172.21.15.119:6789/0 conn(0x55e38f451520 0x55e38f453910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:46064/0 (socket says 172.21.15.170:46064) 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3951378794 >> v1:172.21.15.119:6789/0 conn(0x55e38f451520 legacy=0x55e38f453910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.816+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3951378794 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38f456040 con 0x55e38f3c08e0 2024-03-28T16:50:15.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/3951378794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f450fe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc39800a5c0 tx=0x7fc39801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f384e1436589ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a5fa5640 1 -- 172.21.15.170:0/3951378794 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980aa470 con 0x55e38f3c08e0 2024-03-28T16:50:15.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a5fa5640 1 -- 172.21.15.170:0/3951378794 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980aa5d0 con 0x55e38f3c08e0 2024-03-28T16:50:15.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a5fa5640 1 -- 172.21.15.170:0/3951378794 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980aa890 con 0x55e38f3c08e0 2024-03-28T16:50:15.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a948c980 1 -- 172.21.15.170:0/3951378794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 msgr2=0x55e38f450fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3951378794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f450fe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc39800a5c0 tx=0x7fc39801b340 comp rx=0 tx=0).stop 2024-03-28T16:50:15.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a948c980 1 -- 172.21.15.170:0/3951378794 shutdown_connections 2024-03-28T16:50:15.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3951378794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f450fe0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a948c980 1 -- 172.21.15.170:0/3951378794 >> 172.21.15.170:0/3951378794 conn(0x55e38f446700 msgr2=0x55e38f448b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a948c980 1 -- 172.21.15.170:0/3951378794 shutdown_connections 2024-03-28T16:50:15.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.820+0000 7fc3a948c980 1 -- 172.21.15.170:0/3951378794 wait complete. 2024-03-28T16:50:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a948c980 1 Processor -- start 2024-03-28T16:50:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a948c980 1 -- start start 2024-03-28T16:50:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a948c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f538080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a948c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38f35c620 con 0x55e38f3c08e0 2024-03-28T16:50:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f538080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f538080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55276/0 (socket says 172.21.15.170:55276) 2024-03-28T16:50:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3147683978 learned_addr learned my addr 172.21.15.170:0/3147683978 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3147683978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398005df0 con 0x55e38f3c08e0 2024-03-28T16:50:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/3147683978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f538080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc3980ac040 tx=0x7fc398003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37d4ba51ff0a5b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc294e09640 1 -- 172.21.15.170:0/3147683978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980aa9b0 con 0x55e38f3c08e0 2024-03-28T16:50:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc294e09640 1 -- 172.21.15.170:0/3147683978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980aab30 con 0x55e38f3c08e0 2024-03-28T16:50:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.836+0000 7fc294e09640 1 -- 172.21.15.170:0/3147683978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980aae10 con 0x55e38f3c08e0 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- 172.21.15.170:0/3147683978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 msgr2=0x55e38f538080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3147683978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f538080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc3980ac040 tx=0x7fc398003030 comp rx=0 tx=0).stop 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- 172.21.15.170:0/3147683978 shutdown_connections 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3147683978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f538080 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- 172.21.15.170:0/3147683978 >> 172.21.15.170:0/3147683978 conn(0x55e38f446700 msgr2=0x55e38f533c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- 172.21.15.170:0/3147683978 shutdown_connections 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- 172.21.15.170:0/3147683978 wait complete. 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 Processor -- start 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- start start 2024-03-28T16:50:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f53ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38f357e00 con 0x55e38f3c08e0 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f53ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f53ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55284/0 (socket says 172.21.15.170:55284) 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/162342522 learned_addr learned my addr 172.21.15.170:0/162342522 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38f456040 con 0x55e38f3c08e0 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f53ed60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc398005940 tx=0x7fc3980180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc293606640 1 -- 172.21.15.170:0/162342522 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980c6750 con 0x55e38f3c08e0 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc293606640 1 -- 172.21.15.170:0/162342522 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980c68b0 con 0x55e38f3c08e0 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e38f53f2a0 con 0x55e38f3c08e0 2024-03-28T16:50:15.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc293606640 1 -- 172.21.15.170:0/162342522 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980c6b70 con 0x55e38f3c08e0 2024-03-28T16:50:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.840+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e38f53f6d0 con 0x55e38f3c08e0 2024-03-28T16:50:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc293606640 1 -- 172.21.15.170:0/162342522 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc3980c6cd0 con 0x55e38f3c08e0 2024-03-28T16:50:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc293606640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc28c0342b0 0x7fc28c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc293606640 1 -- 172.21.15.170:0/162342522 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc3980e4270 con 0x55e38f3c08e0 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc28c0342b0 0x7fc28c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f451520 0x55e38f5459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f545f20 con 0x55e38f451520 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a77a8640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f451520 0x55e38f5459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc28c0342b0 0x7fc28c036770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc39c001e50 tx=0x7fc39c0013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a77a8640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f451520 0x55e38f5459e0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a77a8640 1 -- 172.21.15.170:0/162342522 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f451520 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f546280 con 0x55e38f451520 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a77a8640 1 -- 172.21.15.170:0/162342522 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f451520 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f547fa0 0x55e38f54a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f54a8d0 con 0x55e38f547fa0 2024-03-28T16:50:15.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f547fa0 0x55e38f54a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.844+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f547fa0 0x55e38f54a390 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/162342522 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fc3980b0020 con 0x55e38f547fa0 2024-03-28T16:50:15.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f546d80 con 0x55e38f451520 2024-03-28T16:50:15.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a77a8640 1 -- 172.21.15.170:0/162342522 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f451520 2024-03-28T16:50:15.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e38f54bc40 0x55e38f54e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f54e600 con 0x55e38f54bc40 2024-03-28T16:50:15.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e38f54bc40 0x55e38f54e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e38f54bc40 0x55e38f54e030 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a67a6640 1 -- 172.21.15.170:0/162342522 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fc39c00f700 con 0x55e38f54bc40 2024-03-28T16:50:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 Processor -- start 2024-03-28T16:50:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 -- start start 2024-03-28T16:50:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f55c530 0x55e38f55c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38f2f0e30 con 0x55e38f55c530 2024-03-28T16:50:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f55c530 0x55e38f55c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f55c530 0x55e38f55c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55290/0 (socket says 172.21.15.170:55290) 2024-03-28T16:50:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/232212435 learned_addr learned my addr 172.21.15.170:0/232212435 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/232212435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38f55ce90 con 0x55e38f55c530 2024-03-28T16:50:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f55c530 0x55e38f55c950 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc3980aa5a0 tx=0x7fc3980c5e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc290e01640 1 -- 172.21.15.170:0/232212435 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980acc90 con 0x55e38f55c530 2024-03-28T16:50:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e38f55d0e0 con 0x55e38f55c530 2024-03-28T16:50:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc290e01640 1 -- 172.21.15.170:0/232212435 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980acdf0 con 0x55e38f55c530 2024-03-28T16:50:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e38f55dc30 con 0x55e38f55c530 2024-03-28T16:50:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.848+0000 7fc290e01640 1 -- 172.21.15.170:0/232212435 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980b3520 con 0x55e38f55c530 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc290e01640 1 -- 172.21.15.170:0/232212435 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc3980b3680 con 0x55e38f55c530 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc290e01640 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2800342b0 0x7fc280036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc290e01640 1 -- 172.21.15.170:0/232212435 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc3980edbf0 con 0x55e38f55c530 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2800342b0 0x7fc280036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 Processor -- start 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 -- start start 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2800342b0 0x7fc280036770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55e38f5470e0 tx=0x7fc39c00f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f569890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38f2e4790 con 0x55e38f5694b0 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f569890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f569890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55296/0 (socket says 172.21.15.170:55296) 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/2598057920 learned_addr learned my addr 172.21.15.170:0/2598057920 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/2598057920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38f569e60 con 0x55e38f5694b0 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/2598057920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f569890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc3980e14f0 tx=0x7fc3980bce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66384ff95d353a12 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc28a7fc640 1 -- 172.21.15.170:0/2598057920 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980fc070 con 0x55e38f5694b0 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc28a7fc640 1 -- 172.21.15.170:0/2598057920 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980b5c90 con 0x55e38f5694b0 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc28a7fc640 1 -- 172.21.15.170:0/2598057920 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980e2bd0 con 0x55e38f5694b0 2024-03-28T16:50:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 -- 172.21.15.170:0/2598057920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 msgr2=0x55e38f569890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 --2- 172.21.15.170:0/2598057920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f569890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc3980e14f0 tx=0x7fc3980bce00 comp rx=0 tx=0).stop 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 -- 172.21.15.170:0/2598057920 shutdown_connections 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 --2- 172.21.15.170:0/2598057920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f569890 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 -- 172.21.15.170:0/2598057920 >> 172.21.15.170:0/2598057920 conn(0x55e38f5657f0 msgr2=0x55e38f563850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 -- 172.21.15.170:0/2598057920 shutdown_connections 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 -- 172.21.15.170:0/2598057920 wait complete. 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.852+0000 7fc3a948c980 1 Processor -- start 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 1 -- start start 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f570350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38f570890 con 0x55e38f5694b0 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f570350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a6fa7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f570350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55300/0 (socket says 172.21.15.170:55300) 2024-03-28T16:50:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 learned_addr learned my addr 172.21.15.170:0/3507426441 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38f55dc30 con 0x55e38f5694b0 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f570350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc3980b3e90 tx=0x7fc3980afef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc288ff9640 1 -- 172.21.15.170:0/3507426441 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc398108070 con 0x55e38f5694b0 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e38f55dc30 con 0x55e38f5694b0 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc288ff9640 1 -- 172.21.15.170:0/3507426441 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980e4820 con 0x55e38f5694b0 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc288ff9640 1 -- 172.21.15.170:0/3507426441 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3980ac3b0 con 0x55e38f5694b0 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e38f55dc30 con 0x55e38f5694b0 2024-03-28T16:50:15.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc288ff9640 1 -- 172.21.15.170:0/3507426441 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc3980b7a90 con 0x55e38f5694b0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc288ff9640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2700342b0 0x7fc270036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc288ff9640 1 -- 172.21.15.170:0/3507426441 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc398102bd0 con 0x55e38f5694b0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2700342b0 0x7fc270036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f57b4e0 0x55e38f57d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57e0d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a77a8640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f57b4e0 0x55e38f57d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2700342b0 0x7fc270036770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55e38f54e9c0 tx=0x7fc39c010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.856+0000 7fc3a77a8640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f57b4e0 0x55e38f57d980 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 realm 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57e4a0 con 0x55e38f57b4e0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57f050 con 0x55e38f57b4e0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f57fa00 0x55e38f581df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f582370 con 0x55e38f57fa00 2024-03-28T16:50:15.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f57fa00 0x55e38f581df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a6fa7640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f57fa00 0x55e38f581df0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc3980b0020 con 0x55e38f57fa00 2024-03-28T16:50:15.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 20 searching for the correct realm 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e38f584270 0x55e38f586680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f586d50 con 0x55e38f584270 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e38f584270 0x55e38f586680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.860+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e38f584270 0x55e38f586680 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a67a6640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc39c01fbe0 0x7fc39c03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f586d50 con 0x7fc39c01fbe0 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc39c01fbe0 0x7fc39c03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc39c01fbe0 0x7fc39c03a3c0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3980b0020 con 0x55e38f57fa00 2024-03-28T16:50:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3980b0020 con 0x55e38f57fa00 2024-03-28T16:50:15.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3980b0020 con 0x55e38f57fa00 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.864+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3980b0020 con 0x55e38f57fa00 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.868+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f584270 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.872+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57fa00 2024-03-28T16:50:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x55e38f57b4e0 2024-03-28T16:50:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f57ecf0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f582be0 con 0x55e38f57b4e0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f582be0 con 0x55e38f57b4e0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f582bb0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 zone default found 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 4 Realm: () 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 zonegroup default 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f583430 con 0x55e38f57fa00 2024-03-28T16:50:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f583810 con 0x55e38f584270 2024-03-28T16:50:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f588cd0 con 0x55e38f584270 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 started sync module instance, tier type = 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.876+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f588450 con 0x7fc39c01fbe0 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.880+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.880+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94435850878896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f588800 con 0x7fc39c01fbe0 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.880+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f589a30 con 0x55e38f57b4e0 2024-03-28T16:50:15.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.880+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94435850878896] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.880+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.880+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94435850883312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f589a10 con 0x55e38f57b4e0 2024-03-28T16:50:15.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.880+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58a4a0 con 0x55e38f584270 2024-03-28T16:50:15.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94435850887008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58acf0 con 0x55e38f584270 2024-03-28T16:50:15.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58b6a0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94435850883312] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94435850884368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58ba50 con 0x7fc39c01fbe0 2024-03-28T16:50:15.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58c3c0 con 0x55e38f57b4e0 2024-03-28T16:50:15.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.884+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94435850887008] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94435850884368] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94435850891712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58bf20 con 0x55e38f57b4e0 2024-03-28T16:50:15.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58cc10 con 0x55e38f57fa00 2024-03-28T16:50:15.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94435850891712] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94435850897264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58bf20 con 0x55e38f57fa00 2024-03-28T16:50:15.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.888+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58d410 con 0x55e38f57fa00 2024-03-28T16:50:15.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.892+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94435850897264] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.892+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.892+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94435850889296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58dc60 con 0x55e38f57fa00 2024-03-28T16:50:15.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.892+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58e5d0 con 0x55e38f57fa00 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94435850889296] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94435850901632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58e5d0 con 0x55e38f57fa00 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=0 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=1 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=2 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=3 2024-03-28T16:50:15.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=4 2024-03-28T16:50:15.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=5 2024-03-28T16:50:15.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=6 2024-03-28T16:50:15.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94435850901632] v31'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 20 add_watcher() i=7 2024-03-28T16:50:15.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:15.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f591920 con 0x55e38f57b4e0 2024-03-28T16:50:15.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94435850916672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58e740 con 0x55e38f57b4e0 2024-03-28T16:50:15.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.896+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94435850916672] v31'198 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc25d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:15.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:15.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:15.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 5 note: GC not initialized 2024-03-28T16:50:15.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc1d7fff640 20 reqs_thread_entry: start 2024-03-28T16:50:15.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:15.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 20 Filter name: none 2024-03-28T16:50:15.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc1d5ffb640 20 reqs_thread_entry: start 2024-03-28T16:50:15.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b4090 con 0x55e38f57b4e0 2024-03-28T16:50:15.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b4090 con 0x7fc39c01fbe0 2024-03-28T16:50:15.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b4090 con 0x55e38f57b4e0 2024-03-28T16:50:15.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b4090 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b43f0 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b43f0 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b43f0 con 0x55e38f57b4e0 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b43f0 con 0x55e38f57b4e0 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b43f0 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b43f0 con 0x55e38f584270 2024-03-28T16:50:15.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.900+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b43f0 con 0x55e38f57fa00 2024-03-28T16:50:15.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5da0 con 0x55e38f57b4e0 2024-03-28T16:50:15.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5da0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5da0 con 0x55e38f584270 2024-03-28T16:50:15.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5da0 con 0x55e38f57fa00 2024-03-28T16:50:15.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.904+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5da0 con 0x55e38f57fa00 2024-03-28T16:50:15.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5d50 con 0x55e38f584270 2024-03-28T16:50:15.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5d50 con 0x55e38f57fa00 2024-03-28T16:50:15.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5d50 con 0x55e38f57b4e0 2024-03-28T16:50:15.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5d50 con 0x55e38f584270 2024-03-28T16:50:15.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5d00 con 0x55e38f57fa00 2024-03-28T16:50:15.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5d00 con 0x7fc39c01fbe0 2024-03-28T16:50:15.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.908+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f57fa00 2024-03-28T16:50:15.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f57b4e0 2024-03-28T16:50:15.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f57b4e0 2024-03-28T16:50:15.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f584270 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f57b4e0 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f57b4e0 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f584270 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f584270 2024-03-28T16:50:15.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f584270 2024-03-28T16:50:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b5ce0 con 0x55e38f57fa00 2024-03-28T16:50:15.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.912+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc25d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:15.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94435850916672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f5b2770 con 0x55e38f57b4e0 2024-03-28T16:50:15.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94435850916672] v31'199 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94435850878896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f591340 con 0x7fc39c01fbe0 2024-03-28T16:50:15.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94435850878896] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc3a948c980 20 remove_watcher() i=0 2024-03-28T16:50:15.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc3a948c980 2 removed watcher, disabling cache 2024-03-28T16:50:15.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.916+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94435850883312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f587fb0 con 0x55e38f57b4e0 2024-03-28T16:50:15.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.920+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94435850883312] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.920+0000 7fc3a948c980 20 remove_watcher() i=1 2024-03-28T16:50:15.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.920+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94435850887008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f587fb0 con 0x55e38f584270 2024-03-28T16:50:15.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.920+0000 7fc3a67a6640 1 -- 172.21.15.170:0/3507426441 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94435850887008] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39c033040 con 0x55e38f584270 2024-03-28T16:50:15.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.920+0000 7fc3a948c980 20 remove_watcher() i=2 2024-03-28T16:50:15.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.920+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94435850884368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f589f60 con 0x7fc39c01fbe0 2024-03-28T16:50:15.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.924+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94435850884368] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x7fc39c01fbe0 2024-03-28T16:50:15.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.924+0000 7fc3a948c980 20 remove_watcher() i=3 2024-03-28T16:50:15.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.924+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94435850891712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f589f60 con 0x55e38f57b4e0 2024-03-28T16:50:15.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.924+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94435850891712] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc3a000a4d0 con 0x55e38f57b4e0 2024-03-28T16:50:15.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.924+0000 7fc3a948c980 20 remove_watcher() i=4 2024-03-28T16:50:15.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.924+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94435850897264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58b1c0 con 0x55e38f57fa00 2024-03-28T16:50:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.928+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94435850897264] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.928+0000 7fc3a948c980 20 remove_watcher() i=5 2024-03-28T16:50:15.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.928+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94435850889296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f589f60 con 0x55e38f57fa00 2024-03-28T16:50:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.928+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94435850889296] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.928+0000 7fc3a948c980 20 remove_watcher() i=6 2024-03-28T16:50:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.928+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94435850901632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55e38f58b1c0 con 0x55e38f57fa00 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a6fa7640 1 -- 172.21.15.170:0/3507426441 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94435850901632] v31'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc39811b040 con 0x55e38f57fa00 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 20 remove_watcher() i=7 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f57fa00 msgr2=0x55e38f581df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f57fa00 0x55e38f581df0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f57b4e0 msgr2=0x55e38f57d980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f57b4e0 0x55e38f57d980 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc39c01fbe0 msgr2=0x7fc39c03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc39c01fbe0 0x7fc39c03a3c0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e38f584270 msgr2=0x55e38f586680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55e38f584270 0x55e38f586680 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2700342b0 msgr2=0x7fc270036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2700342b0 0x7fc270036770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55e38f54e9c0 tx=0x7fc39c010000 comp rx=0 tx=0).stop 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 msgr2=0x55e38f570350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/3507426441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f5694b0 0x55e38f570350 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc3980b3e90 tx=0x7fc3980afef0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 reap_dead start 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a77a8640 1 -- 172.21.15.170:0/3507426441 reap_dead start 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 shutdown_connections 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 >> 172.21.15.170:0/3507426441 conn(0x55e38f5657f0 msgr2=0x55e38f562e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 shutdown_connections 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/3507426441 wait complete. 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2800342b0 msgr2=0x7fc280036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2800342b0 0x7fc280036770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55e38f5470e0 tx=0x7fc39c00f000 comp rx=0 tx=0).stop 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f55c530 msgr2=0x55e38f55c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f55c530 0x55e38f55c950 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc3980aa5a0 tx=0x7fc3980c5e20 comp rx=0 tx=0).stop 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 shutdown_connections 2024-03-28T16:50:15.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc2800342b0 0x7fc280036770 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/232212435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f55c530 0x55e38f55c950 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 >> 172.21.15.170:0/232212435 conn(0x55e38f551340 msgr2=0x55e38f551e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 shutdown_connections 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/232212435 wait complete. 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f547fa0 msgr2=0x55e38f54a390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55e38f547fa0 0x55e38f54a390 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f451520 msgr2=0x55e38f5459e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55e38f451520 0x55e38f5459e0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e38f54bc40 msgr2=0x55e38f54e030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.932+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55e38f54bc40 0x55e38f54e030 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc28c0342b0 msgr2=0x7fc28c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc28c0342b0 0x7fc28c036770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc39c001e50 tx=0x7fc39c0013b0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 msgr2=0x55e38f53ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 --2- 172.21.15.170:0/162342522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e38f3c08e0 0x55e38f53ed60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc398005940 tx=0x7fc3980180e0 comp rx=0 tx=0).stop 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a77a8640 1 -- 172.21.15.170:0/162342522 reap_dead start 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 shutdown_connections 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 >> 172.21.15.170:0/162342522 conn(0x55e38f446700 msgr2=0x55e38f532720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 shutdown_connections 2024-03-28T16:50:15.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:15.936+0000 7fc3a948c980 1 -- 172.21.15.170:0/162342522 wait complete. 2024-03-28T16:50:15.946 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"anonymous","categories":[{"category":"list_buckets","bytes_sent":237,"bytes_received":0,"ops":1,"successful_ops":1}]}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}]},{"bucket":"myfoo2","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"anonymous","categories":[{"category":"list_buckets","bytes_sent":237,"bytes_received":0,"ops":1,"successful_ops":1}],"total":{"bytes_sent":237,"bytes_received":0,"ops":1,"successful_ops":1}},{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34}}]} 2024-03-28T16:50:15.946 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 237, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 237, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 237, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-03-28T16:50:15.947 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-03-28T16:50:15.947 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-03-28T16:50:15.947 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-03-28T16:50:16.022 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:16.022 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:16.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d3a59980 1 Processor -- start 2024-03-28T16:50:16.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d3a59980 1 -- start start 2024-03-28T16:50:16.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d3a59980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5606754f68b0 0x560675584c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d3a59980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5606754ce520 con 0x560675585170 2024-03-28T16:50:16.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d3a59980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5606754950f0 con 0x5606754f68b0 2024-03-28T16:50:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d1574640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5606754f68b0 0x560675584c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d1574640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5606754f68b0 0x560675584c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55310/0 (socket says 172.21.15.170:55310) 2024-03-28T16:50:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d1574640 1 -- 172.21.15.170:0/1392037763 learned_addr learned my addr 172.21.15.170:0/1392037763 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d0d73640 1 --1- 172.21.15.170:0/1392037763 >> v1:172.21.15.119:6789/0 conn(0x560675585170 0x560675587560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:46076/0 (socket says 172.21.15.170:46076) 2024-03-28T16:50:16.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09c3fff640 1 -- 172.21.15.170:0/1392037763 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073841546 0 0) 0x5606754ce520 con 0x560675585170 2024-03-28T16:50:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d1574640 1 -- 172.21.15.170:0/1392037763 >> v1:172.21.15.119:6789/0 conn(0x560675585170 legacy=0x560675587560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.024+0000 7f09d1574640 1 -- 172.21.15.170:0/1392037763 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560675589cc0 con 0x5606754f68b0 2024-03-28T16:50:16.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d1574640 1 --2- 172.21.15.170:0/1392037763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675584c30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a5c0 tx=0x7f09c401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa06d086a2afd5f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09c3fff640 1 -- 172.21.15.170:0/1392037763 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40aa470 con 0x5606754f68b0 2024-03-28T16:50:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09c3fff640 1 -- 172.21.15.170:0/1392037763 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40aa5d0 con 0x5606754f68b0 2024-03-28T16:50:16.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09c3fff640 1 -- 172.21.15.170:0/1392037763 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40aa890 con 0x5606754f68b0 2024-03-28T16:50:16.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d3a59980 1 -- 172.21.15.170:0/1392037763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 msgr2=0x560675584c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1392037763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675584c30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a5c0 tx=0x7f09c401b340 comp rx=0 tx=0).stop 2024-03-28T16:50:16.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d3a59980 1 -- 172.21.15.170:0/1392037763 shutdown_connections 2024-03-28T16:50:16.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1392037763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675584c30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d3a59980 1 -- 172.21.15.170:0/1392037763 >> 172.21.15.170:0/1392037763 conn(0x56067557c710 msgr2=0x56067557eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d3a59980 1 -- 172.21.15.170:0/1392037763 shutdown_connections 2024-03-28T16:50:16.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.028+0000 7f09d3a59980 1 -- 172.21.15.170:0/1392037763 wait complete. 2024-03-28T16:50:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 Processor -- start 2024-03-28T16:50:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 -- start start 2024-03-28T16:50:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675669a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560675492620 con 0x5606754f68b0 2024-03-28T16:50:16.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675669a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675669a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55320/0 (socket says 172.21.15.170:55320) 2024-03-28T16:50:16.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d1574640 1 -- 172.21.15.170:0/1901889633 learned_addr learned my addr 172.21.15.170:0/1901889633 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d1574640 1 -- 172.21.15.170:0/1901889633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4005df0 con 0x5606754f68b0 2024-03-28T16:50:16.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d1574640 1 --2- 172.21.15.170:0/1901889633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675669a50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f09c40ac040 tx=0x7f09c4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116275c0e8b82046 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f08bf606640 1 -- 172.21.15.170:0/1901889633 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40aa9b0 con 0x5606754f68b0 2024-03-28T16:50:16.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f08bf606640 1 -- 172.21.15.170:0/1901889633 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40aab30 con 0x5606754f68b0 2024-03-28T16:50:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f08bf606640 1 -- 172.21.15.170:0/1901889633 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40aae10 con 0x5606754f68b0 2024-03-28T16:50:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 -- 172.21.15.170:0/1901889633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 msgr2=0x560675669a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1901889633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675669a50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f09c40ac040 tx=0x7f09c4003030 comp rx=0 tx=0).stop 2024-03-28T16:50:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 -- 172.21.15.170:0/1901889633 shutdown_connections 2024-03-28T16:50:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1901889633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x560675669a50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 -- 172.21.15.170:0/1901889633 >> 172.21.15.170:0/1901889633 conn(0x56067557c710 msgr2=0x5606756655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 -- 172.21.15.170:0/1901889633 shutdown_connections 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.044+0000 7f09d3a59980 1 -- 172.21.15.170:0/1901889633 wait complete. 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 Processor -- start 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 -- start start 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x5606756707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067548de00 con 0x5606754f68b0 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x5606756707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x5606756707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55334/0 (socket says 172.21.15.170:55334) 2024-03-28T16:50:16.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d1574640 1 -- 172.21.15.170:0/3458951821 learned_addr learned my addr 172.21.15.170:0/3458951821 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d1574640 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560675589cc0 con 0x5606754f68b0 2024-03-28T16:50:16.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d1574640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x5606756707a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005940 tx=0x7f09c40180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f08bde03640 1 -- 172.21.15.170:0/3458951821 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40c6750 con 0x5606754f68b0 2024-03-28T16:50:16.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f08bde03640 1 -- 172.21.15.170:0/3458951821 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40c68b0 con 0x5606754f68b0 2024-03-28T16:50:16.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560675670ce0 con 0x5606754f68b0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f08bde03640 1 -- 172.21.15.170:0/3458951821 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40c6b70 con 0x5606754f68b0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560675671110 con 0x5606754f68b0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f08bde03640 1 -- 172.21.15.170:0/3458951821 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f09c40c6cd0 con 0x5606754f68b0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f08bde03640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08b80342b0 0x7f08b8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f08bde03640 1 -- 172.21.15.170:0/3458951821 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f09c40e4270 con 0x5606754f68b0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d0d73640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08b80342b0 0x7f08b8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560675585170 0x5606756773d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560675677910 con 0x560675585170 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.048+0000 7f09d1d75640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560675585170 0x5606756773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d0d73640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08b80342b0 0x7f08b8036770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c10 tx=0x7f09c80013c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d1d75640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560675585170 0x5606756773d0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d1d75640 1 -- 172.21.15.170:0/3458951821 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x560675585170 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560675677c70 con 0x560675585170 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d1d75640 1 -- 172.21.15.170:0/3458951821 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f09cc00a4d0 con 0x560675585170 2024-03-28T16:50:16.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560675679990 0x56067567bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56067567c2c0 con 0x560675679990 2024-03-28T16:50:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d1574640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560675679990 0x56067567bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d1574640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560675679990 0x56067567bd80 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d1574640 1 -- 172.21.15.170:0/3458951821 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f09c40b0020 con 0x560675679990 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560675678770 con 0x560675585170 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d1d75640 1 -- 172.21.15.170:0/3458951821 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f09cc00a4d0 con 0x560675585170 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56067567d630 0x56067567fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560675680030 con 0x56067567d630 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d0d73640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56067567d630 0x56067567fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.052+0000 7f09d0d73640 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56067567d630 0x56067567fa20 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d0d73640 1 -- 172.21.15.170:0/3458951821 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f09c800c040 con 0x56067567d630 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d3a59980 1 Processor -- start 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d3a59980 1 -- start start 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d3a59980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067568df60 0x56067568e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d3a59980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560675426e30 con 0x56067568df60 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067568df60 0x56067568e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067568df60 0x56067568e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55336/0 (socket says 172.21.15.170:55336) 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d1574640 1 -- 172.21.15.170:0/255463199 learned_addr learned my addr 172.21.15.170:0/255463199 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d1574640 1 -- 172.21.15.170:0/255463199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56067568e8c0 con 0x56067568df60 2024-03-28T16:50:16.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d1574640 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067568df60 0x56067568e380 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f09c40aa5a0 tx=0x7f09c40c5e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f08b6ffd640 1 -- 172.21.15.170:0/255463199 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40acc90 con 0x56067568df60 2024-03-28T16:50:16.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56067568eb10 con 0x56067568df60 2024-03-28T16:50:16.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f08b6ffd640 1 -- 172.21.15.170:0/255463199 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40acdf0 con 0x56067568df60 2024-03-28T16:50:16.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f08b6ffd640 1 -- 172.21.15.170:0/255463199 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40b3520 con 0x56067568df60 2024-03-28T16:50:16.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56067568f660 con 0x56067568df60 2024-03-28T16:50:16.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f08b6ffd640 1 -- 172.21.15.170:0/255463199 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f09c40b3680 con 0x56067568df60 2024-03-28T16:50:16.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f08b6ffd640 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08a80342b0 0x7f08a8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.056+0000 7f08b6ffd640 1 -- 172.21.15.170:0/255463199 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f09c40edbf0 con 0x56067568df60 2024-03-28T16:50:16.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d0d73640 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08a80342b0 0x7f08a8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 Processor -- start 2024-03-28T16:50:16.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- start start 2024-03-28T16:50:16.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x56067569b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067541a790 con 0x56067569af60 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d0d73640 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08a80342b0 0x7f08a8036770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09c800bfe0 tx=0x7f09c8006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x56067569b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x56067569b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55344/0 (socket says 172.21.15.170:55344) 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 -- 172.21.15.170:0/3269657517 learned_addr learned my addr 172.21.15.170:0/3269657517 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 -- 172.21.15.170:0/3269657517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56067569b930 con 0x56067569af60 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 --2- 172.21.15.170:0/3269657517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x56067569b360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cf6c0 tx=0x7f09c40e2980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34423fb301e9ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f08b4ff9640 1 -- 172.21.15.170:0/3269657517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40fc070 con 0x56067569af60 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f08b4ff9640 1 -- 172.21.15.170:0/3269657517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40b5dc0 con 0x56067569af60 2024-03-28T16:50:16.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f08b4ff9640 1 -- 172.21.15.170:0/3269657517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40b1440 con 0x56067569af60 2024-03-28T16:50:16.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- 172.21.15.170:0/3269657517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 msgr2=0x56067569b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3269657517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x56067569b360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cf6c0 tx=0x7f09c40e2980 comp rx=0 tx=0).stop 2024-03-28T16:50:16.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- 172.21.15.170:0/3269657517 shutdown_connections 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3269657517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x56067569b360 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- 172.21.15.170:0/3269657517 >> 172.21.15.170:0/3269657517 conn(0x5606756972a0 msgr2=0x560675695300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- 172.21.15.170:0/3269657517 shutdown_connections 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- 172.21.15.170:0/3269657517 wait complete. 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 Processor -- start 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- start start 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x5606756a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d3a59980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606756a22f0 con 0x56067569af60 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x5606756a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x5606756a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55354/0 (socket says 172.21.15.170:55354) 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 learned_addr learned my addr 172.21.15.170:0/1144465911 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.060+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56067568f660 con 0x56067569af60 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d1574640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x5606756a1db0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cf6c0 tx=0x7f09c40fc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f08a6ffd640 1 -- 172.21.15.170:0/1144465911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c4108040 con 0x56067569af60 2024-03-28T16:50:16.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f08a6ffd640 1 -- 172.21.15.170:0/1144465911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c40e4840 con 0x56067569af60 2024-03-28T16:50:16.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56067568f660 con 0x56067569af60 2024-03-28T16:50:16.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f08a6ffd640 1 -- 172.21.15.170:0/1144465911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09c40ac3b0 con 0x56067569af60 2024-03-28T16:50:16.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56067568f660 con 0x56067569af60 2024-03-28T16:50:16.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f08a6ffd640 1 -- 172.21.15.170:0/1144465911 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f09c40ac5d0 con 0x56067569af60 2024-03-28T16:50:16.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f08a6ffd640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f089c0342b0 0x7f089c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f08a6ffd640 1 -- 172.21.15.170:0/1144465911 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f09c410ebe0 con 0x56067569af60 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d0d73640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f089c0342b0 0x7f089c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606756ad020 0x5606756af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756afc30 con 0x5606756ad020 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d1d75640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606756ad020 0x5606756af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d0d73640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f089c0342b0 0x7f089c036770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x560675678ad0 tx=0x7f09c8023040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d1d75640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606756ad020 0x5606756af4e0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 realm 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0000 con 0x5606756ad020 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.069 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0bb0 con 0x5606756ad020 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606756b1560 0x5606756b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.064+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b3ed0 con 0x5606756b1560 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d1574640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606756b1560 0x5606756b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d1574640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606756b1560 0x5606756b3950 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d3a59980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:16.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d3a59980 20 searching for the correct realm 2024-03-28T16:50:16.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606756b5dd0 0x5606756b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b88b0 con 0x5606756b5dd0 2024-03-28T16:50:16.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d0d73640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606756b5dd0 0x5606756b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d0d73640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606756b5dd0 0x5606756b81e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d0d73640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f09c801f430 0x7f09c801f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b88b0 con 0x7f09c801f430 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d1d75640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f09c801f430 0x7f09c801f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d1d75640 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f09c801f430 0x7f09c801f850 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.068+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x7f09c801f430 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.072+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x7f09c801f430 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x7f09c801f430 2024-03-28T16:50:16.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.076+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x7f09c801f430 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b5dd0 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756b1560 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x5606756ad020 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.080+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b0850 con 0x7f09c801f430 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b4740 con 0x5606756ad020 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b4740 con 0x5606756ad020 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b4740 con 0x7f09c801f430 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 zone default found 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 4 Realm: () 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 zonegroup default 2024-03-28T16:50:16.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b4f90 con 0x5606756b1560 2024-03-28T16:50:16.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b5390 con 0x5606756b5dd0 2024-03-28T16:50:16.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756ba820 con 0x5606756b5dd0 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 started sync module instance, tier type = 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.084+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b9fb0 con 0x7f09c801f430 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.088+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.088+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94585739778832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756ba360 con 0x7f09c801f430 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.088+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bb580 con 0x5606756ad020 2024-03-28T16:50:16.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.088+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94585739778832] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.088+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.088+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94585739783232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bb5a0 con 0x5606756ad020 2024-03-28T16:50:16.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.088+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bc070 con 0x5606756b5dd0 2024-03-28T16:50:16.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94585739787088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bc8c0 con 0x5606756b5dd0 2024-03-28T16:50:16.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bd270 con 0x7f09c801f430 2024-03-28T16:50:16.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94585739783232] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94585739784288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bd620 con 0x7f09c801f430 2024-03-28T16:50:16.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bdf90 con 0x5606756ad020 2024-03-28T16:50:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94585739784288] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.092+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94585739787088] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.096+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.096+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94585739789344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bdaf0 con 0x5606756ad020 2024-03-28T16:50:16.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.096+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756be2f0 con 0x5606756b1560 2024-03-28T16:50:16.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.096+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94585739789344] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.096+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.096+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94585739795184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bdf10 con 0x5606756b1560 2024-03-28T16:50:16.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.096+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bef90 con 0x5606756b1560 2024-03-28T16:50:16.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94585739801328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bfbb0 con 0x5606756b1560 2024-03-28T16:50:16.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756c05c0 con 0x5606756b1560 2024-03-28T16:50:16.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94585739795184] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94585739801328] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94585739798096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bf750 con 0x5606756b1560 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 20 add_watcher() i=0 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 20 add_watcher() i=1 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 20 add_watcher() i=2 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 20 add_watcher() i=3 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 20 add_watcher() i=4 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 20 add_watcher() i=5 2024-03-28T16:50:16.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.100+0000 7f09d3a59980 20 add_watcher() i=6 2024-03-28T16:50:16.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94585739798096] v31'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 20 add_watcher() i=7 2024-03-28T16:50:16.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:16.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756c36c0 con 0x5606756ad020 2024-03-28T16:50:16.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94585739804896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756c0b30 con 0x5606756ad020 2024-03-28T16:50:16.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94585739804896] v31'200 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f08877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:16.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:16.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 5 note: GC not initialized 2024-03-28T16:50:16.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f08867e4640 20 reqs_thread_entry: start 2024-03-28T16:50:16.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:16.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 20 Filter name: none 2024-03-28T16:50:16.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f07f7fff640 20 reqs_thread_entry: start 2024-03-28T16:50:16.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.104+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756e5030 con 0x5606756ad020 2024-03-28T16:50:16.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f09d3a59980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:16.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756e5030 con 0x5606756b1560 2024-03-28T16:50:16.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f08877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94585739804896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756e4ff0 con 0x5606756ad020 2024-03-28T16:50:16.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94585739804896] v31'201 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.108+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94585739778832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756e4ff0 con 0x7f09c801f430 2024-03-28T16:50:16.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.112+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94585739778832] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.112+0000 7f09d3a59980 20 remove_watcher() i=0 2024-03-28T16:50:16.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.112+0000 7f09d3a59980 2 removed watcher, disabling cache 2024-03-28T16:50:16.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.112+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94585739783232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756e4ff0 con 0x5606756ad020 2024-03-28T16:50:16.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.112+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94585739783232] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.112+0000 7f09d3a59980 20 remove_watcher() i=1 2024-03-28T16:50:16.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.112+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94585739787088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756e4ff0 con 0x5606756b5dd0 2024-03-28T16:50:16.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.116+0000 7f09d0d73640 1 -- 172.21.15.170:0/1144465911 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94585739787088] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c8033040 con 0x5606756b5dd0 2024-03-28T16:50:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.116+0000 7f09d3a59980 20 remove_watcher() i=2 2024-03-28T16:50:16.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.116+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94585739784288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b9b10 con 0x7f09c801f430 2024-03-28T16:50:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.116+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94585739784288] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x7f09c801f430 2024-03-28T16:50:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.116+0000 7f09d3a59980 20 remove_watcher() i=3 2024-03-28T16:50:16.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.116+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94585739789344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b9b10 con 0x5606756ad020 2024-03-28T16:50:16.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.120+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94585739789344] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09cc00a4d0 con 0x5606756ad020 2024-03-28T16:50:16.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.120+0000 7f09d3a59980 20 remove_watcher() i=4 2024-03-28T16:50:16.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.120+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94585739795184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b9b10 con 0x5606756b1560 2024-03-28T16:50:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.120+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94585739795184] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.120+0000 7f09d3a59980 20 remove_watcher() i=5 2024-03-28T16:50:16.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.120+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94585739801328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756bdaf0 con 0x5606756b1560 2024-03-28T16:50:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94585739801328] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 20 remove_watcher() i=6 2024-03-28T16:50:16.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94585739798096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5606756b9b10 con 0x5606756b1560 2024-03-28T16:50:16.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d1574640 1 -- 172.21.15.170:0/1144465911 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94585739798096] v31'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f09c411e380 con 0x5606756b1560 2024-03-28T16:50:16.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 20 remove_watcher() i=7 2024-03-28T16:50:16.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606756b1560 msgr2=0x5606756b3950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5606756b1560 0x5606756b3950 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606756ad020 msgr2=0x5606756af4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5606756ad020 0x5606756af4e0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f09c801f430 msgr2=0x7f09c801f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f09c801f430 0x7f09c801f850 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606756b5dd0 msgr2=0x5606756b81e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5606756b5dd0 0x5606756b81e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f089c0342b0 msgr2=0x7f089c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.124+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f089c0342b0 0x7f089c036770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x560675678ad0 tx=0x7f09c8023040 comp rx=0 tx=0).stop 2024-03-28T16:50:16.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 msgr2=0x5606756a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/1144465911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067569af60 0x5606756a1db0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cf6c0 tx=0x7f09c40fc000 comp rx=0 tx=0).stop 2024-03-28T16:50:16.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d1d75640 1 -- 172.21.15.170:0/1144465911 reap_dead start 2024-03-28T16:50:16.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 shutdown_connections 2024-03-28T16:50:16.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 >> 172.21.15.170:0/1144465911 conn(0x5606756972a0 msgr2=0x560675694900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 shutdown_connections 2024-03-28T16:50:16.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/1144465911 wait complete. 2024-03-28T16:50:16.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08a80342b0 msgr2=0x7f08a8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08a80342b0 0x7f08a8036770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09c800bfe0 tx=0x7f09c8006000 comp rx=0 tx=0).stop 2024-03-28T16:50:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067568df60 msgr2=0x56067568e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067568df60 0x56067568e380 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f09c40aa5a0 tx=0x7f09c40c5e20 comp rx=0 tx=0).stop 2024-03-28T16:50:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 shutdown_connections 2024-03-28T16:50:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08a80342b0 0x7f08a8036770 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/255463199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56067568df60 0x56067568e380 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 >> 172.21.15.170:0/255463199 conn(0x560675682d70 msgr2=0x560675683880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 shutdown_connections 2024-03-28T16:50:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/255463199 wait complete. 2024-03-28T16:50:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560675679990 msgr2=0x56067567bd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560675679990 0x56067567bd80 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560675585170 msgr2=0x5606756773d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560675585170 0x5606756773d0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56067567d630 msgr2=0x56067567fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56067567d630 0x56067567fa20 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08b80342b0 msgr2=0x7f08b8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f08b80342b0 0x7f08b8036770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c10 tx=0x7f09c80013c0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 msgr2=0x5606756707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 --2- 172.21.15.170:0/3458951821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5606754f68b0 0x5606756707a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005940 tx=0x7f09c40180e0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d1d75640 1 -- 172.21.15.170:0/3458951821 reap_dead start 2024-03-28T16:50:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 shutdown_connections 2024-03-28T16:50:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 >> 172.21.15.170:0/3458951821 conn(0x56067557c710 msgr2=0x5606756640f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 shutdown_connections 2024-03-28T16:50:16.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.128+0000 7f09d3a59980 1 -- 172.21.15.170:0/3458951821 wait complete. 2024-03-28T16:50:16.140 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}]},{"bucket":"myfoo2","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34}}]} 2024-03-28T16:50:16.141 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-03-28T16:50:16.141 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-03-28T16:50:16.141 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-03-28T16:50:16.142 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-03-28T16:50:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:16.217 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:16.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.216+0000 7fc65d474980 1 Processor -- start 2024-03-28T16:50:16.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.216+0000 7fc65d474980 1 -- start start 2024-03-28T16:50:16.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.216+0000 7fc65d474980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5635e92c7720 0x5635e92c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.216+0000 7fc65d474980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5635e920d4c0 con 0x5635e9235080 2024-03-28T16:50:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.216+0000 7fc65d474980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5635e91d40f0 con 0x5635e92c7720 2024-03-28T16:50:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc65a78e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5635e92c7720 0x5635e92c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc65a78e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5635e92c7720 0x5635e92c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55368/0 (socket says 172.21.15.170:55368) 2024-03-28T16:50:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc65a78e640 1 -- 172.21.15.170:0/1855088184 learned_addr learned my addr 172.21.15.170:0/1855088184 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc65a78e640 1 -- 172.21.15.170:0/1855088184 >> v1:172.21.15.119:6789/0 conn(0x5635e9235080 legacy=0x5635e92c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc65a78e640 1 -- 172.21.15.170:0/1855088184 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e92cb060 con 0x5635e92c7720 2024-03-28T16:50:16.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc65a78e640 1 --2- 172.21.15.170:0/1855088184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e92c7720 0x5635e92c9b10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc64400a5c0 tx=0x7fc64401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e983e1a1ff4657 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc659f8d640 1 -- 172.21.15.170:0/1855088184 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6440aa470 con 0x5635e92c7720 2024-03-28T16:50:16.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc659f8d640 1 -- 172.21.15.170:0/1855088184 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440aa5d0 con 0x5635e92c7720 2024-03-28T16:50:16.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.220+0000 7fc659f8d640 1 -- 172.21.15.170:0/1855088184 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6440aa890 con 0x5635e92c7720 2024-03-28T16:50:16.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.224+0000 7fc65d474980 1 -- 172.21.15.170:0/1855088184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e92c7720 msgr2=0x5635e92c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.224+0000 7fc65d474980 1 --2- 172.21.15.170:0/1855088184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e92c7720 0x5635e92c9b10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc64400a5c0 tx=0x7fc64401b340 comp rx=0 tx=0).stop 2024-03-28T16:50:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.224+0000 7fc65d474980 1 -- 172.21.15.170:0/1855088184 shutdown_connections 2024-03-28T16:50:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.224+0000 7fc65d474980 1 --2- 172.21.15.170:0/1855088184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e92c7720 0x5635e92c9b10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.224+0000 7fc65d474980 1 -- 172.21.15.170:0/1855088184 >> 172.21.15.170:0/1855088184 conn(0x5635e92bb6e0 msgr2=0x5635e92bdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.224+0000 7fc65d474980 1 -- 172.21.15.170:0/1855088184 shutdown_connections 2024-03-28T16:50:16.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.224+0000 7fc65d474980 1 -- 172.21.15.170:0/1855088184 wait complete. 2024-03-28T16:50:16.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.236+0000 7fc65d474980 1 Processor -- start 2024-03-28T16:50:16.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.236+0000 7fc65d474980 1 -- start start 2024-03-28T16:50:16.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.236+0000 7fc65d474980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.236+0000 7fc65d474980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e91d1620 con 0x5635e9235080 2024-03-28T16:50:16.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55380/0 (socket says 172.21.15.170:55380) 2024-03-28T16:50:16.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 -- 172.21.15.170:0/3039689797 learned_addr learned my addr 172.21.15.170:0/3039689797 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 -- 172.21.15.170:0/3039689797 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644005df0 con 0x5635e9235080 2024-03-28T16:50:16.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 --2- 172.21.15.170:0/3039689797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93ad1d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc64c006fb0 tx=0x7fc64c001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab44ee45c7e082f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc548e09640 1 -- 172.21.15.170:0/3039689797 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c00f640 con 0x5635e9235080 2024-03-28T16:50:16.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc548e09640 1 -- 172.21.15.170:0/3039689797 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c00f7c0 con 0x5635e9235080 2024-03-28T16:50:16.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc548e09640 1 -- 172.21.15.170:0/3039689797 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c00fae0 con 0x5635e9235080 2024-03-28T16:50:16.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 -- 172.21.15.170:0/3039689797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 msgr2=0x5635e93ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 --2- 172.21.15.170:0/3039689797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93ad1d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc64c006fb0 tx=0x7fc64c001940 comp rx=0 tx=0).stop 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 -- 172.21.15.170:0/3039689797 shutdown_connections 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 --2- 172.21.15.170:0/3039689797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93ad1d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 -- 172.21.15.170:0/3039689797 >> 172.21.15.170:0/3039689797 conn(0x5635e92bb6e0 msgr2=0x5635e93a8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 -- 172.21.15.170:0/3039689797 shutdown_connections 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 -- 172.21.15.170:0/3039689797 wait complete. 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 Processor -- start 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 -- start start 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65d474980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e91cce00 con 0x5635e9235080 2024-03-28T16:50:16.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55392/0 (socket says 172.21.15.170:55392) 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 -- 172.21.15.170:0/313324079 learned_addr learned my addr 172.21.15.170:0/313324079 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.240+0000 7fc65af8f640 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e920d4c0 con 0x5635e9235080 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65af8f640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93b4270 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x5635e93adf80 tx=0x7fc64c000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc547606640 1 -- 172.21.15.170:0/313324079 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c005450 con 0x5635e9235080 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc547606640 1 -- 172.21.15.170:0/313324079 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c0055b0 con 0x5635e9235080 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5635e93b47b0 con 0x5635e9235080 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc547606640 1 -- 172.21.15.170:0/313324079 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c0058b0 con 0x5635e9235080 2024-03-28T16:50:16.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e93b4be0 con 0x5635e9235080 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc547606640 1 -- 172.21.15.170:0/313324079 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc64c005b20 con 0x5635e9235080 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc547606640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5400342b0 0x7fc540036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc547606640 1 -- 172.21.15.170:0/313324079 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc64c049220 con 0x5635e9235080 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65a78e640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5400342b0 0x7fc540036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e92c7720 0x5635e93baea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93bb3e0 con 0x5635e92c7720 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65b790640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e92c7720 0x5635e93baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65a78e640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5400342b0 0x7fc540036770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ae000 tx=0x7fc6440a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65b790640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e92c7720 0x5635e93baea0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65b790640 1 -- 172.21.15.170:0/313324079 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e92c7720 2024-03-28T16:50:16.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93bb740 con 0x5635e92c7720 2024-03-28T16:50:16.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65b790640 1 -- 172.21.15.170:0/313324079 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e92c7720 2024-03-28T16:50:16.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93bd460 0x5635e93bf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93bfd90 con 0x5635e93bd460 2024-03-28T16:50:16.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.244+0000 7fc65af8f640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93bd460 0x5635e93bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65af8f640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93bd460 0x5635e93bf850 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65af8f640 1 -- 172.21.15.170:0/313324079 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fc64c023070 con 0x5635e93bd460 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93bc2a0 con 0x5635e92c7720 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65b790640 1 -- 172.21.15.170:0/313324079 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e92c7720 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635e93c1120 0x5635e93c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93c3b60 con 0x5635e93c1120 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65a78e640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635e93c1120 0x5635e93c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65a78e640 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635e93c1120 0x5635e93c3510 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65a78e640 1 -- 172.21.15.170:0/313324079 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93c1120 2024-03-28T16:50:16.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65d474980 1 Processor -- start 2024-03-28T16:50:16.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65d474980 1 -- start start 2024-03-28T16:50:16.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65d474980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93d19f0 0x5635e93d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65d474980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9165e30 con 0x5635e93d19f0 2024-03-28T16:50:16.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93d19f0 0x5635e93d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93d19f0 0x5635e93d1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55406/0 (socket says 172.21.15.170:55406) 2024-03-28T16:50:16.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.248+0000 7fc65af8f640 1 -- 172.21.15.170:0/2379849588 learned_addr learned my addr 172.21.15.170:0/2379849588 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65af8f640 1 -- 172.21.15.170:0/2379849588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e93d2350 con 0x5635e93d19f0 2024-03-28T16:50:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65af8f640 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93d19f0 0x5635e93d1e10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00afd0 tx=0x7fc64c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc544e01640 1 -- 172.21.15.170:0/2379849588 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c01dcc0 con 0x5635e93d19f0 2024-03-28T16:50:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5635e93d25a0 con 0x5635e93d19f0 2024-03-28T16:50:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc544e01640 1 -- 172.21.15.170:0/2379849588 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c01de20 con 0x5635e93d19f0 2024-03-28T16:50:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc544e01640 1 -- 172.21.15.170:0/2379849588 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c047b40 con 0x5635e93d19f0 2024-03-28T16:50:16.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e93d30f0 con 0x5635e93d19f0 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc544e01640 1 -- 172.21.15.170:0/2379849588 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc64c047ca0 con 0x5635e93d19f0 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc544e01640 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5340342b0 0x7fc534036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc544e01640 1 -- 172.21.15.170:0/2379849588 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc64c04acb0 con 0x5635e93d19f0 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65a78e640 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5340342b0 0x7fc534036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65d474980 1 Processor -- start 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65d474980 1 -- start start 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65a78e640 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5340342b0 0x7fc534036770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5635e93bc600 tx=0x7fc644005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65d474980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93de8f0 0x5635e93decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65d474980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e9159790 con 0x5635e93de8f0 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93de8f0 0x5635e93decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93de8f0 0x5635e93decf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55422/0 (socket says 172.21.15.170:55422) 2024-03-28T16:50:16.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65af8f640 1 -- 172.21.15.170:0/440763369 learned_addr learned my addr 172.21.15.170:0/440763369 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65af8f640 1 -- 172.21.15.170:0/440763369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e93df2c0 con 0x5635e93de8f0 2024-03-28T16:50:16.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc65af8f640 1 --2- 172.21.15.170:0/440763369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93de8f0 0x5635e93decf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc64c01dfb0 tx=0x7fc64c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa64a269e037455 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.252+0000 7fc53e7fc640 1 -- 172.21.15.170:0/440763369 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c041380 con 0x5635e93de8f0 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc53e7fc640 1 -- 172.21.15.170:0/440763369 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c026400 con 0x5635e93de8f0 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc53e7fc640 1 -- 172.21.15.170:0/440763369 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c041b10 con 0x5635e93de8f0 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- 172.21.15.170:0/440763369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93de8f0 msgr2=0x5635e93decf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 --2- 172.21.15.170:0/440763369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93de8f0 0x5635e93decf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc64c01dfb0 tx=0x7fc64c041000 comp rx=0 tx=0).stop 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- 172.21.15.170:0/440763369 shutdown_connections 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 --2- 172.21.15.170:0/440763369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93de8f0 0x5635e93decf0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- 172.21.15.170:0/440763369 >> 172.21.15.170:0/440763369 conn(0x5635e93dac30 msgr2=0x5635e93d8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- 172.21.15.170:0/440763369 shutdown_connections 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- 172.21.15.170:0/440763369 wait complete. 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 Processor -- start 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- start start 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93e5930 0x5635e93e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e93e6290 con 0x5635e93e5930 2024-03-28T16:50:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93e5930 0x5635e93e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65af8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93e5930 0x5635e93e5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55438/0 (socket says 172.21.15.170:55438) 2024-03-28T16:50:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 learned_addr learned my addr 172.21.15.170:0/2005325986 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e93d30f0 con 0x5635e93e5930 2024-03-28T16:50:16.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65af8f640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93e5930 0x5635e93e5d50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc64c055040 tx=0x7fc64c01eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc53cff9640 1 -- 172.21.15.170:0/2005325986 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c026570 con 0x5635e93e5930 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5635e93e6490 con 0x5635e93e5930 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc53cff9640 1 -- 172.21.15.170:0/2005325986 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c062070 con 0x5635e93e5930 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc53cff9640 1 -- 172.21.15.170:0/2005325986 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64c039c20 con 0x5635e93e5930 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e93e9000 con 0x5635e93e5930 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.256+0000 7fc53cff9640 1 -- 172.21.15.170:0/2005325986 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc64c064030 con 0x5635e93e5930 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc53cff9640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5240342b0 0x7fc524036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc53cff9640 1 -- 172.21.15.170:0/2005325986 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc64c068be0 con 0x5635e93e5930 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65a78e640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5240342b0 0x7fc524036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e93f1180 0x5635e93f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f3d90 con 0x5635e93f1180 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65b790640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e93f1180 0x5635e93f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65a78e640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5240342b0 0x7fc524036770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc64401b380 tx=0x7fc6440c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65b790640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e93f1180 0x5635e93f3640 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 realm 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4160 con 0x5635e93f1180 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:16.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f49b0 con 0x5635e93f1180 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93f5a20 0x5635e93f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f8390 con 0x5635e93f5a20 2024-03-28T16:50:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65af8f640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93f5a20 0x5635e93f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.260+0000 7fc65af8f640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93f5a20 0x5635e93f7e10 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65d474980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65d474980 20 searching for the correct realm 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5635e93fa290 0x5635e93fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93fcdb0 con 0x5635e93fa290 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65a78e640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5635e93fa290 0x5635e93fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65a78e640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5635e93fa290 0x5635e93fc6e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65a78e640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc6440e0040 0x7fc6440d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93fcdb0 con 0x7fc6440e0040 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc6440e0040 0x7fc6440d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc6440e0040 0x7fc6440d3400 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.264+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x7fc6440e0040 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x7fc6440e0040 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.268+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.273 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x7fc6440e0040 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.272+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x7fc6440e0040 2024-03-28T16:50:16.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93fa290 2024-03-28T16:50:16.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f5a20 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x5635e93f1180 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f4d10 con 0x7fc6440e0040 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:16.280 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f8c00 con 0x5635e93f1180 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f8c00 con 0x5635e93f1180 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f8c00 con 0x7fc6440e0040 2024-03-28T16:50:16.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 zone default found 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 4 Realm: () 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 zonegroup default 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.276+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f9450 con 0x5635e93f5a20 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93f9850 con 0x5635e93fa290 2024-03-28T16:50:16.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93febf0 con 0x5635e93fa290 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 20 started sync module instance, tier type = 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:16.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93fe4b0 con 0x7fc6440e0040 2024-03-28T16:50:16.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94789546532880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93fe860 con 0x7fc6440e0040 2024-03-28T16:50:16.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93ff950 con 0x5635e93f1180 2024-03-28T16:50:16.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.280+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94789546532880] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.284+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.284+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94789546536976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93ff930 con 0x5635e93f1180 2024-03-28T16:50:16.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.284+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e94003b0 con 0x5635e93fa290 2024-03-28T16:50:16.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.284+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94789546536976] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.284+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.284+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94789546540720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9400790 con 0x5635e93fa290 2024-03-28T16:50:16.286 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.284+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9401100 con 0x7fc6440e0040 2024-03-28T16:50:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94789546544496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e94017f0 con 0x7fc6440e0040 2024-03-28T16:50:16.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e94022c0 con 0x5635e93f1180 2024-03-28T16:50:16.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94789546540720] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94789546544496] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94789546548640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9402280 con 0x5635e93f1180 2024-03-28T16:50:16.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.288+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e94013c0 con 0x5635e93f5a20 2024-03-28T16:50:16.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94789546548640] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94789546538032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9402b10 con 0x5635e93f5a20 2024-03-28T16:50:16.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9403370 con 0x5635e93f5a20 2024-03-28T16:50:16.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94789546555216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9404010 con 0x5635e93f5a20 2024-03-28T16:50:16.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e94049e0 con 0x5635e93f5a20 2024-03-28T16:50:16.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.292+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94789546538032] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94789546555216] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94789546556272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e94026b0 con 0x5635e93f5a20 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=0 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=1 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=2 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=3 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=4 2024-03-28T16:50:16.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=5 2024-03-28T16:50:16.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=6 2024-03-28T16:50:16.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94789546556272] v31'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 20 add_watcher() i=7 2024-03-28T16:50:16.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:16.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9407c80 con 0x5635e93f1180 2024-03-28T16:50:16.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.296+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94789546558688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9405190 con 0x5635e93f1180 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94789546558688] v31'202 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc5117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 5 note: GC not initialized 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc48bfff640 20 reqs_thread_entry: start 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:16.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 20 Filter name: none 2024-03-28T16:50:16.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc489ffb640 20 reqs_thread_entry: start 2024-03-28T16:50:16.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9429700 con 0x5635e93f1180 2024-03-28T16:50:16.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:16.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9429700 con 0x5635e93f5a20 2024-03-28T16:50:16.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.300+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc5117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94789546558688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9429700 con 0x5635e93f1180 2024-03-28T16:50:16.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94789546558688] v31'203 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.305 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94789546532880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9429700 con 0x7fc6440e0040 2024-03-28T16:50:16.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94789546532880] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc65d474980 20 remove_watcher() i=0 2024-03-28T16:50:16.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc65d474980 2 removed watcher, disabling cache 2024-03-28T16:50:16.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.304+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94789546536976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9429700 con 0x5635e93f1180 2024-03-28T16:50:16.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.308+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94789546536976] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.308+0000 7fc65d474980 20 remove_watcher() i=1 2024-03-28T16:50:16.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.308+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94789546540720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9429700 con 0x5635e93fa290 2024-03-28T16:50:16.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65a78e640 1 -- 172.21.15.170:0/2005325986 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94789546540720] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6440b0020 con 0x5635e93fa290 2024-03-28T16:50:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65d474980 20 remove_watcher() i=2 2024-03-28T16:50:16.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94789546544496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93fe010 con 0x7fc6440e0040 2024-03-28T16:50:16.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94789546544496] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x7fc6440e0040 2024-03-28T16:50:16.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65d474980 20 remove_watcher() i=3 2024-03-28T16:50:16.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94789546548640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93ff010 con 0x5635e93f1180 2024-03-28T16:50:16.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94789546548640] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc6540083f0 con 0x5635e93f1180 2024-03-28T16:50:16.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65d474980 20 remove_watcher() i=4 2024-03-28T16:50:16.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.312+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94789546538032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93ffeb0 con 0x5635e93f5a20 2024-03-28T16:50:16.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.316+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94789546538032] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.316+0000 7fc65d474980 20 remove_watcher() i=5 2024-03-28T16:50:16.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.316+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94789546555216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e9400d70 con 0x5635e93f5a20 2024-03-28T16:50:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94789546555216] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 20 remove_watcher() i=6 2024-03-28T16:50:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94789546556272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5635e93fe010 con 0x5635e93f5a20 2024-03-28T16:50:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65af8f640 1 -- 172.21.15.170:0/2005325986 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94789546556272] v31'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc64c059020 con 0x5635e93f5a20 2024-03-28T16:50:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 20 remove_watcher() i=7 2024-03-28T16:50:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93f5a20 msgr2=0x5635e93f7e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93f5a20 0x5635e93f7e10 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e93f1180 msgr2=0x5635e93f3640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e93f1180 0x5635e93f3640 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc6440e0040 msgr2=0x7fc6440d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc6440e0040 0x7fc6440d3400 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5635e93fa290 msgr2=0x5635e93fc6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5635e93fa290 0x5635e93fc6e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5240342b0 msgr2=0x7fc524036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5240342b0 0x7fc524036770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc64401b380 tx=0x7fc6440c1040 comp rx=0 tx=0).stop 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93e5930 msgr2=0x5635e93e5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65b790640 1 -- 172.21.15.170:0/2005325986 reap_dead start 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93e5930 0x5635e93e5d50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc64c055040 tx=0x7fc64c01eed0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 shutdown_connections 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2005325986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93e5930 0x5635e93e5d50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 >> 172.21.15.170:0/2005325986 conn(0x5635e93dac30 msgr2=0x5635e93d8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 shutdown_connections 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2005325986 wait complete. 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5340342b0 msgr2=0x7fc534036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5340342b0 0x7fc534036770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5635e93bc600 tx=0x7fc644005ad0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93d19f0 msgr2=0x5635e93d1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93d19f0 0x5635e93d1e10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00afd0 tx=0x7fc64c005f60 comp rx=0 tx=0).stop 2024-03-28T16:50:16.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 shutdown_connections 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5340342b0 0x7fc534036770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 --2- 172.21.15.170:0/2379849588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e93d19f0 0x5635e93d1e10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 >> 172.21.15.170:0/2379849588 conn(0x5635e93c6800 msgr2=0x5635e93c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 shutdown_connections 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.320+0000 7fc65d474980 1 -- 172.21.15.170:0/2379849588 wait complete. 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93bd460 msgr2=0x5635e93bf850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5635e93bd460 0x5635e93bf850 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e92c7720 msgr2=0x5635e93baea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5635e92c7720 0x5635e93baea0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635e93c1120 msgr2=0x5635e93c3510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5635e93c1120 0x5635e93c3510 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5400342b0 msgr2=0x7fc540036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fc5400342b0 0x7fc540036770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ae000 tx=0x7fc6440a6000 comp rx=0 tx=0).stop 2024-03-28T16:50:16.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 msgr2=0x5635e93b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 --2- 172.21.15.170:0/313324079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5635e9235080 0x5635e93b4270 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x5635e93adf80 tx=0x7fc64c000f50 comp rx=0 tx=0).stop 2024-03-28T16:50:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65b790640 1 -- 172.21.15.170:0/313324079 reap_dead start 2024-03-28T16:50:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 shutdown_connections 2024-03-28T16:50:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 >> 172.21.15.170:0/313324079 conn(0x5635e92bb6e0 msgr2=0x5635e93a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 shutdown_connections 2024-03-28T16:50:16.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.324+0000 7fc65d474980 1 -- 172.21.15.170:0/313324079 wait complete. 2024-03-28T16:50:16.335 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5}]},{"bucket":"myfoo2","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8}],"total":{"bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8}}]} 2024-03-28T16:50:16.335 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}]}, {'bucket': 'myfoo2', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}}]} 2024-03-28T16:50:16.335 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-03-28T16:50:16.335 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-03-28T16:50:16.336 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-03-28T16:50:16.417 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:16.417 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:16.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.416+0000 7f703636c980 1 Processor -- start 2024-03-28T16:50:16.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.416+0000 7f703636c980 1 -- start start 2024-03-28T16:50:16.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f703636c980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55bebacd8360 0x55bebacda750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f703636c980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bebac204c0 con 0x55bebac488c0 2024-03-28T16:50:16.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f703636c980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bebabe70f0 con 0x55bebacd8360 2024-03-28T16:50:16.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7033e87640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55bebac488c0 0x55bebacd6c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:46090/0 (socket says 172.21.15.170:46090) 2024-03-28T16:50:16.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7033e87640 1 -- 172.21.15.170:0/3497441322 learned_addr learned my addr 172.21.15.170:0/3497441322 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:16.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7033686640 1 --2- 172.21.15.170:0/3497441322 >> v2:172.21.15.119:3300/0 conn(0x55bebacd8360 0x55bebacda750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7032e85640 1 -- 172.21.15.170:0/3497441322 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270839605 0 0) 0x55bebac204c0 con 0x55bebac488c0 2024-03-28T16:50:16.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7032e85640 1 -- 172.21.15.170:0/3497441322 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f701c001e20 con 0x55bebac488c0 2024-03-28T16:50:16.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7033686640 1 -- 172.21.15.170:0/3497441322 >> v1:172.21.15.119:6789/0 conn(0x55bebac488c0 legacy=0x55bebacd6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7033686640 1 -- 172.21.15.170:0/3497441322 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bebacdbcd0 con 0x55bebacd8360 2024-03-28T16:50:16.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.420+0000 7f7033686640 1 --2- 172.21.15.170:0/3497441322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd8360 0x55bebacda750 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f702800a480 tx=0x7f702801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbf0def9d5ea733 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f7032e85640 1 -- 172.21.15.170:0/3497441322 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70280ab540 con 0x55bebacd8360 2024-03-28T16:50:16.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f7032e85640 1 -- 172.21.15.170:0/3497441322 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70280ab6a0 con 0x55bebacd8360 2024-03-28T16:50:16.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f7032e85640 1 -- 172.21.15.170:0/3497441322 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70280ab940 con 0x55bebacd8360 2024-03-28T16:50:16.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f703636c980 1 -- 172.21.15.170:0/3497441322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd8360 msgr2=0x55bebacda750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f703636c980 1 --2- 172.21.15.170:0/3497441322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd8360 0x55bebacda750 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f702800a480 tx=0x7f702801b280 comp rx=0 tx=0).stop 2024-03-28T16:50:16.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f703636c980 1 -- 172.21.15.170:0/3497441322 shutdown_connections 2024-03-28T16:50:16.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f703636c980 1 --2- 172.21.15.170:0/3497441322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd8360 0x55bebacda750 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f703636c980 1 -- 172.21.15.170:0/3497441322 >> 172.21.15.170:0/3497441322 conn(0x55bebacce720 msgr2=0x55bebacd0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f703636c980 1 -- 172.21.15.170:0/3497441322 shutdown_connections 2024-03-28T16:50:16.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.424+0000 7f703636c980 1 -- 172.21.15.170:0/3497441322 wait complete. 2024-03-28T16:50:16.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 Processor -- start 2024-03-28T16:50:16.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- start start 2024-03-28T16:50:16.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadbdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebabe4620 con 0x55bebac488c0 2024-03-28T16:50:16.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadbdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadbdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55454/0 (socket says 172.21.15.170:55454) 2024-03-28T16:50:16.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f7033e87640 1 -- 172.21.15.170:0/3605186307 learned_addr learned my addr 172.21.15.170:0/3605186307 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f7033e87640 1 -- 172.21.15.170:0/3605186307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028005d30 con 0x55bebac488c0 2024-03-28T16:50:16.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f7033e87640 1 --2- 172.21.15.170:0/3605186307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadbdcf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7024001c30 tx=0x7f702400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85af729f6c94e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f6f21e0b640 1 -- 172.21.15.170:0/3605186307 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f702400e690 con 0x55bebac488c0 2024-03-28T16:50:16.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f6f21e0b640 1 -- 172.21.15.170:0/3605186307 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702400e7f0 con 0x55bebac488c0 2024-03-28T16:50:16.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f6f21e0b640 1 -- 172.21.15.170:0/3605186307 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f702400eb50 con 0x55bebac488c0 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- 172.21.15.170:0/3605186307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 msgr2=0x55bebadbdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 --2- 172.21.15.170:0/3605186307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadbdcf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7024001c30 tx=0x7f702400b6d0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- 172.21.15.170:0/3605186307 shutdown_connections 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 --2- 172.21.15.170:0/3605186307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadbdcf0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- 172.21.15.170:0/3605186307 >> 172.21.15.170:0/3605186307 conn(0x55bebacce720 msgr2=0x55bebadb9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- 172.21.15.170:0/3605186307 shutdown_connections 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- 172.21.15.170:0/3605186307 wait complete. 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 Processor -- start 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.440+0000 7f703636c980 1 -- start start 2024-03-28T16:50:16.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f703636c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadc4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f703636c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebabdfe00 con 0x55bebac488c0 2024-03-28T16:50:16.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadc4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadc4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55466/0 (socket says 172.21.15.170:55466) 2024-03-28T16:50:16.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7033e87640 1 -- 172.21.15.170:0/3355757445 learned_addr learned my addr 172.21.15.170:0/3355757445 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7033e87640 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c001e20 con 0x55bebac488c0 2024-03-28T16:50:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7033e87640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadc4ba0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f702400b710 tx=0x7f7024000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f6f1bfff640 1 -- 172.21.15.170:0/3355757445 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f702401f070 con 0x55bebac488c0 2024-03-28T16:50:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bebadc50e0 con 0x55bebac488c0 2024-03-28T16:50:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f6f1bfff640 1 -- 172.21.15.170:0/3355757445 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702401a2e0 con 0x55bebac488c0 2024-03-28T16:50:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f6f1bfff640 1 -- 172.21.15.170:0/3355757445 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f702401a820 con 0x55bebac488c0 2024-03-28T16:50:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bebadc5510 con 0x55bebac488c0 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f6f1bfff640 1 -- 172.21.15.170:0/3355757445 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f702400a040 con 0x55bebac488c0 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f6f1bfff640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f1c0342b0 0x7f6f1c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f6f1bfff640 1 -- 172.21.15.170:0/3355757445 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f70240498b0 con 0x55bebac488c0 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7033686640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f1c0342b0 0x7f6f1c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebacd8360 0x55bebadcb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebadcbd10 con 0x55bebacd8360 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7034688640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebacd8360 0x55bebadcb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7033686640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f1c0342b0 0x7f6f1c036770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f70280ad000 tx=0x7f70280a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.444+0000 7f7034688640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebacd8360 0x55bebadcb7d0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7034688640 1 -- 172.21.15.170:0/3355757445 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebacd8360 2024-03-28T16:50:16.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebadcc070 con 0x55bebacd8360 2024-03-28T16:50:16.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7034688640 1 -- 172.21.15.170:0/3355757445 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebacd8360 2024-03-28T16:50:16.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebadcdd90 0x55bebadd0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebadd06c0 con 0x55bebadcdd90 2024-03-28T16:50:16.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7033e87640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebadcdd90 0x55bebadd0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7033e87640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebadcdd90 0x55bebadd0180 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7033e87640 1 -- 172.21.15.170:0/3355757445 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f7024014070 con 0x55bebadcdd90 2024-03-28T16:50:16.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebadccba0 con 0x55bebacd8360 2024-03-28T16:50:16.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7034688640 1 -- 172.21.15.170:0/3355757445 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebacd8360 2024-03-28T16:50:16.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55bebadd1a50 0x55bebadd3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebadd4490 con 0x55bebadd1a50 2024-03-28T16:50:16.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7033686640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55bebadd1a50 0x55bebadd3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.450 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7033686640 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55bebadd1a50 0x55bebadd3e40 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f7033686640 1 -- 172.21.15.170:0/3355757445 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f70280be040 con 0x55bebadd1a50 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.448+0000 7f703636c980 1 Processor -- start 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 -- start start 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd47a0 0x55bebacd2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebab78e30 con 0x55bebacd47a0 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd47a0 0x55bebacd2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd47a0 0x55bebacd2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55470/0 (socket says 172.21.15.170:55470) 2024-03-28T16:50:16.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f7033e87640 1 -- 172.21.15.170:0/4088754245 learned_addr learned my addr 172.21.15.170:0/4088754245 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f7033e87640 1 -- 172.21.15.170:0/4088754245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bebacd3180 con 0x55bebacd47a0 2024-03-28T16:50:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f7033e87640 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd47a0 0x55bebacd2c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7024047cf0 tx=0x7f7024017e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f6f197fa640 1 -- 172.21.15.170:0/4088754245 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7024041250 con 0x55bebacd47a0 2024-03-28T16:50:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bebacd33d0 con 0x55bebacd47a0 2024-03-28T16:50:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f6f197fa640 1 -- 172.21.15.170:0/4088754245 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70240413b0 con 0x55bebacd47a0 2024-03-28T16:50:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f6f197fa640 1 -- 172.21.15.170:0/4088754245 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7024041960 con 0x55bebacd47a0 2024-03-28T16:50:16.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bebacd3860 con 0x55bebacd47a0 2024-03-28T16:50:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f6f197fa640 1 -- 172.21.15.170:0/4088754245 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7024049110 con 0x55bebacd47a0 2024-03-28T16:50:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f6f197fa640 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f0c0342b0 0x7f6f0c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f6f197fa640 1 -- 172.21.15.170:0/4088754245 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f702403fba0 con 0x55bebacd47a0 2024-03-28T16:50:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f7033686640 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f0c0342b0 0x7f6f0c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 Processor -- start 2024-03-28T16:50:16.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 -- start start 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f703636c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebab6c790 con 0x55bebadf1490 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.452+0000 7f7033686640 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f0c0342b0 0x7f6f0c036770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55bebadc9220 tx=0x7f7028005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55484/0 (socket says 172.21.15.170:55484) 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 -- 172.21.15.170:0/1515028472 learned_addr learned my addr 172.21.15.170:0/1515028472 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 -- 172.21.15.170:0/1515028472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bebadf1e60 con 0x55bebadf1490 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 --2- 172.21.15.170:0/1515028472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf1890 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7024017fd0 tx=0x7f7024015a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4197c9c3a2532bb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f6f12ffd640 1 -- 172.21.15.170:0/1515028472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7024014ca0 con 0x55bebadf1490 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f6f12ffd640 1 -- 172.21.15.170:0/1515028472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7024024420 con 0x55bebadf1490 2024-03-28T16:50:16.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f6f12ffd640 1 -- 172.21.15.170:0/1515028472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70240493c0 con 0x55bebadf1490 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- 172.21.15.170:0/1515028472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 msgr2=0x55bebadf1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 --2- 172.21.15.170:0/1515028472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf1890 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7024017fd0 tx=0x7f7024015a00 comp rx=0 tx=0).stop 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- 172.21.15.170:0/1515028472 shutdown_connections 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 --2- 172.21.15.170:0/1515028472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf1890 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- 172.21.15.170:0/1515028472 >> 172.21.15.170:0/1515028472 conn(0x55bebaded820 msgr2=0x55bebadeb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- 172.21.15.170:0/1515028472 shutdown_connections 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- 172.21.15.170:0/1515028472 wait complete. 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 Processor -- start 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- start start 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bebadf8700 con 0x55bebadf1490 2024-03-28T16:50:16.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55498/0 (socket says 172.21.15.170:55498) 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 learned_addr learned my addr 172.21.15.170:0/877355170 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bebacd33d0 con 0x55bebadf1490 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f7033e87640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf81c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7024024df0 tx=0x7f702400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f6f117fa640 1 -- 172.21.15.170:0/877355170 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7024016c20 con 0x55bebadf1490 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f6f117fa640 1 -- 172.21.15.170:0/877355170 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7024016d80 con 0x55bebadf1490 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f6f117fa640 1 -- 172.21.15.170:0/877355170 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f702402e180 con 0x55bebadf1490 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.456+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bebacd33d0 con 0x55bebadf1490 2024-03-28T16:50:16.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bebacd33d0 con 0x55bebadf1490 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f6f117fa640 1 -- 172.21.15.170:0/877355170 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f702402e2e0 con 0x55bebadf1490 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f6f117fa640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6efc0342b0 0x7f6efc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f6f117fa640 1 -- 172.21.15.170:0/877355170 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f702406fbf0 con 0x55bebadf1490 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7033686640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6efc0342b0 0x7f6efc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebae03430 0x55bebae058f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06040 con 0x55bebae03430 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7034688640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebae03430 0x55bebae058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7033686640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6efc0342b0 0x7f6efc036770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70280c0010 tx=0x7f70280b7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7034688640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebae03430 0x55bebae058f0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 realm 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06410 con 0x55bebae03430 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.465 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06fc0 con 0x55bebae03430 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebae07970 0x55bebae09d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0a2e0 con 0x55bebae07970 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.460+0000 7f7033e87640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebae07970 0x55bebae09d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033e87640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebae07970 0x55bebae09d60 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f703636c980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f703636c980 20 searching for the correct realm 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55bebae0c170 0x55bebae0e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0ecd0 con 0x55bebae0c170 2024-03-28T16:50:16.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033686640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55bebae0c170 0x55bebae0e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033686640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55bebae0c170 0x55bebae0e600 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033686640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70280cc040 0x7f70280b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0ecd0 con 0x7f70280cc040 2024-03-28T16:50:16.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7034688640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70280cc040 0x7f70280b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7034688640 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70280cc040 0x7f70280b5410 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.469 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.464+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x7f70280cc040 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.468+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x7f70280cc040 2024-03-28T16:50:16.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x7f70280cc040 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.472+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x7f70280cc040 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae0c170 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae07970 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x55bebae03430 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.476+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae06c60 con 0x7f70280cc040 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0aae0 con 0x55bebae03430 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0aae0 con 0x55bebae03430 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0aae0 con 0x7f70280cc040 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 zone default found 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 4 Realm: () 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 zonegroup default 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0b330 con 0x55bebae07970 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0b710 con 0x55bebae0c170 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae10b40 con 0x55bebae0c170 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 started sync module instance, tier type = 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.480+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae103d0 con 0x7f70280cc040 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.484+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.484+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94277667454768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae10780 con 0x7f70280cc040 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.484+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae118a0 con 0x55bebae03430 2024-03-28T16:50:16.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.484+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94277667454768] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.484+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.484+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94277667458912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae118e0 con 0x55bebae03430 2024-03-28T16:50:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.484+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae12380 con 0x55bebae0c170 2024-03-28T16:50:16.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94277667458912] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94277667462752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae12730 con 0x55bebae0c170 2024-03-28T16:50:16.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae130a0 con 0x7f70280cc040 2024-03-28T16:50:16.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94277667466512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae13770 con 0x7f70280cc040 2024-03-28T16:50:16.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae14240 con 0x55bebae03430 2024-03-28T16:50:16.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.488+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94277667462752] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94277667466512] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94277667470624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae14200 con 0x55bebae03430 2024-03-28T16:50:16.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae145f0 con 0x55bebae07970 2024-03-28T16:50:16.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94277667470624] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94277667459968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae149a0 con 0x55bebae07970 2024-03-28T16:50:16.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.492+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae15310 con 0x55bebae07970 2024-03-28T16:50:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94277667477104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae15f30 con 0x55bebae07970 2024-03-28T16:50:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae168e0 con 0x55bebae07970 2024-03-28T16:50:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94277667459968] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94277667477104] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94277667478160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae16440 con 0x55bebae07970 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 20 add_watcher() i=0 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 20 add_watcher() i=1 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 20 add_watcher() i=2 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 20 add_watcher() i=3 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 20 add_watcher() i=4 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 20 add_watcher() i=5 2024-03-28T16:50:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.496+0000 7f703636c980 20 add_watcher() i=6 2024-03-28T16:50:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94277667478160] v31'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 20 add_watcher() i=7 2024-03-28T16:50:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae19aa0 con 0x55bebae03430 2024-03-28T16:50:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94277667467728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae16e50 con 0x55bebae03430 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94277667467728] v31'204 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f6eea7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 5 note: GC not initialized 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f6ee97e2640 20 reqs_thread_entry: start 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 20 Filter name: none 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f6ee27fc640 20 reqs_thread_entry: start 2024-03-28T16:50:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae3b590 con 0x55bebae03430 2024-03-28T16:50:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.500+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae3b590 con 0x55bebae07970 2024-03-28T16:50:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.504+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.504+0000 7f6eea7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.504+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94277667467728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae3b590 con 0x55bebae03430 2024-03-28T16:50:16.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.504+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94277667467728] v31'205 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.504+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94277667454768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae3b590 con 0x7f70280cc040 2024-03-28T16:50:16.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.508+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94277667454768] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.508+0000 7f703636c980 20 remove_watcher() i=0 2024-03-28T16:50:16.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.508+0000 7f703636c980 2 removed watcher, disabling cache 2024-03-28T16:50:16.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.508+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94277667458912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae3b590 con 0x55bebae03430 2024-03-28T16:50:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.508+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94277667458912] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.508+0000 7f703636c980 20 remove_watcher() i=1 2024-03-28T16:50:16.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.508+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94277667462752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae3b590 con 0x55bebae0c170 2024-03-28T16:50:16.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.512+0000 7f7033686640 1 -- 172.21.15.170:0/877355170 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94277667462752] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f70280be040 con 0x55bebae0c170 2024-03-28T16:50:16.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.512+0000 7f703636c980 20 remove_watcher() i=2 2024-03-28T16:50:16.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.512+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94277667466512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae0ff30 con 0x7f70280cc040 2024-03-28T16:50:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.512+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94277667466512] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x7f70280cc040 2024-03-28T16:50:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.512+0000 7f703636c980 20 remove_watcher() i=3 2024-03-28T16:50:16.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.512+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94277667470624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae10f60 con 0x55bebae03430 2024-03-28T16:50:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.516+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94277667470624] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f702c00a4d0 con 0x55bebae03430 2024-03-28T16:50:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.516+0000 7f703636c980 20 remove_watcher() i=4 2024-03-28T16:50:16.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.516+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94277667459968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae12d10 con 0x55bebae07970 2024-03-28T16:50:16.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.516+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94277667459968] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.516+0000 7f703636c980 20 remove_watcher() i=5 2024-03-28T16:50:16.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.516+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94277667477104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae11400 con 0x55bebae07970 2024-03-28T16:50:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94277667477104] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 20 remove_watcher() i=6 2024-03-28T16:50:16.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94277667478160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55bebae11400 con 0x55bebae07970 2024-03-28T16:50:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f7033e87640 1 -- 172.21.15.170:0/877355170 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94277667478160] v31'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f7024014070 con 0x55bebae07970 2024-03-28T16:50:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 20 remove_watcher() i=7 2024-03-28T16:50:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebae07970 msgr2=0x55bebae09d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebae07970 0x55bebae09d60 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebae03430 msgr2=0x55bebae058f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebae03430 0x55bebae058f0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70280cc040 msgr2=0x7f70280b5410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f70280cc040 0x7f70280b5410 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55bebae0c170 msgr2=0x55bebae0e600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55bebae0c170 0x55bebae0e600 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6efc0342b0 msgr2=0x7f6efc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6efc0342b0 0x7f6efc036770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f70280c0010 tx=0x7f70280b7040 comp rx=0 tx=0).stop 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 msgr2=0x55bebadf81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f7034688640 1 -- 172.21.15.170:0/877355170 reap_dead start 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf81c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7024024df0 tx=0x7f702400ff60 comp rx=0 tx=0).stop 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 shutdown_connections 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/877355170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebadf1490 0x55bebadf81c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.524 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 >> 172.21.15.170:0/877355170 conn(0x55bebaded820 msgr2=0x55bebadeae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 shutdown_connections 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/877355170 wait complete. 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f0c0342b0 msgr2=0x7f6f0c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f0c0342b0 0x7f6f0c036770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55bebadc9220 tx=0x7f7028005a10 comp rx=0 tx=0).stop 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd47a0 msgr2=0x55bebacd2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd47a0 0x55bebacd2c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7024047cf0 tx=0x7f7024017e80 comp rx=0 tx=0).stop 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 shutdown_connections 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f0c0342b0 0x7f6f0c036770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 --2- 172.21.15.170:0/4088754245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebacd47a0 0x55bebacd2c40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 >> 172.21.15.170:0/4088754245 conn(0x55bebadd71c0 msgr2=0x55bebadd7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 shutdown_connections 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.520+0000 7f703636c980 1 -- 172.21.15.170:0/4088754245 wait complete. 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebadcdd90 msgr2=0x55bebadd0180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55bebadcdd90 0x55bebadd0180 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebacd8360 msgr2=0x55bebadcb7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55bebacd8360 0x55bebadcb7d0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55bebadd1a50 msgr2=0x55bebadd3e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55bebadd1a50 0x55bebadd3e40 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f1c0342b0 msgr2=0x7f6f1c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6f1c0342b0 0x7f6f1c036770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f70280ad000 tx=0x7f70280a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 msgr2=0x55bebadc4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 --2- 172.21.15.170:0/3355757445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bebac488c0 0x55bebadc4ba0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f702400b710 tx=0x7f7024000bd0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f7034688640 1 -- 172.21.15.170:0/3355757445 reap_dead start 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 shutdown_connections 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 >> 172.21.15.170:0/3355757445 conn(0x55bebacce720 msgr2=0x55bebadb8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 shutdown_connections 2024-03-28T16:50:16.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.524+0000 7f703636c980 1 -- 172.21.15.170:0/3355757445 wait complete. 2024-03-28T16:50:16.534 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}]},{"bucket":"myfoo2","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo3","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo4","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]}]}],"summary":[{"user":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}}]} 2024-03-28T16:50:16.534 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}}]} 2024-03-28T16:50:16.535 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-03-28T16:50:16.535 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-03-28T16:50:16.535 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-03-28T16:50:16.622 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:16.622 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:16.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be4649980 1 Processor -- start 2024-03-28T16:50:16.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be4649980 1 -- start start 2024-03-28T16:50:16.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be4649980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55dae1e33e50 0x55dae1e34230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be4649980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55dae1d814c0 con 0x55dae1e34770 2024-03-28T16:50:16.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be4649980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55dae1d480f0 con 0x55dae1e33e50 2024-03-28T16:50:16.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be2164640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55dae1e33e50 0x55dae1e34230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be2164640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55dae1e33e50 0x55dae1e34230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55502/0 (socket says 172.21.15.170:55502) 2024-03-28T16:50:16.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5bdbfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55dae1e34770 0x55dae1e35130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:46094/0 (socket says 172.21.15.170:46094) 2024-03-28T16:50:16.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be2164640 1 -- 172.21.15.170:0/3157069202 learned_addr learned my addr 172.21.15.170:0/3157069202 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be2164640 1 -- 172.21.15.170:0/3157069202 >> v1:172.21.15.119:6789/0 conn(0x55dae1e34770 legacy=0x55dae1e35130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.624+0000 7f5be2164640 1 -- 172.21.15.170:0/3157069202 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dae1e35960 con 0x55dae1e33e50 2024-03-28T16:50:16.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be2164640 1 --2- 172.21.15.170:0/3157069202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1e34230 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400a5c0 tx=0x7f5bd401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1e06cf322ca4a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be1963640 1 -- 172.21.15.170:0/3157069202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40aa470 con 0x55dae1e33e50 2024-03-28T16:50:16.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be1963640 1 -- 172.21.15.170:0/3157069202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd40aa5d0 con 0x55dae1e33e50 2024-03-28T16:50:16.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be1963640 1 -- 172.21.15.170:0/3157069202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40aa890 con 0x55dae1e33e50 2024-03-28T16:50:16.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be4649980 1 -- 172.21.15.170:0/3157069202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 msgr2=0x55dae1e34230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be4649980 1 --2- 172.21.15.170:0/3157069202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1e34230 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400a5c0 tx=0x7f5bd401b340 comp rx=0 tx=0).stop 2024-03-28T16:50:16.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be4649980 1 -- 172.21.15.170:0/3157069202 shutdown_connections 2024-03-28T16:50:16.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be4649980 1 --2- 172.21.15.170:0/3157069202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1e34230 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be4649980 1 -- 172.21.15.170:0/3157069202 >> 172.21.15.170:0/3157069202 conn(0x55dae1e2f720 msgr2=0x55dae1e31b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be4649980 1 -- 172.21.15.170:0/3157069202 shutdown_connections 2024-03-28T16:50:16.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.628+0000 7f5be4649980 1 -- 172.21.15.170:0/3157069202 wait complete. 2024-03-28T16:50:16.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 Processor -- start 2024-03-28T16:50:16.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 -- start start 2024-03-28T16:50:16.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1da9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae1d45620 con 0x55dae1e33e50 2024-03-28T16:50:16.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1da9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1da9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55512/0 (socket says 172.21.15.170:55512) 2024-03-28T16:50:16.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be2164640 1 -- 172.21.15.170:0/2716555514 learned_addr learned my addr 172.21.15.170:0/2716555514 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be2164640 1 -- 172.21.15.170:0/2716555514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4005df0 con 0x55dae1e33e50 2024-03-28T16:50:16.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be2164640 1 --2- 172.21.15.170:0/2716555514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1da9020 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40ac040 tx=0x7f5bd4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c56e7566981ea4a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5ad0608640 1 -- 172.21.15.170:0/2716555514 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40aa9b0 con 0x55dae1e33e50 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5ad0608640 1 -- 172.21.15.170:0/2716555514 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd40aab30 con 0x55dae1e33e50 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5ad0608640 1 -- 172.21.15.170:0/2716555514 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40aae10 con 0x55dae1e33e50 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 -- 172.21.15.170:0/2716555514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 msgr2=0x55dae1da9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 --2- 172.21.15.170:0/2716555514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1da9020 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40ac040 tx=0x7f5bd4003030 comp rx=0 tx=0).stop 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 -- 172.21.15.170:0/2716555514 shutdown_connections 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 --2- 172.21.15.170:0/2716555514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1da9020 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.644+0000 7f5be4649980 1 -- 172.21.15.170:0/2716555514 >> 172.21.15.170:0/2716555514 conn(0x55dae1e2f720 msgr2=0x55dae1f19ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 -- 172.21.15.170:0/2716555514 shutdown_connections 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 -- 172.21.15.170:0/2716555514 wait complete. 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 Processor -- start 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 -- start start 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1f28380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae1d40e00 con 0x55dae1e33e50 2024-03-28T16:50:16.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1f28380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1f28380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55520/0 (socket says 172.21.15.170:55520) 2024-03-28T16:50:16.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be2164640 1 -- 172.21.15.170:0/2324832007 learned_addr learned my addr 172.21.15.170:0/2324832007 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be2164640 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dae1e35960 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be2164640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1f28380 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005940 tx=0x7f5bd40180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5acee05640 1 -- 172.21.15.170:0/2324832007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40c6690 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dae1f288c0 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5acee05640 1 -- 172.21.15.170:0/2324832007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd40c67f0 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5acee05640 1 -- 172.21.15.170:0/2324832007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40c6ab0 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dae1f28cf0 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5acee05640 1 -- 172.21.15.170:0/2324832007 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f5bd40c6c10 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5acee05640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5ac80342b0 0x7f5ac8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5acee05640 1 -- 172.21.15.170:0/2324832007 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5bd40e40e0 con 0x55dae1e33e50 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5ac80342b0 0x7f5ac8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1e34770 0x55dae1f2f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.648+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f2f540 con 0x55dae1e34770 2024-03-28T16:50:16.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2965640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1e34770 0x55dae1f2f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5ac80342b0 0x7f5ac8036770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000f70 tx=0x7f5bcc001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2965640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1e34770 0x55dae1f2f000 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2965640 1 -- 172.21.15.170:0/2324832007 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1e34770 2024-03-28T16:50:16.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f2f8a0 con 0x55dae1e34770 2024-03-28T16:50:16.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2965640 1 -- 172.21.15.170:0/2324832007 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1e34770 2024-03-28T16:50:16.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f315c0 0x55dae1f339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f33ef0 con 0x55dae1f315c0 2024-03-28T16:50:16.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2164640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f315c0 0x55dae1f339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2164640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f315c0 0x55dae1f339b0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2164640 1 -- 172.21.15.170:0/2324832007 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f315c0 2024-03-28T16:50:16.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f303d0 con 0x55dae1e34770 2024-03-28T16:50:16.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be2965640 1 -- 172.21.15.170:0/2324832007 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1e34770 2024-03-28T16:50:16.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55dae1f35260 0x55dae1f37650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f37ca0 con 0x55dae1f35260 2024-03-28T16:50:16.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.652+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55dae1f35260 0x55dae1f37650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55dae1f35260 0x55dae1f37650 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5bdbfff640 1 -- 172.21.15.170:0/2324832007 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f5bcc00f6a0 con 0x55dae1f35260 2024-03-28T16:50:16.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be4649980 1 Processor -- start 2024-03-28T16:50:16.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be4649980 1 -- start start 2024-03-28T16:50:16.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be4649980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f45bd0 0x55dae1f45ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be4649980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae1cd9e30 con 0x55dae1f45bd0 2024-03-28T16:50:16.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f45bd0 0x55dae1f45ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f45bd0 0x55dae1f45ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55536/0 (socket says 172.21.15.170:55536) 2024-03-28T16:50:16.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be2164640 1 -- 172.21.15.170:0/1451998423 learned_addr learned my addr 172.21.15.170:0/1451998423 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be2164640 1 -- 172.21.15.170:0/1451998423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dae1f46530 con 0x55dae1f45bd0 2024-03-28T16:50:16.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be2164640 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f45bd0 0x55dae1f45ff0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40bc560 tx=0x7f5bd40c5bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5ac7fff640 1 -- 172.21.15.170:0/1451998423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40aca80 con 0x55dae1f45bd0 2024-03-28T16:50:16.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5ac7fff640 1 -- 172.21.15.170:0/1451998423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd40e0040 con 0x55dae1f45bd0 2024-03-28T16:50:16.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dae1f46780 con 0x55dae1f45bd0 2024-03-28T16:50:16.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5ac7fff640 1 -- 172.21.15.170:0/1451998423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40acd50 con 0x55dae1f45bd0 2024-03-28T16:50:16.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.656+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dae1f472d0 con 0x55dae1f45bd0 2024-03-28T16:50:16.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5ac7fff640 1 -- 172.21.15.170:0/1451998423 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f5bd40b33f0 con 0x55dae1f45bd0 2024-03-28T16:50:16.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5ac7fff640 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5abc0342b0 0x7f5abc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5ac7fff640 1 -- 172.21.15.170:0/1451998423 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5bd40eeba0 con 0x55dae1f45bd0 2024-03-28T16:50:16.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5abc0342b0 0x7f5abc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 Processor -- start 2024-03-28T16:50:16.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- start start 2024-03-28T16:50:16.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5abc0342b0 0x7f5abc036770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55dae1f30730 tx=0x7f5bcc00f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f52f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae1ccd790 con 0x55dae1f52b20 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f52f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f52f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55550/0 (socket says 172.21.15.170:55550) 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be2164640 1 -- 172.21.15.170:0/1559898841 learned_addr learned my addr 172.21.15.170:0/1559898841 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be2164640 1 -- 172.21.15.170:0/1559898841 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dae1f53510 con 0x55dae1f52b20 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be2164640 1 --2- 172.21.15.170:0/1559898841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f52f40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b3c80 tx=0x7f5bd40b1a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c52aa1adcfa44d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5ac5ffb640 1 -- 172.21.15.170:0/1559898841 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40b5e10 con 0x55dae1f52b20 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5ac5ffb640 1 -- 172.21.15.170:0/1559898841 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd40fb070 con 0x55dae1f52b20 2024-03-28T16:50:16.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5ac5ffb640 1 -- 172.21.15.170:0/1559898841 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40bc790 con 0x55dae1f52b20 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- 172.21.15.170:0/1559898841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 msgr2=0x55dae1f52f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 --2- 172.21.15.170:0/1559898841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f52f40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b3c80 tx=0x7f5bd40b1a60 comp rx=0 tx=0).stop 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- 172.21.15.170:0/1559898841 shutdown_connections 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 --2- 172.21.15.170:0/1559898841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f52f40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- 172.21.15.170:0/1559898841 >> 172.21.15.170:0/1559898841 conn(0x55dae1f4eeb0 msgr2=0x55dae1f4cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- 172.21.15.170:0/1559898841 shutdown_connections 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- 172.21.15.170:0/1559898841 wait complete. 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 Processor -- start 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- start start 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.660+0000 7f5be4649980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dae1f59ee0 con 0x55dae1f52b20 2024-03-28T16:50:16.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2164640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55562/0 (socket says 172.21.15.170:55562) 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 learned_addr learned my addr 172.21.15.170:0/3890879551 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dae1f472d0 con 0x55dae1f52b20 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2164640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f599a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b5d90 tx=0x7f5bd40c5a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5ab3fff640 1 -- 172.21.15.170:0/3890879551 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd4108070 con 0x55dae1f52b20 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5ab3fff640 1 -- 172.21.15.170:0/3890879551 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd40c5510 con 0x55dae1f52b20 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dae1f472d0 con 0x55dae1f52b20 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5ab3fff640 1 -- 172.21.15.170:0/3890879551 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5bd40b1440 con 0x55dae1f52b20 2024-03-28T16:50:16.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dae1f472d0 con 0x55dae1f52b20 2024-03-28T16:50:16.667 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5ab3fff640 1 -- 172.21.15.170:0/3890879551 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f5bd40fd030 con 0x55dae1f52b20 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5ab3fff640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5aac034260 0x7f5aac036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5ab3fff640 1 -- 172.21.15.170:0/3890879551 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5bd4118bf0 con 0x55dae1f52b20 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5aac034260 0x7f5aac036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1f64c10 0x55dae1f670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f67800 con 0x55dae1f64c10 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2965640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1f64c10 0x55dae1f670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5aac034260 0x7f5aac036720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c010 tx=0x7f5bcc010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2965640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1f64c10 0x55dae1f670b0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 20 realm 2024-03-28T16:50:16.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.664+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f67bd0 con 0x55dae1f64c10 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68780 con 0x55dae1f64c10 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f69130 0x55dae1f6b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f6baa0 con 0x55dae1f69130 2024-03-28T16:50:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be2164640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f69130 0x55dae1f6b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be2164640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f69130 0x55dae1f6b520 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 20 searching for the correct realm 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55dae1f6d930 0x55dae1f6fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f704c0 con 0x55dae1f6d930 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55dae1f6d930 0x55dae1f6fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55dae1f6d930 0x55dae1f6fdf0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5bdbfff640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f5bcc01fc60 0x7f5bcc03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.668+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f704c0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f5bcc01fc60 0x7f5bcc03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f5bcc01fc60 0x7f5bcc03a3c0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x7f5bcc01fc60 2024-03-28T16:50:16.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.672+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x7f5bcc01fc60 2024-03-28T16:50:16.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.676+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x7f5bcc01fc60 2024-03-28T16:50:16.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x7f5bcc01fc60 2024-03-28T16:50:16.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.680+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f6d930 2024-03-28T16:50:16.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f69130 2024-03-28T16:50:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x55dae1f64c10 2024-03-28T16:50:16.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f68420 con 0x7f5bcc01fc60 2024-03-28T16:50:16.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:16.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:16.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f6c2a0 con 0x55dae1f64c10 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f6c2a0 con 0x55dae1f64c10 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f6c2a0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 zone default found 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 4 Realm: () 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 zonegroup default 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f6caf0 con 0x55dae1f69130 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f6ced0 con 0x55dae1f6d930 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.684+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f723a0 con 0x55dae1f6d930 2024-03-28T16:50:16.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 20 started sync module instance, tier type = 2024-03-28T16:50:16.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:16.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f71bc0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94398582298400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f71f70 con 0x7f5bcc01fc60 2024-03-28T16:50:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f73100 con 0x55dae1f64c10 2024-03-28T16:50:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94398582298400] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94398582302656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f73140 con 0x55dae1f64c10 2024-03-28T16:50:16.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.688+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f73bb0 con 0x55dae1f6d930 2024-03-28T16:50:16.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.692+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94398582302656] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.692+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.692+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94398582306448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f73f60 con 0x55dae1f6d930 2024-03-28T16:50:16.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.692+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f748d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94398582310208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f75120 con 0x7f5bcc01fc60 2024-03-28T16:50:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f75af0 con 0x55dae1f64c10 2024-03-28T16:50:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94398582306448] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94398582310208] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94398582314480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f74c00 con 0x55dae1f64c10 2024-03-28T16:50:16.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.696+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f76340 con 0x55dae1f69130 2024-03-28T16:50:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94398582314480] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94398582316704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f74c00 con 0x55dae1f69130 2024-03-28T16:50:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f76b40 con 0x55dae1f69130 2024-03-28T16:50:16.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94398582316704] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94398582303712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f77340 con 0x55dae1f69130 2024-03-28T16:50:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.700+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f77cb0 con 0x55dae1f69130 2024-03-28T16:50:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94398582303712] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94398582320800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f77e90 con 0x55dae1f69130 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=0 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=1 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=2 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=3 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=4 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=5 2024-03-28T16:50:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=6 2024-03-28T16:50:16.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94398582320800] v31'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 20 add_watcher() i=7 2024-03-28T16:50:16.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:16.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f7b0c0 con 0x55dae1f64c10 2024-03-28T16:50:16.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.704+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94398582323792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f78310 con 0x55dae1f64c10 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94398582323792] v31'206 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5a98fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 5 note: GC not initialized 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5a137fe640 20 reqs_thread_entry: start 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:16.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 20 Filter name: none 2024-03-28T16:50:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5a117fa640 20 reqs_thread_entry: start 2024-03-28T16:50:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f9cbe0 con 0x55dae1f64c10 2024-03-28T16:50:16.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:16.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f9cbe0 con 0x55dae1f69130 2024-03-28T16:50:16.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.708+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5a98fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94398582323792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f9cbe0 con 0x55dae1f64c10 2024-03-28T16:50:16.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94398582323792] v31'207 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94398582298400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f9cbe0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94398582298400] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5be4649980 20 remove_watcher() i=0 2024-03-28T16:50:16.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5be4649980 2 removed watcher, disabling cache 2024-03-28T16:50:16.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.712+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94398582302656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f9cbe0 con 0x55dae1f64c10 2024-03-28T16:50:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.716+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94398582302656] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.716+0000 7f5be4649980 20 remove_watcher() i=1 2024-03-28T16:50:16.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.716+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94398582306448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f9cbe0 con 0x55dae1f6d930 2024-03-28T16:50:16.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.716+0000 7f5bdbfff640 1 -- 172.21.15.170:0/3890879551 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94398582306448] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bcc033040 con 0x55dae1f6d930 2024-03-28T16:50:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.716+0000 7f5be4649980 20 remove_watcher() i=2 2024-03-28T16:50:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.716+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94398582310208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f71720 con 0x7f5bcc01fc60 2024-03-28T16:50:16.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.720+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94398582310208] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x7f5bcc01fc60 2024-03-28T16:50:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.720+0000 7f5be4649980 20 remove_watcher() i=3 2024-03-28T16:50:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.720+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94398582314480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f727c0 con 0x55dae1f64c10 2024-03-28T16:50:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.720+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94398582314480] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bdc00a4d0 con 0x55dae1f64c10 2024-03-28T16:50:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.720+0000 7f5be4649980 20 remove_watcher() i=4 2024-03-28T16:50:16.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.720+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94398582316704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f74540 con 0x55dae1f69130 2024-03-28T16:50:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.724+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94398582316704] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.724+0000 7f5be4649980 20 remove_watcher() i=5 2024-03-28T16:50:16.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.724+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94398582303712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f71720 con 0x55dae1f69130 2024-03-28T16:50:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.724+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94398582303712] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.724+0000 7f5be4649980 20 remove_watcher() i=6 2024-03-28T16:50:16.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.724+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94398582320800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55dae1f72c60 con 0x55dae1f69130 2024-03-28T16:50:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be2164640 1 -- 172.21.15.170:0/3890879551 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94398582320800] v31'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f5bd40b0020 con 0x55dae1f69130 2024-03-28T16:50:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 20 remove_watcher() i=7 2024-03-28T16:50:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f69130 msgr2=0x55dae1f6b520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f69130 0x55dae1f6b520 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1f64c10 msgr2=0x55dae1f670b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1f64c10 0x55dae1f670b0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f5bcc01fc60 msgr2=0x7f5bcc03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f5bcc01fc60 0x7f5bcc03a3c0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55dae1f6d930 msgr2=0x55dae1f6fdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55dae1f6d930 0x55dae1f6fdf0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5aac034260 msgr2=0x7f5aac036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5aac034260 0x7f5aac036720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c010 tx=0x7f5bcc010000 comp rx=0 tx=0).stop 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be2965640 1 -- 172.21.15.170:0/3890879551 reap_dead start 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 msgr2=0x55dae1f599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f599a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40b5d90 tx=0x7f5bd40c5a60 comp rx=0 tx=0).stop 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 shutdown_connections 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/3890879551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f52b20 0x55dae1f599a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 >> 172.21.15.170:0/3890879551 conn(0x55dae1f4eeb0 msgr2=0x55dae1f4c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 shutdown_connections 2024-03-28T16:50:16.730 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/3890879551 wait complete. 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5abc0342b0 msgr2=0x7f5abc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5abc0342b0 0x7f5abc036770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55dae1f30730 tx=0x7f5bcc00f000 comp rx=0 tx=0).stop 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f45bd0 msgr2=0x55dae1f45ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f45bd0 0x55dae1f45ff0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40bc560 tx=0x7f5bd40c5bb0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 shutdown_connections 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5abc0342b0 0x7f5abc036770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/1451998423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1f45bd0 0x55dae1f45ff0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 >> 172.21.15.170:0/1451998423 conn(0x55dae1f3a9e0 msgr2=0x55dae1f3b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 shutdown_connections 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/1451998423 wait complete. 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f315c0 msgr2=0x55dae1f339b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55dae1f315c0 0x55dae1f339b0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1e34770 msgr2=0x55dae1f2f000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55dae1e34770 0x55dae1f2f000 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55dae1f35260 msgr2=0x55dae1f37650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55dae1f35260 0x55dae1f37650 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5ac80342b0 msgr2=0x7f5ac8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f5ac80342b0 0x7f5ac8036770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000f70 tx=0x7f5bcc001380 comp rx=0 tx=0).stop 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 msgr2=0x55dae1f28380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 --2- 172.21.15.170:0/2324832007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dae1e33e50 0x55dae1f28380 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005940 tx=0x7f5bd40180e0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be2965640 1 -- 172.21.15.170:0/2324832007 reap_dead start 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 shutdown_connections 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 >> 172.21.15.170:0/2324832007 conn(0x55dae1e2f720 msgr2=0x55dae1f18a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 shutdown_connections 2024-03-28T16:50:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.728+0000 7f5be4649980 1 -- 172.21.15.170:0/2324832007 wait complete. 2024-03-28T16:50:16.740 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}]},{"bucket":"myfoo2","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo3","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo4","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]}]}],"summary":[{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}}]} 2024-03-28T16:50:16.740 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-03-28T16:50:16.741 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-03-28T16:50:16.741 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-03-28T16:50:16.741 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-03-28T16:50:16.815 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:16.815 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:16.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49dd1c5980 1 Processor -- start 2024-03-28T16:50:16.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49dd1c5980 1 -- start start 2024-03-28T16:50:16.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49dd1c5980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x561e0b264640 0x561e0b266a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49dd1c5980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561e0b1aa4c0 con 0x561e0b260140 2024-03-28T16:50:16.816 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49dd1c5980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x561e0b1710f0 con 0x561e0b264640 2024-03-28T16:50:16.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49dace0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x561e0b260140 0x561e0b260520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:46098/0 (socket says 172.21.15.170:46098) 2024-03-28T16:50:16.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49dace0640 1 -- 172.21.15.170:0/1201103335 learned_addr learned my addr 172.21.15.170:0/1201103335 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:16.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49da4df640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x561e0b264640 0x561e0b266a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49da4df640 1 -- 172.21.15.170:0/1201103335 >> v1:172.21.15.119:6789/0 conn(0x561e0b260140 legacy=0x561e0b260520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.816+0000 7f49da4df640 1 -- 172.21.15.170:0/1201103335 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0b267f80 con 0x561e0b264640 2024-03-28T16:50:16.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49da4df640 1 --2- 172.21.15.170:0/1201103335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b264640 0x561e0b266a30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f49d000a480 tx=0x7f49d001b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ae8a4235356d9f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49d9cde640 1 -- 172.21.15.170:0/1201103335 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d00ab540 con 0x561e0b264640 2024-03-28T16:50:16.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49d9cde640 1 -- 172.21.15.170:0/1201103335 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d00ab6a0 con 0x561e0b264640 2024-03-28T16:50:16.820 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49d9cde640 1 -- 172.21.15.170:0/1201103335 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d00ab960 con 0x561e0b264640 2024-03-28T16:50:16.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1201103335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b264640 msgr2=0x561e0b266a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/1201103335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b264640 0x561e0b266a30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f49d000a480 tx=0x7f49d001b280 comp rx=0 tx=0).stop 2024-03-28T16:50:16.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1201103335 shutdown_connections 2024-03-28T16:50:16.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/1201103335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b264640 0x561e0b266a30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1201103335 >> 172.21.15.170:0/1201103335 conn(0x561e0b1d3860 msgr2=0x561e0b1d3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1201103335 shutdown_connections 2024-03-28T16:50:16.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.820+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1201103335 wait complete. 2024-03-28T16:50:16.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dd1c5980 1 Processor -- start 2024-03-28T16:50:16.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dd1c5980 1 -- start start 2024-03-28T16:50:16.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dd1c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b34a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dd1c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b16e620 con 0x561e0b260140 2024-03-28T16:50:16.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b34a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b34a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55584/0 (socket says 172.21.15.170:55584) 2024-03-28T16:50:16.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dace0640 1 -- 172.21.15.170:0/4125301670 learned_addr learned my addr 172.21.15.170:0/4125301670 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dace0640 1 -- 172.21.15.170:0/4125301670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0005d30 con 0x561e0b260140 2024-03-28T16:50:16.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f49dace0640 1 --2- 172.21.15.170:0/4125301670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b34a0d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f49cc001c90 tx=0x7f49cc00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c7cdbff60c1609 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f48c8e09640 1 -- 172.21.15.170:0/4125301670 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc00e690 con 0x561e0b260140 2024-03-28T16:50:16.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f48c8e09640 1 -- 172.21.15.170:0/4125301670 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00e7f0 con 0x561e0b260140 2024-03-28T16:50:16.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.836+0000 7f48c8e09640 1 -- 172.21.15.170:0/4125301670 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc00eaf0 con 0x561e0b260140 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- 172.21.15.170:0/4125301670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 msgr2=0x561e0b34a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/4125301670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b34a0d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f49cc001c90 tx=0x7f49cc00b6d0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- 172.21.15.170:0/4125301670 shutdown_connections 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/4125301670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b34a0d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- 172.21.15.170:0/4125301670 >> 172.21.15.170:0/4125301670 conn(0x561e0b1d3860 msgr2=0x561e0b345c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- 172.21.15.170:0/4125301670 shutdown_connections 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- 172.21.15.170:0/4125301670 wait complete. 2024-03-28T16:50:16.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 Processor -- start 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- start start 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b350f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b169e00 con 0x561e0b260140 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b350f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b350f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55600/0 (socket says 172.21.15.170:55600) 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dace0640 1 -- 172.21.15.170:0/3198971858 learned_addr learned my addr 172.21.15.170:0/3198971858 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dace0640 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0b1aa4c0 con 0x561e0b260140 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dace0640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b350f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f49cc001ee0 tx=0x7f49cc000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f48c7606640 1 -- 172.21.15.170:0/3198971858 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc01f070 con 0x561e0b260140 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f48c7606640 1 -- 172.21.15.170:0/3198971858 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc018830 con 0x561e0b260140 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e0b351460 con 0x561e0b260140 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f48c7606640 1 -- 172.21.15.170:0/3198971858 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc018ad0 con 0x561e0b260140 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.840+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e0b351890 con 0x561e0b260140 2024-03-28T16:50:16.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f48c7606640 1 -- 172.21.15.170:0/3198971858 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f49cc00a040 con 0x561e0b260140 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f48c7606640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48c0034350 0x7f48c0036810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f48c7606640 1 -- 172.21.15.170:0/3198971858 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f49cc0497e0 con 0x561e0b260140 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49da4df640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48c0034350 0x7f48c0036810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b264640 0x561e0b357ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3580e0 con 0x561e0b264640 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49db4e1640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b264640 0x561e0b357ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49da4df640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48c0034350 0x7f48c0036810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f49d00ad000 tx=0x7f49d00a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49db4e1640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b264640 0x561e0b357ba0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49db4e1640 1 -- 172.21.15.170:0/3198971858 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b264640 2024-03-28T16:50:16.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b358440 con 0x561e0b264640 2024-03-28T16:50:16.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49db4e1640 1 -- 172.21.15.170:0/3198971858 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b264640 2024-03-28T16:50:16.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b359dd0 0x561e0b35c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b35c990 con 0x561e0b359dd0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49dace0640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b359dd0 0x561e0b35c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.844+0000 7f49dace0640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b359dd0 0x561e0b35c200 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dace0640 1 -- 172.21.15.170:0/3198971858 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f49cc014070 con 0x561e0b359dd0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b358c40 con 0x561e0b264640 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49db4e1640 1 -- 172.21.15.170:0/3198971858 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b264640 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561e0b35ddb0 0x561e0b3601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b360860 con 0x561e0b35ddb0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49da4df640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561e0b35ddb0 0x561e0b3601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49da4df640 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561e0b35ddb0 0x561e0b3601a0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49da4df640 1 -- 172.21.15.170:0/3198971858 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b35ddb0 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 Processor -- start 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 -- start start 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b36e790 0x561e0b36ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b102e30 con 0x561e0b36e790 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b36e790 0x561e0b36ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b36e790 0x561e0b36ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55608/0 (socket says 172.21.15.170:55608) 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dace0640 1 -- 172.21.15.170:0/1042591449 learned_addr learned my addr 172.21.15.170:0/1042591449 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dace0640 1 -- 172.21.15.170:0/1042591449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0b36f0f0 con 0x561e0b36e790 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dace0640 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b36e790 0x561e0b36ebb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f49cc049010 tx=0x7f49cc02d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f48c4e01640 1 -- 172.21.15.170:0/1042591449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc00d3e0 con 0x561e0b36e790 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f48c4e01640 1 -- 172.21.15.170:0/1042591449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc037040 con 0x561e0b36e790 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e0b36f340 con 0x561e0b36e790 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f48c4e01640 1 -- 172.21.15.170:0/1042591449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc00d690 con 0x561e0b36e790 2024-03-28T16:50:16.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.848+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e0b36fe90 con 0x561e0b36e790 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f48c4e01640 1 -- 172.21.15.170:0/1042591449 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f49cc0490b0 con 0x561e0b36e790 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f48c4e01640 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48b4034300 0x7f48b40367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f48c4e01640 1 -- 172.21.15.170:0/1042591449 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f49cc019c80 con 0x561e0b36e790 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49da4df640 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48b4034300 0x7f48b40367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 Processor -- start 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- start start 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b37ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b0f6790 con 0x561e0b37b640 2024-03-28T16:50:16.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49da4df640 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48b4034300 0x7f48b40367c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561e0b35cd20 tx=0x7f49d0005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b37ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b37ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55614/0 (socket says 172.21.15.170:55614) 2024-03-28T16:50:16.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dace0640 1 -- 172.21.15.170:0/2541285628 learned_addr learned my addr 172.21.15.170:0/2541285628 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dace0640 1 -- 172.21.15.170:0/2541285628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0b37bff0 con 0x561e0b37b640 2024-03-28T16:50:16.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dace0640 1 --2- 172.21.15.170:0/2541285628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b37ba20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f49cc023c70 tx=0x7f49cc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e059a6a8ba361e78 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f48be7fc640 1 -- 172.21.15.170:0/2541285628 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc022a80 con 0x561e0b37b640 2024-03-28T16:50:16.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f48be7fc640 1 -- 172.21.15.170:0/2541285628 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc022be0 con 0x561e0b37b640 2024-03-28T16:50:16.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f48be7fc640 1 -- 172.21.15.170:0/2541285628 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc00c4a0 con 0x561e0b37b640 2024-03-28T16:50:16.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- 172.21.15.170:0/2541285628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 msgr2=0x561e0b37ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/2541285628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b37ba20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f49cc023c70 tx=0x7f49cc00c7b0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- 172.21.15.170:0/2541285628 shutdown_connections 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/2541285628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b37ba20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- 172.21.15.170:0/2541285628 >> 172.21.15.170:0/2541285628 conn(0x561e0b377980 msgr2=0x561e0b375a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- 172.21.15.170:0/2541285628 shutdown_connections 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- 172.21.15.170:0/2541285628 wait complete. 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 Processor -- start 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.852+0000 7f49dd1c5980 1 -- start start 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dd1c5980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b382430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dd1c5980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e0b382970 con 0x561e0b37b640 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b382430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b382430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55626/0 (socket says 172.21.15.170:55626) 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 learned_addr learned my addr 172.21.15.170:0/98930777 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e0b36fe90 con 0x561e0b37b640 2024-03-28T16:50:16.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dace0640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b382430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f49cc023c70 tx=0x7f49cc00de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f48bcff9640 1 -- 172.21.15.170:0/98930777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc05f040 con 0x561e0b37b640 2024-03-28T16:50:16.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f48bcff9640 1 -- 172.21.15.170:0/98930777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc046280 con 0x561e0b37b640 2024-03-28T16:50:16.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e0b36fe90 con 0x561e0b37b640 2024-03-28T16:50:16.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f48bcff9640 1 -- 172.21.15.170:0/98930777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49cc031aa0 con 0x561e0b37b640 2024-03-28T16:50:16.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e0b36fe90 con 0x561e0b37b640 2024-03-28T16:50:16.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f48bcff9640 1 -- 172.21.15.170:0/98930777 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f49cc031c00 con 0x561e0b37b640 2024-03-28T16:50:16.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f48bcff9640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48a40342b0 0x7f48a4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f48bcff9640 1 -- 172.21.15.170:0/98930777 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f49cc070bf0 con 0x561e0b37b640 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49da4df640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48a40342b0 0x7f48a4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b38d690 0x561e0b38fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390280 con 0x561e0b38d690 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49db4e1640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b38d690 0x561e0b38fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49da4df640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48a40342b0 0x7f48a4036770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49d00ab670 tx=0x7f49d00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.856+0000 7f49db4e1640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b38d690 0x561e0b38fb30 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 realm 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390650 con 0x561e0b38d690 2024-03-28T16:50:16.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b391200 con 0x561e0b38d690 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b391bb0 0x561e0b393fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b394520 con 0x561e0b391bb0 2024-03-28T16:50:16.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dace0640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b391bb0 0x561e0b393fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dace0640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b391bb0 0x561e0b393fa0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f49cc014070 con 0x561e0b391bb0 2024-03-28T16:50:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 20 searching for the correct realm 2024-03-28T16:50:16.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561e0b396420 0x561e0b398830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b398f00 con 0x561e0b396420 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49da4df640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561e0b396420 0x561e0b398830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49da4df640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561e0b396420 0x561e0b398830 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49da4df640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f49d00d9040 0x7f49d00d8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b398f00 con 0x7f49d00d9040 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.860+0000 7f49db4e1640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f49d00d9040 0x7f49d00d8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f49d00d9040 0x7f49d00d8920 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc014070 con 0x561e0b391bb0 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x7f49d00d9040 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.864+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x7f49d00d9040 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.868+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x7f49d00d9040 2024-03-28T16:50:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x7f49d00d9040 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.872+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b396420 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b391bb0 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x561e0b38d690 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b390ea0 con 0x7f49d00d9040 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b394d90 con 0x561e0b38d690 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b394d60 con 0x561e0b38d690 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b394d60 con 0x7f49d00d9040 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 zone default found 2024-03-28T16:50:16.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 4 Realm: () 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 zonegroup default 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3955e0 con 0x561e0b391bb0 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3959e0 con 0x561e0b396420 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39ad50 con 0x561e0b396420 2024-03-28T16:50:16.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 started sync module instance, tier type = 2024-03-28T16:50:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:16.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.876+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39a600 con 0x7f49d00d9040 2024-03-28T16:50:16.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.880+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.880+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94687037333856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39a9b0 con 0x7f49d00d9040 2024-03-28T16:50:16.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.880+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39ba90 con 0x561e0b38d690 2024-03-28T16:50:16.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.880+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94687037333856] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.880+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.880+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94687037337936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39ba10 con 0x561e0b38d690 2024-03-28T16:50:16.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.880+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39c290 con 0x561e0b396420 2024-03-28T16:50:16.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.884+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94687037337936] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.884+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.884+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94687037339264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39c640 con 0x561e0b396420 2024-03-28T16:50:16.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.884+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39cfb0 con 0x7f49d00d9040 2024-03-28T16:50:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.884+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.884+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94687037344800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39d6a0 con 0x7f49d00d9040 2024-03-28T16:50:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.884+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39e170 con 0x561e0b38d690 2024-03-28T16:50:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94687037339264] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94687037344800] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94687037348944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39e130 con 0x561e0b38d690 2024-03-28T16:50:16.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39d270 con 0x561e0b391bb0 2024-03-28T16:50:16.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94687037348944] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94687037340384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39e9c0 con 0x561e0b391bb0 2024-03-28T16:50:16.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.888+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39f1c0 con 0x561e0b391bb0 2024-03-28T16:50:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.892+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.892+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94687037355296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39fde0 con 0x561e0b391bb0 2024-03-28T16:50:16.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.892+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3a07f0 con 0x561e0b391bb0 2024-03-28T16:50:16.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.892+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94687037340384] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.892+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94687037355296] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94687037351200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3a0b50 con 0x561e0b391bb0 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=0 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=1 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=2 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=3 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=4 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=5 2024-03-28T16:50:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=6 2024-03-28T16:50:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94687037351200] v31'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 20 add_watcher() i=7 2024-03-28T16:50:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3a4020 con 0x561e0b38d690 2024-03-28T16:50:16.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.896+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94687037360976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3a1410 con 0x561e0b38d690 2024-03-28T16:50:16.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94687037360976] v31'208 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f48917e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:16.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:16.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 5 note: GC not initialized 2024-03-28T16:50:16.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f480bfff640 20 reqs_thread_entry: start 2024-03-28T16:50:16.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:16.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 20 Filter name: none 2024-03-28T16:50:16.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f4809ffb640 20 reqs_thread_entry: start 2024-03-28T16:50:16.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 20 rados->read ofs=0 len=0 2024-03-28T16:50:16.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3c58c0 con 0x561e0b38d690 2024-03-28T16:50:16.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:16.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3c58c0 con 0x561e0b391bb0 2024-03-28T16:50:16.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f48917e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:16.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.900+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94687037360976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3c58c0 con 0x561e0b38d690 2024-03-28T16:50:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.904+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94687037360976] v31'209 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.904+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94687037333856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3c58c0 con 0x7f49d00d9040 2024-03-28T16:50:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.904+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94687037333856] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.904+0000 7f49dd1c5980 20 remove_watcher() i=0 2024-03-28T16:50:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.904+0000 7f49dd1c5980 2 removed watcher, disabling cache 2024-03-28T16:50:16.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.904+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94687037337936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3c58c0 con 0x561e0b38d690 2024-03-28T16:50:16.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94687037337936] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49dd1c5980 20 remove_watcher() i=1 2024-03-28T16:50:16.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94687037339264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b3c58c0 con 0x561e0b396420 2024-03-28T16:50:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49da4df640 1 -- 172.21.15.170:0/98930777 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94687037339264] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d00abc60 con 0x561e0b396420 2024-03-28T16:50:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49dd1c5980 20 remove_watcher() i=2 2024-03-28T16:50:16.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94687037344800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39b150 con 0x7f49d00d9040 2024-03-28T16:50:16.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94687037344800] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x7f49d00d9040 2024-03-28T16:50:16.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49dd1c5980 20 remove_watcher() i=3 2024-03-28T16:50:16.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.908+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94687037348944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39cc20 con 0x561e0b38d690 2024-03-28T16:50:16.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.912+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94687037348944] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49d400a4d0 con 0x561e0b38d690 2024-03-28T16:50:16.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.912+0000 7f49dd1c5980 20 remove_watcher() i=4 2024-03-28T16:50:16.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.912+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94687037340384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39b150 con 0x561e0b391bb0 2024-03-28T16:50:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.916+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94687037340384] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.916+0000 7f49dd1c5980 20 remove_watcher() i=5 2024-03-28T16:50:16.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.916+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94687037355296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39bb20 con 0x561e0b391bb0 2024-03-28T16:50:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.916+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94687037355296] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.916+0000 7f49dd1c5980 20 remove_watcher() i=6 2024-03-28T16:50:16.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.916+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94687037351200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x561e0b39bb20 con 0x561e0b391bb0 2024-03-28T16:50:16.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dace0640 1 -- 172.21.15.170:0/98930777 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94687037351200] v31'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f49cc079040 con 0x561e0b391bb0 2024-03-28T16:50:16.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 20 remove_watcher() i=7 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b391bb0 msgr2=0x561e0b393fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b391bb0 0x561e0b393fa0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b38d690 msgr2=0x561e0b38fb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b38d690 0x561e0b38fb30 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f49d00d9040 msgr2=0x7f49d00d8920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f49d00d9040 0x7f49d00d8920 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561e0b396420 msgr2=0x561e0b398830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x561e0b396420 0x561e0b398830 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48a40342b0 msgr2=0x7f48a4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48a40342b0 0x7f48a4036770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49d00ab670 tx=0x7f49d00a6000 comp rx=0 tx=0).stop 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 msgr2=0x561e0b382430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/98930777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b37b640 0x561e0b382430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f49cc023c70 tx=0x7f49cc00de60 comp rx=0 tx=0).stop 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 reap_dead start 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49db4e1640 1 -- 172.21.15.170:0/98930777 reap_dead start 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 shutdown_connections 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 >> 172.21.15.170:0/98930777 conn(0x561e0b377980 msgr2=0x561e0b375030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 shutdown_connections 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/98930777 wait complete. 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48b4034300 msgr2=0x7f48b40367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48b4034300 0x7f48b40367c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561e0b35cd20 tx=0x7f49d0005a10 comp rx=0 tx=0).stop 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b36e790 msgr2=0x561e0b36ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b36e790 0x561e0b36ebb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f49cc049010 tx=0x7f49cc02d710 comp rx=0 tx=0).stop 2024-03-28T16:50:16.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 shutdown_connections 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48b4034300 0x7f48b40367c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/1042591449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b36e790 0x561e0b36ebb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 >> 172.21.15.170:0/1042591449 conn(0x561e0b3635a0 msgr2=0x561e0b3640b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 shutdown_connections 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/1042591449 wait complete. 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b359dd0 msgr2=0x561e0b35c200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x561e0b359dd0 0x561e0b35c200 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b264640 msgr2=0x561e0b357ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x561e0b264640 0x561e0b357ba0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561e0b35ddb0 msgr2=0x561e0b3601a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x561e0b35ddb0 0x561e0b3601a0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48c0034350 msgr2=0x7f48c0036810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f48c0034350 0x7f48c0036810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f49d00ad000 tx=0x7f49d00a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:16.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 msgr2=0x561e0b350f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 --2- 172.21.15.170:0/3198971858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561e0b260140 0x561e0b350f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f49cc001ee0 tx=0x7f49cc000ee0 comp rx=0 tx=0).stop 2024-03-28T16:50:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49db4e1640 1 -- 172.21.15.170:0/3198971858 reap_dead start 2024-03-28T16:50:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 shutdown_connections 2024-03-28T16:50:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 >> 172.21.15.170:0/3198971858 conn(0x561e0b1d3860 msgr2=0x561e0b344770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 shutdown_connections 2024-03-28T16:50:16.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:16.920+0000 7f49dd1c5980 1 -- 172.21.15.170:0/3198971858 wait complete. 2024-03-28T16:50:16.933 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1}]},{"bucket":"myfoo2","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-03-28T16:00:00.000000Z","epoch":1711641600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4}],"total":{"bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4}}]} 2024-03-28T16:50:16.933 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}]}, {'bucket': 'myfoo2', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-03-28T16:00:00.000000Z', 'epoch': 1711641600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}}]} 2024-03-28T16:50:16.933 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-03-28T16:50:16.933 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-03-28T16:50:16.933 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-03-28T16:50:17.015 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:17.015 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:17.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c9c415980 1 Processor -- start 2024-03-28T16:50:17.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c9c415980 1 -- start start 2024-03-28T16:50:17.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c9c415980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5646f0cae170 0x5646f0cae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c9c415980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5646f0bf84c0 con 0x5646f0caea90 2024-03-28T16:50:17.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c9c415980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5646f0bbf0f0 con 0x5646f0cae170 2024-03-28T16:50:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c99f30640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5646f0cae170 0x5646f0cae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c99f30640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5646f0cae170 0x5646f0cae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55630/0 (socket says 172.21.15.170:55630) 2024-03-28T16:50:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c99f30640 1 -- 172.21.15.170:0/4265204035 learned_addr learned my addr 172.21.15.170:0/4265204035 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c9972f640 1 --1- 172.21.15.170:0/4265204035 >> v1:172.21.15.119:6789/0 conn(0x5646f0caea90 0x5646f0caf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:46100/0 (socket says 172.21.15.170:46100) 2024-03-28T16:50:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c99f30640 1 -- 172.21.15.170:0/4265204035 >> v1:172.21.15.119:6789/0 conn(0x5646f0caea90 legacy=0x5646f0caf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.016+0000 7f9c99f30640 1 -- 172.21.15.170:0/4265204035 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f0cafc70 con 0x5646f0cae170 2024-03-28T16:50:17.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c99f30640 1 --2- 172.21.15.170:0/4265204035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0cae550 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400a5c0 tx=0x7f9c9401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29189933f782927c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c98f2e640 1 -- 172.21.15.170:0/4265204035 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940aa470 con 0x5646f0cae170 2024-03-28T16:50:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c98f2e640 1 -- 172.21.15.170:0/4265204035 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940aa5d0 con 0x5646f0cae170 2024-03-28T16:50:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c98f2e640 1 -- 172.21.15.170:0/4265204035 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940aa890 con 0x5646f0cae170 2024-03-28T16:50:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c9c415980 1 -- 172.21.15.170:0/4265204035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 msgr2=0x5646f0cae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c9c415980 1 --2- 172.21.15.170:0/4265204035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0cae550 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400a5c0 tx=0x7f9c9401b340 comp rx=0 tx=0).stop 2024-03-28T16:50:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c9c415980 1 -- 172.21.15.170:0/4265204035 shutdown_connections 2024-03-28T16:50:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c9c415980 1 --2- 172.21.15.170:0/4265204035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0cae550 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c9c415980 1 -- 172.21.15.170:0/4265204035 >> 172.21.15.170:0/4265204035 conn(0x5646f0c21890 msgr2=0x5646f0c21ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c9c415980 1 -- 172.21.15.170:0/4265204035 shutdown_connections 2024-03-28T16:50:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.020+0000 7f9c9c415980 1 -- 172.21.15.170:0/4265204035 wait complete. 2024-03-28T16:50:17.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c9c415980 1 Processor -- start 2024-03-28T16:50:17.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c9c415980 1 -- start start 2024-03-28T16:50:17.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c9c415980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0d98700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c9c415980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f0bbc620 con 0x5646f0cae170 2024-03-28T16:50:17.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0d98700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0d98700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55636/0 (socket says 172.21.15.170:55636) 2024-03-28T16:50:17.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c99f30640 1 -- 172.21.15.170:0/3144021757 learned_addr learned my addr 172.21.15.170:0/3144021757 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c99f30640 1 -- 172.21.15.170:0/3144021757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94005df0 con 0x5646f0cae170 2024-03-28T16:50:17.038 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.036+0000 7f9c99f30640 1 --2- 172.21.15.170:0/3144021757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0d98700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9c940ac040 tx=0x7f9c94003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ec684937b7943e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9b87e07640 1 -- 172.21.15.170:0/3144021757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940aa9b0 con 0x5646f0cae170 2024-03-28T16:50:17.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9b87e07640 1 -- 172.21.15.170:0/3144021757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940aab30 con 0x5646f0cae170 2024-03-28T16:50:17.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9b87e07640 1 -- 172.21.15.170:0/3144021757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940aae10 con 0x5646f0cae170 2024-03-28T16:50:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- 172.21.15.170:0/3144021757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 msgr2=0x5646f0d98700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3144021757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0d98700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9c940ac040 tx=0x7f9c94003030 comp rx=0 tx=0).stop 2024-03-28T16:50:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- 172.21.15.170:0/3144021757 shutdown_connections 2024-03-28T16:50:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3144021757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0cae170 0x5646f0d98700 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- 172.21.15.170:0/3144021757 >> 172.21.15.170:0/3144021757 conn(0x5646f0c21890 msgr2=0x5646f0d94290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- 172.21.15.170:0/3144021757 shutdown_connections 2024-03-28T16:50:17.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- 172.21.15.170:0/3144021757 wait complete. 2024-03-28T16:50:17.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 Processor -- start 2024-03-28T16:50:17.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- start start 2024-03-28T16:50:17.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0caea90 0x5646f0d9f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f0bb7e00 con 0x5646f0caea90 2024-03-28T16:50:17.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0caea90 0x5646f0d9f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0caea90 0x5646f0d9f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55640/0 (socket says 172.21.15.170:55640) 2024-03-28T16:50:17.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c99f30640 1 -- 172.21.15.170:0/375742683 learned_addr learned my addr 172.21.15.170:0/375742683 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c99f30640 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f0cafc70 con 0x5646f0caea90 2024-03-28T16:50:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c99f30640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0caea90 0x5646f0d9f780 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c10 tx=0x7f9c940180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9b86604640 1 -- 172.21.15.170:0/375742683 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940aa950 con 0x5646f0caea90 2024-03-28T16:50:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5646f0d9fcc0 con 0x5646f0caea90 2024-03-28T16:50:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9b86604640 1 -- 172.21.15.170:0/375742683 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940c6650 con 0x5646f0caea90 2024-03-28T16:50:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9b86604640 1 -- 172.21.15.170:0/375742683 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940c6930 con 0x5646f0caea90 2024-03-28T16:50:17.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.040+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646f0da0170 con 0x5646f0caea90 2024-03-28T16:50:17.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9b86604640 1 -- 172.21.15.170:0/375742683 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9c940c6a90 con 0x5646f0caea90 2024-03-28T16:50:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9b86604640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b800342b0 0x7f9b80036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9b86604640 1 -- 172.21.15.170:0/375742683 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9c940e49b0 con 0x5646f0caea90 2024-03-28T16:50:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9972f640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b800342b0 0x7f9b80036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0da4300 0x5646f0da66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0da6c30 con 0x5646f0da4300 2024-03-28T16:50:17.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9a731640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0da4300 0x5646f0da66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9972f640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b800342b0 0x7f9b80036770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000f70 tx=0x7f9c90001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9a731640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0da4300 0x5646f0da66f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9a731640 1 -- 172.21.15.170:0/375742683 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0da4300 2024-03-28T16:50:17.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0da6f90 con 0x5646f0da4300 2024-03-28T16:50:17.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9a731640 1 -- 172.21.15.170:0/375742683 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0da4300 2024-03-28T16:50:17.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0da8cb0 0x5646f0dab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dab5e0 con 0x5646f0da8cb0 2024-03-28T16:50:17.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c99f30640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0da8cb0 0x5646f0dab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.044+0000 7f9c99f30640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0da8cb0 0x5646f0dab0a0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c99f30640 1 -- 172.21.15.170:0/375742683 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0da8cb0 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0da7af0 con 0x5646f0da4300 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9a731640 1 -- 172.21.15.170:0/375742683 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0da4300 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5646f0dac950 0x5646f0daed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0daf390 con 0x5646f0dac950 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9972f640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5646f0dac950 0x5646f0daed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9972f640 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5646f0dac950 0x5646f0daed40 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9972f640 1 -- 172.21.15.170:0/375742683 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f9c9000f6a0 con 0x5646f0dac950 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 Processor -- start 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 -- start start 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dbd2c0 0x5646f0dbd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f0b50e30 con 0x5646f0dbd2c0 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dbd2c0 0x5646f0dbd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dbd2c0 0x5646f0dbd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55650/0 (socket says 172.21.15.170:55650) 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c99f30640 1 -- 172.21.15.170:0/2732919106 learned_addr learned my addr 172.21.15.170:0/2732919106 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c99f30640 1 -- 172.21.15.170:0/2732919106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f0dbdc20 con 0x5646f0dbd2c0 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c99f30640 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dbd2c0 0x5646f0dbd6e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9c940cfe90 tx=0x7f9c940cf710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9b7f7fe640 1 -- 172.21.15.170:0/2732919106 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940c7ba0 con 0x5646f0dbd2c0 2024-03-28T16:50:17.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9b7f7fe640 1 -- 172.21.15.170:0/2732919106 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940c7d00 con 0x5646f0dbd2c0 2024-03-28T16:50:17.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5646f0dbde70 con 0x5646f0dbd2c0 2024-03-28T16:50:17.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9b7f7fe640 1 -- 172.21.15.170:0/2732919106 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940b33f0 con 0x5646f0dbd2c0 2024-03-28T16:50:17.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.048+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646f0dbe9c0 con 0x5646f0dbd2c0 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9b7f7fe640 1 -- 172.21.15.170:0/2732919106 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9c940b3570 con 0x5646f0dbd2c0 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9b7f7fe640 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b74034300 0x7f9b740367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9b7f7fe640 1 -- 172.21.15.170:0/2732919106 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9c940edba0 con 0x5646f0dbd2c0 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9972f640 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b74034300 0x7f9b740367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 Processor -- start 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 -- start start 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f0b44790 con 0x5646f0dca260 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9972f640 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b74034300 0x7f9b740367c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5646f0da7e50 tx=0x7f9c9000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55664/0 (socket says 172.21.15.170:55664) 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c99f30640 1 -- 172.21.15.170:0/93668208 learned_addr learned my addr 172.21.15.170:0/93668208 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c99f30640 1 -- 172.21.15.170:0/93668208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f0dcac10 con 0x5646f0dca260 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c99f30640 1 --2- 172.21.15.170:0/93668208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dca640 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9c940e1010 tx=0x7f9c940eb640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f560fb5196668 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9b7d7fa640 1 -- 172.21.15.170:0/93668208 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940fb070 con 0x5646f0dca260 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9b7d7fa640 1 -- 172.21.15.170:0/93668208 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940db430 con 0x5646f0dca260 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9b7d7fa640 1 -- 172.21.15.170:0/93668208 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940d0400 con 0x5646f0dca260 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 -- 172.21.15.170:0/93668208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 msgr2=0x5646f0dca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 --2- 172.21.15.170:0/93668208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dca640 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9c940e1010 tx=0x7f9c940eb640 comp rx=0 tx=0).stop 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 -- 172.21.15.170:0/93668208 shutdown_connections 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 --2- 172.21.15.170:0/93668208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dca640 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 -- 172.21.15.170:0/93668208 >> 172.21.15.170:0/93668208 conn(0x5646f0dc65a0 msgr2=0x5646f0dc4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 -- 172.21.15.170:0/93668208 shutdown_connections 2024-03-28T16:50:17.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.052+0000 7f9c9c415980 1 -- 172.21.15.170:0/93668208 wait complete. 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 Processor -- start 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 -- start start 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dd1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646f0dd15d0 con 0x5646f0dca260 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dd1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c99f30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dd1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55676/0 (socket says 172.21.15.170:55676) 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 learned_addr learned my addr 172.21.15.170:0/3618130895 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646f0dbe9c0 con 0x5646f0dca260 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c99f30640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dd1090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9c94001ec0 tx=0x7f9c940eb6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9b6b7fe640 1 -- 172.21.15.170:0/3618130895 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c94107040 con 0x5646f0dca260 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5646f0dbe9c0 con 0x5646f0dca260 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9b6b7fe640 1 -- 172.21.15.170:0/3618130895 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940d0990 con 0x5646f0dca260 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9b6b7fe640 1 -- 172.21.15.170:0/3618130895 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c940d8710 con 0x5646f0dca260 2024-03-28T16:50:17.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646f0dbe9c0 con 0x5646f0dca260 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9b6b7fe640 1 -- 172.21.15.170:0/3618130895 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9c940d8870 con 0x5646f0dca260 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9b6b7fe640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b640342b0 0x7f9b64036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9b6b7fe640 1 -- 172.21.15.170:0/3618130895 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9c9410cbd0 con 0x5646f0dca260 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9972f640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b640342b0 0x7f9b64036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0ddc130 0x5646f0dde5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dded40 con 0x5646f0ddc130 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.056+0000 7f9c9a731640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0ddc130 0x5646f0dde5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9972f640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b640342b0 0x7f9b64036770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c010 tx=0x7f9c90010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9a731640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0ddc130 0x5646f0dde5f0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 realm 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf110 con 0x5646f0ddc130 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddfcf0 con 0x5646f0ddc130 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0de06a0 0x5646f0de2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de30a0 con 0x5646f0de06a0 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c99f30640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0de06a0 0x5646f0de2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c99f30640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0de06a0 0x5646f0de2a90 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:17.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 20 searching for the correct realm 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5646f0de52b0 0x5646f0de7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de7e40 con 0x5646f0de52b0 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9972f640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5646f0de52b0 0x5646f0de7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.060+0000 7f9c9972f640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5646f0de52b0 0x5646f0de7770 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9972f640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9c9001fc60 0x7f9c9003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de7e40 con 0x7f9c9001fc60 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9c9001fc60 0x7f9c9003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9c9001fc60 0x7f9c9003a3c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x7f9c9001fc60 2024-03-28T16:50:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.064+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x7f9c9001fc60 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.068+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x7f9c9001fc60 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x7f9c9001fc60 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.072+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de52b0 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0de06a0 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x5646f0ddc130 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ddf990 con 0x7f9c9001fc60 2024-03-28T16:50:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:17.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de36b0 con 0x5646f0ddc130 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de36b0 con 0x5646f0ddc130 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de3680 con 0x7f9c9001fc60 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:17.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 zone default found 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 4 Realm: () 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 zonegroup default 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de3f00 con 0x5646f0de06a0 2024-03-28T16:50:17.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de4360 con 0x5646f0de52b0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de9e30 con 0x5646f0de52b0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 started sync module instance, tier type = 2024-03-28T16:50:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.076+0000 7f9c9c415980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de9540 con 0x7f9c9001fc60 2024-03-28T16:50:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94862688817312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de98f0 con 0x7f9c9001fc60 2024-03-28T16:50:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0deab90 con 0x5646f0ddc130 2024-03-28T16:50:17.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94862688817312] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94862688821840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de98f0 con 0x5646f0ddc130 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.080+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0deb230 con 0x5646f0de52b0 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.084+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94862688821840] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.084+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.084+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94862688824624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0deb5e0 con 0x5646f0de52b0 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.084+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0debf50 con 0x7f9c9001fc60 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.084+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.084+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94862688828352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dec640 con 0x7f9c9001fc60 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.084+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ded110 con 0x5646f0ddc130 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94862688824624] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.087 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94862688828352] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94862688832496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ded0d0 con 0x5646f0ddc130 2024-03-28T16:50:17.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dea7e0 con 0x5646f0de06a0 2024-03-28T16:50:17.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94862688832496] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94862688829600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0ded960 con 0x5646f0de06a0 2024-03-28T16:50:17.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.088+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dee160 con 0x5646f0de06a0 2024-03-28T16:50:17.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.092+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.092+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94862688837312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dee960 con 0x5646f0de06a0 2024-03-28T16:50:17.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.092+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0def430 con 0x5646f0de06a0 2024-03-28T16:50:17.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.092+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94862688829600] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94862688837312] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94862688841488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dee3c0 con 0x5646f0de06a0 2024-03-28T16:50:17.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=0 2024-03-28T16:50:17.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=1 2024-03-28T16:50:17.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=2 2024-03-28T16:50:17.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=3 2024-03-28T16:50:17.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=4 2024-03-28T16:50:17.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=5 2024-03-28T16:50:17.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=6 2024-03-28T16:50:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94862688841488] v31'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 20 add_watcher() i=7 2024-03-28T16:50:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:17.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0df2420 con 0x5646f0ddc130 2024-03-28T16:50:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.096+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94862688834672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0def890 con 0x5646f0ddc130 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94862688834672] v31'210 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9b4b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 5 note: GC not initialized 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9b4a7fc640 20 reqs_thread_entry: start 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 Filter name: none 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9abbfff640 20 reqs_thread_entry: start 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e14310 con 0x5646f0de06a0 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e12ae0 con 0x5646f0ddc130 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:17.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-03-28T16:50:17.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-03-28T16:50:17.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.100+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e12ec0 con 0x5646f0de06a0 2024-03-28T16:50:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~346] v31'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c9c415980 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-03-28T16:50:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c9c415980 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:50:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c9c415980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-03-28T16:50:17.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94862688992624 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e17870 con 0x5646f0de06a0 2024-03-28T16:50:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== watch-notify(notify (1) cookie 94862688841488 notify 133143986209 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x5646f0dcac10 con 0x5646f0de06a0 2024-03-28T16:50:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9b7d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986209 cookie 94862688841488 notifier 4718 bl.length()=522 2024-03-28T16:50:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(65 notify.7 [notify cookie 94862688992624 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9c940cd070 con 0x5646f0de06a0 2024-03-28T16:50:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9b7d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-03-28T16:50:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9b7d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:50:17.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9b7d7fa640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f9b5c003ea0 con 0x5646f0de06a0 2024-03-28T16:50:17.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== watch-notify(notify_complete (2) cookie 94862688992624 notify 133143986209 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f9c9412d020 con 0x5646f0de06a0 2024-03-28T16:50:17.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c9412f040 con 0x5646f0de06a0 2024-03-28T16:50:17.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.104+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e13d70 con 0x5646f0ddc130 2024-03-28T16:50:17.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c9c415980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-28T16:50:17.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c9c415980 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:17.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c9c415980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-03-28T16:50:17.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94862688992624 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e17840 con 0x5646f0de06a0 2024-03-28T16:50:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== watch-notify(notify (1) cookie 94862688829600 notify 133143986210 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7f9c9412d020 con 0x5646f0de06a0 2024-03-28T16:50:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9b7d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986210 cookie 94862688829600 notifier 4718 bl.length()=177 2024-03-28T16:50:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9b7d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-28T16:50:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9b7d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9b7d7fa640 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f9b5c003ea0 con 0x5646f0de06a0 2024-03-28T16:50:17.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(68 notify.5 [notify cookie 94862688992624 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f9c9412f040 con 0x5646f0de06a0 2024-03-28T16:50:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== watch-notify(notify_complete (2) cookie 94862688992624 notify 133143986210 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5646f0dbdc20 con 0x5646f0de06a0 2024-03-28T16:50:17.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c9412f040 con 0x5646f0de06a0 2024-03-28T16:50:17.109 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"bar","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bar","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:50:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9b4b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:17.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.108+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94862688834672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e13570 con 0x5646f0ddc130 2024-03-28T16:50:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94862688834672] v31'211 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94862688817312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e13570 con 0x7f9c9001fc60 2024-03-28T16:50:17.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94862688817312] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9c415980 20 remove_watcher() i=0 2024-03-28T16:50:17.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9c415980 2 removed watcher, disabling cache 2024-03-28T16:50:17.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94862688821840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e13570 con 0x5646f0ddc130 2024-03-28T16:50:17.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94862688821840] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9c415980 20 remove_watcher() i=1 2024-03-28T16:50:17.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.112+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94862688824624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0e13570 con 0x5646f0de52b0 2024-03-28T16:50:17.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.116+0000 7f9c9972f640 1 -- 172.21.15.170:0/3618130895 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94862688824624] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c90033040 con 0x5646f0de52b0 2024-03-28T16:50:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.116+0000 7f9c9c415980 20 remove_watcher() i=2 2024-03-28T16:50:17.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.116+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94862688828352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de90a0 con 0x7f9c9001fc60 2024-03-28T16:50:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.116+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94862688828352] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x7f9c9001fc60 2024-03-28T16:50:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.116+0000 7f9c9c415980 20 remove_watcher() i=3 2024-03-28T16:50:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.116+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94862688832496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dea250 con 0x5646f0ddc130 2024-03-28T16:50:17.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.120+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94862688832496] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c8c00a4d0 con 0x5646f0ddc130 2024-03-28T16:50:17.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.120+0000 7f9c9c415980 20 remove_watcher() i=4 2024-03-28T16:50:17.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.120+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94862688829600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dead30 con 0x5646f0de06a0 2024-03-28T16:50:17.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.120+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94862688829600] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c9412f040 con 0x5646f0de06a0 2024-03-28T16:50:17.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.120+0000 7f9c9c415980 20 remove_watcher() i=5 2024-03-28T16:50:17.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.120+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94862688837312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0de90a0 con 0x5646f0de06a0 2024-03-28T16:50:17.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.124+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94862688837312] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c9412f040 con 0x5646f0de06a0 2024-03-28T16:50:17.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.124+0000 7f9c9c415980 20 remove_watcher() i=6 2024-03-28T16:50:17.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.124+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94862688841488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5646f0dea250 con 0x5646f0de06a0 2024-03-28T16:50:17.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.124+0000 7f9c99f30640 1 -- 172.21.15.170:0/3618130895 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94862688841488] v31'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9c9412f040 con 0x5646f0de06a0 2024-03-28T16:50:17.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.124+0000 7f9c9c415980 20 remove_watcher() i=7 2024-03-28T16:50:17.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.124+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0de06a0 msgr2=0x5646f0de2a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0de06a0 0x5646f0de2a90 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0ddc130 msgr2=0x5646f0dde5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0ddc130 0x5646f0dde5f0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9c9001fc60 msgr2=0x7f9c9003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9c9001fc60 0x7f9c9003a3c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5646f0de52b0 msgr2=0x5646f0de7770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5646f0de52b0 0x5646f0de7770 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b640342b0 msgr2=0x7f9b64036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b640342b0 0x7f9b64036770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c010 tx=0x7f9c90010000 comp rx=0 tx=0).stop 2024-03-28T16:50:17.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 msgr2=0x5646f0dd1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9a731640 1 -- 172.21.15.170:0/3618130895 reap_dead start 2024-03-28T16:50:17.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dd1090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9c94001ec0 tx=0x7f9c940eb6b0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 shutdown_connections 2024-03-28T16:50:17.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/3618130895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dca260 0x5646f0dd1090 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 >> 172.21.15.170:0/3618130895 conn(0x5646f0dc65a0 msgr2=0x5646f0dc3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 shutdown_connections 2024-03-28T16:50:17.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/3618130895 wait complete. 2024-03-28T16:50:17.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b74034300 msgr2=0x7f9b740367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b74034300 0x7f9b740367c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x5646f0da7e50 tx=0x7f9c9000f000 comp rx=0 tx=0).stop 2024-03-28T16:50:17.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dbd2c0 msgr2=0x5646f0dbd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dbd2c0 0x5646f0dbd6e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9c940cfe90 tx=0x7f9c940cf710 comp rx=0 tx=0).stop 2024-03-28T16:50:17.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 shutdown_connections 2024-03-28T16:50:17.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b74034300 0x7f9b740367c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/2732919106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0dbd2c0 0x5646f0dbd6e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 >> 172.21.15.170:0/2732919106 conn(0x5646f0db20d0 msgr2=0x5646f0db2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 shutdown_connections 2024-03-28T16:50:17.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/2732919106 wait complete. 2024-03-28T16:50:17.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0da8cb0 msgr2=0x5646f0dab0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5646f0da8cb0 0x5646f0dab0a0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0da4300 msgr2=0x5646f0da66f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5646f0da4300 0x5646f0da66f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5646f0dac950 msgr2=0x5646f0daed40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5646f0dac950 0x5646f0daed40 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b800342b0 msgr2=0x7f9b80036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9b800342b0 0x7f9b80036770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000f70 tx=0x7f9c90001380 comp rx=0 tx=0).stop 2024-03-28T16:50:17.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0caea90 msgr2=0x5646f0d9f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 --2- 172.21.15.170:0/375742683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5646f0caea90 0x5646f0d9f780 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c10 tx=0x7f9c940180e0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9a731640 1 -- 172.21.15.170:0/375742683 reap_dead start 2024-03-28T16:50:17.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 shutdown_connections 2024-03-28T16:50:17.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 >> 172.21.15.170:0/375742683 conn(0x5646f0c21890 msgr2=0x5646f0d92da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 shutdown_connections 2024-03-28T16:50:17.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.128+0000 7f9c9c415980 1 -- 172.21.15.170:0/375742683 wait complete. 2024-03-28T16:50:17.139 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:50:17.155 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:50:17.162 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-03-28T16:50:17.162 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-03-28T16:50:17.162 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-03-28T16:50:17.162 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-03-28T16:50:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:17.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.216+0000 7fc06a3cc980 1 Processor -- start 2024-03-28T16:50:17.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.216+0000 7fc06a3cc980 1 -- start start 2024-03-28T16:50:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.216+0000 7fc06a3cc980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55740bdec080 0x55740be7ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.216+0000 7fc06a3cc980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55740bdc44c0 con 0x55740be7b140 2024-03-28T16:50:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.216+0000 7fc06a3cc980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55740bd8b0f0 con 0x55740bdec080 2024-03-28T16:50:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.220+0000 7fc067ee7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55740bdec080 0x55740be7ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.220+0000 7fc067ee7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55740bdec080 0x55740be7ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55678/0 (socket says 172.21.15.170:55678) 2024-03-28T16:50:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.220+0000 7fc067ee7640 1 -- 172.21.15.170:0/1712266951 learned_addr learned my addr 172.21.15.170:0/1712266951 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.220+0000 7fc0676e6640 1 --1- 172.21.15.170:0/1712266951 >> v1:172.21.15.119:6789/0 conn(0x55740be7b140 0x55740be7d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:46102/0 (socket says 172.21.15.170:46102) 2024-03-28T16:50:17.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.220+0000 7fc067ee7640 1 -- 172.21.15.170:0/1712266951 >> v1:172.21.15.119:6789/0 conn(0x55740be7b140 legacy=0x55740be7d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.220+0000 7fc066ee5640 1 -- 172.21.15.170:0/1712266951 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242231218 0 0) 0x55740bdc44c0 con 0x55740be7b140 2024-03-28T16:50:17.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.220+0000 7fc067ee7640 1 -- 172.21.15.170:0/1712266951 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740be7fcf0 con 0x55740bdec080 2024-03-28T16:50:17.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc067ee7640 1 --2- 172.21.15.170:0/1712266951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be7ac00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc05800a5c0 tx=0x7fc05801b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f8953dec3d4cf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc066ee5640 1 -- 172.21.15.170:0/1712266951 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580aa470 con 0x55740bdec080 2024-03-28T16:50:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc066ee5640 1 -- 172.21.15.170:0/1712266951 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580aa5d0 con 0x55740bdec080 2024-03-28T16:50:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc066ee5640 1 -- 172.21.15.170:0/1712266951 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580aa890 con 0x55740bdec080 2024-03-28T16:50:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1712266951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 msgr2=0x55740be7ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/1712266951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be7ac00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc05800a5c0 tx=0x7fc05801b340 comp rx=0 tx=0).stop 2024-03-28T16:50:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1712266951 shutdown_connections 2024-03-28T16:50:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/1712266951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be7ac00 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1712266951 >> 172.21.15.170:0/1712266951 conn(0x55740be726e0 msgr2=0x55740be74b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1712266951 shutdown_connections 2024-03-28T16:50:17.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.224+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1712266951 wait complete. 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.236+0000 7fc06a3cc980 1 Processor -- start 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.236+0000 7fc06a3cc980 1 -- start start 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.236+0000 7fc06a3cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740bf61c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.236+0000 7fc06a3cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740bd88620 con 0x55740bdec080 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.236+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740bf61c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.236+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740bf61c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55684/0 (socket says 172.21.15.170:55684) 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.236+0000 7fc067ee7640 1 -- 172.21.15.170:0/3518026939 learned_addr learned my addr 172.21.15.170:0/3518026939 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc067ee7640 1 -- 172.21.15.170:0/3518026939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058005df0 con 0x55740bdec080 2024-03-28T16:50:17.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc067ee7640 1 --2- 172.21.15.170:0/3518026939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740bf61c60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc0580ac040 tx=0x7fc058003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f463a979248bb4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fbf55e0b640 1 -- 172.21.15.170:0/3518026939 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580aa9b0 con 0x55740bdec080 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fbf55e0b640 1 -- 172.21.15.170:0/3518026939 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580aab30 con 0x55740bdec080 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fbf55e0b640 1 -- 172.21.15.170:0/3518026939 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580aae10 con 0x55740bdec080 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3518026939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 msgr2=0x55740bf61c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/3518026939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740bf61c60 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc0580ac040 tx=0x7fc058003030 comp rx=0 tx=0).stop 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3518026939 shutdown_connections 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/3518026939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740bf61c60 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3518026939 >> 172.21.15.170:0/3518026939 conn(0x55740be726e0 msgr2=0x55740bf5d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3518026939 shutdown_connections 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3518026939 wait complete. 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 Processor -- start 2024-03-28T16:50:17.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- start start 2024-03-28T16:50:17.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be78310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740bd83e00 con 0x55740bdec080 2024-03-28T16:50:17.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be78310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be78310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55698/0 (socket says 172.21.15.170:55698) 2024-03-28T16:50:17.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc067ee7640 1 -- 172.21.15.170:0/381926293 learned_addr learned my addr 172.21.15.170:0/381926293 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc067ee7640 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740be7fcf0 con 0x55740bdec080 2024-03-28T16:50:17.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc067ee7640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be78310 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc058005940 tx=0x7fc0580180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fbf4ffff640 1 -- 172.21.15.170:0/381926293 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580c6750 con 0x55740bdec080 2024-03-28T16:50:17.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fbf4ffff640 1 -- 172.21.15.170:0/381926293 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580c68b0 con 0x55740bdec080 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55740be78850 con 0x55740bdec080 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fbf4ffff640 1 -- 172.21.15.170:0/381926293 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580c6b70 con 0x55740bdec080 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.240+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55740be76d40 con 0x55740bdec080 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fbf4ffff640 1 -- 172.21.15.170:0/381926293 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc0580c6cd0 con 0x55740bdec080 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fbf4ffff640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf500342b0 0x7fbf50036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fbf4ffff640 1 -- 172.21.15.170:0/381926293 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc0580e4330 con 0x55740bdec080 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc0676e6640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf500342b0 0x7fbf50036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740be7b140 0x55740bf718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bf71e30 con 0x55740be7b140 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc0686e8640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740be7b140 0x55740bf718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc0676e6640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf500342b0 0x7fbf50036770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c10 tx=0x7fc05c0013c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc0686e8640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740be7b140 0x55740bf718f0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc0686e8640 1 -- 172.21.15.170:0/381926293 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740be7b140 2024-03-28T16:50:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bf72190 con 0x55740be7b140 2024-03-28T16:50:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc0686e8640 1 -- 172.21.15.170:0/381926293 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740be7b140 2024-03-28T16:50:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bf73a60 0x55740bf75e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bf76610 con 0x55740bf73a60 2024-03-28T16:50:17.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc067ee7640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bf73a60 0x55740bf75e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.244+0000 7fc067ee7640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bf73a60 0x55740bf75e80 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc067ee7640 1 -- 172.21.15.170:0/381926293 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fc0580b0020 con 0x55740bf73a60 2024-03-28T16:50:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bf72990 con 0x55740be7b140 2024-03-28T16:50:17.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc0686e8640 1 -- 172.21.15.170:0/381926293 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740be7b140 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55740bf77a30 0x55740bf79e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bf7a4c0 con 0x55740bf77a30 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc0676e6640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55740bf77a30 0x55740bf79e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc0676e6640 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55740bf77a30 0x55740bf79e20 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc0676e6640 1 -- 172.21.15.170:0/381926293 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fc05c00c040 con 0x55740bf77a30 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 Processor -- start 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 -- start start 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf883f0 0x55740bf88810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740bd1ce30 con 0x55740bf883f0 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf883f0 0x55740bf88810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf883f0 0x55740bf88810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55706/0 (socket says 172.21.15.170:55706) 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc067ee7640 1 -- 172.21.15.170:0/3981757835 learned_addr learned my addr 172.21.15.170:0/3981757835 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc067ee7640 1 -- 172.21.15.170:0/3981757835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740bf88d50 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc067ee7640 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf883f0 0x55740bf88810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc05801b380 tx=0x7fc0580bdae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fbf4d7fa640 1 -- 172.21.15.170:0/3981757835 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580cf590 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fbf4d7fa640 1 -- 172.21.15.170:0/3981757835 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580e2b20 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55740bf88fa0 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fbf4d7fa640 1 -- 172.21.15.170:0/3981757835 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580e2dc0 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.248+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55740bf89af0 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fbf4d7fa640 1 -- 172.21.15.170:0/3981757835 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc0580aca80 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fbf4d7fa640 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf400342b0 0x7fbf40036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fbf4d7fa640 1 -- 172.21.15.170:0/3981757835 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc0580edba0 con 0x55740bf883f0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc0676e6640 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf400342b0 0x7fbf40036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 Processor -- start 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- start start 2024-03-28T16:50:17.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc0676e6640 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf400342b0 0x7fbf40036770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00bfe0 tx=0x7fc05c006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf95740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740bd10790 con 0x55740bf95320 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf95740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf95740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55720/0 (socket says 172.21.15.170:55720) 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc067ee7640 1 -- 172.21.15.170:0/1086872418 learned_addr learned my addr 172.21.15.170:0/1086872418 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc067ee7640 1 -- 172.21.15.170:0/1086872418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740bf95d10 con 0x55740bf95320 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc067ee7640 1 --2- 172.21.15.170:0/1086872418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf95740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc0580cfd20 tx=0x7fc0580ede80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cbce5b7785d454 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fbf46ffd640 1 -- 172.21.15.170:0/1086872418 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580fc070 con 0x55740bf95320 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fbf46ffd640 1 -- 172.21.15.170:0/1086872418 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580b3520 con 0x55740bf95320 2024-03-28T16:50:17.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fbf46ffd640 1 -- 172.21.15.170:0/1086872418 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580bc6e0 con 0x55740bf95320 2024-03-28T16:50:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1086872418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 msgr2=0x55740bf95740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/1086872418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf95740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc0580cfd20 tx=0x7fc0580ede80 comp rx=0 tx=0).stop 2024-03-28T16:50:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1086872418 shutdown_connections 2024-03-28T16:50:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/1086872418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf95740 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1086872418 >> 172.21.15.170:0/1086872418 conn(0x55740bf916b0 msgr2=0x55740bf8f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1086872418 shutdown_connections 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- 172.21.15.170:0/1086872418 wait complete. 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 Processor -- start 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- start start 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf9c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.252+0000 7fc06a3cc980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55740bf9c700 con 0x55740bf95320 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf9c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc067ee7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf9c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:55736/0 (socket says 172.21.15.170:55736) 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 learned_addr learned my addr 172.21.15.170:0/2788609699 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55740bf89af0 con 0x55740bf95320 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc067ee7640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf9c1c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc0580cfd20 tx=0x7fc0580b6f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fbf457fa640 1 -- 172.21.15.170:0/2788609699 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc058108070 con 0x55740bf95320 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55740bf89af0 con 0x55740bf95320 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fbf457fa640 1 -- 172.21.15.170:0/2788609699 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580b7a90 con 0x55740bf95320 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fbf457fa640 1 -- 172.21.15.170:0/2788609699 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0580b5ea0 con 0x55740bf95320 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55740bf89af0 con 0x55740bf95320 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fbf457fa640 1 -- 172.21.15.170:0/2788609699 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fc0580e4610 con 0x55740bf95320 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fbf457fa640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf300342b0 0x7fbf30036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fbf457fa640 1 -- 172.21.15.170:0/2788609699 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc058116bb0 con 0x55740bf95320 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc0676e6640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf300342b0 0x7fbf30036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740bfa7490 0x55740bfa9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaa0a0 con 0x55740bfa7490 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc0686e8640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740bfa7490 0x55740bfa9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc0676e6640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf300342b0 0x7fbf30036770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55740bf769a0 tx=0x7fc05c023040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc0686e8640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740bfa7490 0x55740bfa9950 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 20 realm 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.256+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaa470 con 0x55740bfa7490 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfab050 con 0x55740bfa7490 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bfaba00 0x55740bfaddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfae400 con 0x55740bfaba00 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc067ee7640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bfaba00 0x55740bfaddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc067ee7640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bfaba00 0x55740bfaddf0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 20 searching for the correct realm 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55740bfb05e0 0x55740bfb2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb3170 con 0x55740bfb05e0 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0676e6640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55740bfb05e0 0x55740bfb2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0676e6640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55740bfb05e0 0x55740bfb2aa0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0676e6640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc05c01f430 0x7fc05c01f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb3170 con 0x7fc05c01f430 2024-03-28T16:50:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0686e8640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc05c01f430 0x7fc05c01f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.260+0000 7fc0686e8640 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc05c01f430 0x7fc05c01f850 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x7fc05c01f430 2024-03-28T16:50:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.264+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x7fc05c01f430 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.268+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x7fc05c01f430 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x7fc05c01f430 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfb05e0 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfaba00 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x55740bfa7490 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.272+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaacf0 con 0x7fc05c01f430 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfae9e0 con 0x55740bfa7490 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfae9e0 con 0x55740bfa7490 2024-03-28T16:50:17.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfae9b0 con 0x7fc05c01f430 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 zone default found 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 4 Realm: () 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 zonegroup default 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaf230 con 0x55740bfaba00 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfaf630 con 0x55740bfb05e0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb50e0 con 0x55740bfb05e0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 started sync module instance, tier type = 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb4870 con 0x7fc05c01f430 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93956905583568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb4c20 con 0x7fc05c01f430 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.276+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb5e40 con 0x55740bfa7490 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.280+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93956905583568] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.280+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.280+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93956905587968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb5e20 con 0x55740bfa7490 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.280+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb68e0 con 0x55740bfb05e0 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.280+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93956905587968] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93956905591712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb6c90 con 0x55740bfb05e0 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb7600 con 0x7fc05c01f430 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93956905595504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb7cf0 con 0x7fc05c01f430 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb87c0 con 0x55740bfa7490 2024-03-28T16:50:17.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93956905591712] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93956905595504] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93956905599648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb8780 con 0x55740bfa7490 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.284+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb78c0 con 0x55740bfaba00 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93956905599648] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93956905589024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb9010 con 0x55740bfaba00 2024-03-28T16:50:17.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb9810 con 0x55740bfaba00 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93956905589024] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93956905601904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfba010 con 0x55740bfaba00 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.288+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfba980 con 0x55740bfaba00 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93956905608896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfbb0e0 con 0x55740bfaba00 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=0 2024-03-28T16:50:17.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=1 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=2 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=3 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=4 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=5 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93956905601904] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=6 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93956905608896] v31'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 20 add_watcher() i=7 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.292+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb9b70 con 0x55740bfa7490 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93956905610112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfb9b70 con 0x55740bfa7490 2024-03-28T16:50:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93956905610112] v31'212 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fbf1e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 5 note: GC not initialized 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fbf1d7e2640 20 reqs_thread_entry: start 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 20 Filter name: none 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fbf167fc640 20 reqs_thread_entry: start 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.296+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0050 con 0x55740bfaba00 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 bar [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe3d20 con 0x7fc05c01f430 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(63 bud [create,call,writefull 0~181] v31'1 uv1 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 93956905773776 in=369b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4980 con 0x55740bfaba00 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== watch-notify(notify (1) cookie 93956905589024 notify 133143986211 ret 0) v3 ==== 399+0+0 (crc 0 0 0) 0x55740bf95d10 con 0x55740bfaba00 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fbf46ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986211 cookie 93956905589024 notifier 4725 bl.length()=357 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fbf46ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fbf46ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fbf46ffd640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fbf2c003ea0 con 0x55740bfaba00 2024-03-28T16:50:17.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(64 notify.5 [notify cookie 93956905773776 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== watch-notify(notify_complete (2) cookie 93956905773776 notify 133143986211 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55740bf88d50 con 0x55740bfaba00 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4900 con 0x55740bfa7490 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 : miss 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 8.19 8:9e623195:root::.bucket.meta.myfoo6%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0ef0 con 0x55740bfa7490 2024-03-28T16:50:17.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(67 .bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 333+0+635 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 20 rados_obj.operate() r=0 bl.length=353 2024-03-28T16:50:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 info.flags=0x17 2024-03-28T16:50:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 to cache LRU end 2024-03-28T16:50:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:50:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 2024-03-28T16:50:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.300+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 8.19 8:9e623195:root::.bucket.meta.myfoo6%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.9:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfeb550 con 0x55740bfa7490 2024-03-28T16:50:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 .bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 [delete,create,call,call,writefull 0~353,setxattr (117)] v31'4 uv4 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 info.flags=0x17 2024-03-28T16:50:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 to cache LRU end 2024-03-28T16:50:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-03-28T16:50:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9, ofs0, ns] 2024-03-28T16:50:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify cookie 93956905767728 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe7230 con 0x55740bfb05e0 2024-03-28T16:50:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== watch-notify(notify (1) cookie 93956905591712 notify 133143986189 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x7fc05c046040 con 0x55740bfb05e0 2024-03-28T16:50:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fbf467fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986189 cookie 93956905591712 notifier 4725 bl.length()=721 2024-03-28T16:50:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fbf467fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 info.flags=0x17 2024-03-28T16:50:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fbf467fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 to cache LRU end 2024-03-28T16:50:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fbf467fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-03-28T16:50:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(69 notify.2 [notify cookie 93956905767728 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fbf467fc640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fbe90003530 con 0x55740bfb05e0 2024-03-28T16:50:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(70 notify.2 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== watch-notify(notify_complete (2) cookie 93956905767728 notify 133143986189 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fc05c02e620 con 0x55740bfb05e0 2024-03-28T16:50:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 20 handle_bi_update(): bucket=:myfoo6[b88c2392-6ef8-4660-b080-19480b748fc0.4221.9]): orig_sources= new_sources= 2024-03-28T16:50:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 20 handle_bi_update(): bucket=:myfoo6[b88c2392-6ef8-4660-b080-19480b748fc0.4221.9]): potential sources added=[] removed=[] 2024-03-28T16:50:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 20 handle_bi_update(): bucket=:myfoo6[b88c2392-6ef8-4660-b080-19480b748fc0.4221.9]): orig_dests= new_dests= 2024-03-28T16:50:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 20 handle_bi_update(): bucket=:myfoo6[b88c2392-6ef8-4660-b080-19480b748fc0.4221.9]): potential dests added=[] removed=[] 2024-03-28T16:50:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:71 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe5d80 con 0x55740bfa7490 2024-03-28T16:50:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(71 bud.buckets [call] v31'5 uv0 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-03-28T16:50:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 20 rados->read ofs=0 len=0 2024-03-28T16:50:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4d70 con 0x55740bfa7490 2024-03-28T16:50:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(72 myfoo6 [call out=48b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 234+0+289 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 20 rados_obj.operate() r=0 bl.length=166 2024-03-28T16:50:17.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x13 2024-03-28T16:50:17.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-03-28T16:50:17.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:17.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 0 bucket entry point user mismatch, can't unlink bucket: bar != bud 2024-03-28T16:50:17.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 4.2e 4:762cdbfd:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4d70 con 0x55740bfb05e0 2024-03-28T16:50:17.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(73 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.304+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:74 4.20 4:07332749:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4d70 con 0x55740bfa7490 2024-03-28T16:50:17.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(74 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.1 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:75 4.25 4:a45194ce:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4d70 con 0x55740bfb05e0 2024-03-28T16:50:17.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(75 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.2 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 4.9 4:93e87b1b:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4d70 con 0x55740bfa7490 2024-03-28T16:50:17.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(76 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.3 [call out=85b] v0'0 uv17 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 4.11 4:89771fdb:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe4d70 con 0x55740bfaba00 2024-03-28T16:50:17.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(77 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.320 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 4.1a 4:599930eb:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe6240 con 0x7fc05c01f430 2024-03-28T16:50:17.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(78 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 4.3a 4:5cb22805:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe6240 con 0x55740bfb05e0 2024-03-28T16:50:17.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(79 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:80 4.1d 4:bb1d92dd:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe6240 con 0x55740bfb05e0 2024-03-28T16:50:17.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(80 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:81 4.11 4:8820f779:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe6240 con 0x55740bfaba00 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(81 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.8 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.308+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:82 4.9 4:93231880:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe6240 con 0x55740bfa7490 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(82 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.9 [call out=377b] v0'0 uv20 ondisk = 0) v8 ==== 194+0+377 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 10 cls_bucket_list_unordered: got thirteen 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 4.1b 4:d82277df:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe6240 con 0x7fc05c01f430 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(83 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.10 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 get_obj_state: octx=0x7fc06000aac0 obj=myfoo6:thirteen state=0x55740bfe5418 s->prefetch_data=0 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:84 3.3c 3:3d712630:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.9_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe6b50 con 0x7fc05c01f430 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(84 b88c2392-6ef8-4660-b080-19480b748fc0.4221.9_thirteen [getxattrs out=854b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 238+0+870 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 10 manifest: total_size = 8 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.5540729297799455356 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.acl 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.content_type 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.etag 2024-03-28T16:50:17.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.idtag 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.manifest 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.source_zone 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 get_obj_state: octx=0x7fc06000aac0 obj=myfoo6:thirteen state=0x55740bfe5418 s->prefetch_data=0 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 : hit (requested=0x17, cached=0x17) 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:b88c2392-6ef8-4660-b080-19480b748fc0.4221.9 2024-03-28T16:50:17.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.9 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:85 4.9 4:93231880:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfeaa60 con 0x55740bfa7490 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(85 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.9 [stat,call,call] v31'21 uv21 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.312+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:86 3.3c 3:3d712630:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.9_thirteen:head [cmpxattr user.rgw.idtag (62) op 1 mode 1 in=76b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfeaa60 con 0x7fc05c01f430 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(86 b88c2392-6ef8-4660-b080-19480b748fc0.4221.9_thirteen [cmpxattr (62) op 1 mode 1,setxattr (117),setxattr (33)] v31'2 uv2 ondisk = 0) v8 ==== 280+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:87 4.9 4:93231880:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=323b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfeaa60 con 0x55740bfa7490 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:1 objects processed in :myfoo6[b88c2392-6ef8-4660-b080-19480b748fc0.4221.9]). Next marker thirteen 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:88 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfeaa60 con 0x7fc05c01f430 2024-03-28T16:50:17.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(88 bud [delete,create,call,writefull 0~346] v31'2 uv2 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc06a3cc980 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc06a3cc980 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc06a3cc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:89 7.3 7:c609908c:::notify.5:head [notify cookie 93956905789072 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfeaa60 con 0x55740bfaba00 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(87 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.9.9 [stat,call,call] v31'22 uv22 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fbf46ffd640 20 handle_completion(): completion ok for obj=thirteen 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== watch-notify(notify (1) cookie 93956905589024 notify 133143986212 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x7fc058137040 con 0x55740bfaba00 2024-03-28T16:50:17.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fbf467fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986212 cookie 93956905589024 notifier 4725 bl.length()=522 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fbf467fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fbf467fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fbf467fc640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fbe900030d0 con 0x55740bfaba00 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(89 notify.5 [notify cookie 93956905789072 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(90 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== watch-notify(notify_complete (2) cookie 93956905789072 notify 133143986212 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55740bf88d50 con 0x55740bfaba00 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.316+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:91 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0850 con 0x55740bfa7490 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(91 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v31'2 uv2 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:17.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 7.3 7:c609908c:::notify.5:head [notify cookie 93956905789072 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe50d0 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== watch-notify(notify (1) cookie 93956905589024 notify 133143986213 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55740bf88d50 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fbf46ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986213 cookie 93956905589024 notifier 4725 bl.length()=177 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fbf46ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fbf46ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(92 notify.5 [notify cookie 93956905789072 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fbf46ffd640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:93 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fbf2c003ea0 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== watch-notify(notify_complete (2) cookie 93956905789072 notify 133143986213 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55740be76d40 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(93 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 10 removing user index: bar 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 10 removing default.rgw.meta+users.uid+bar from cache 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:94 7.9 7:93e5b521:::notify.7:head [notify cookie 93956905789072 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe50d0 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== watch-notify(notify (1) cookie 93956905608896 notify 133143986214 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55740be76d40 con 0x55740bfaba00 2024-03-28T16:50:17.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fbf467fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986214 cookie 93956905608896 notifier 4725 bl.length()=152 2024-03-28T16:50:17.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fbf467fc640 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:95 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fbe900030d0 con 0x55740bfaba00 2024-03-28T16:50:17.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(94 notify.7 [notify cookie 93956905789072 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.320+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(95 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.324+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== watch-notify(notify_complete (2) cookie 93956905789072 notify 133143986214 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55740be78850 con 0x55740bfaba00 2024-03-28T16:50:17.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.324+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:96 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0850 con 0x55740bfaba00 2024-03-28T16:50:17.340 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.324+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(96 bar [delete] v31'4 uv4 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.340 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"bud","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bud","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:50:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.324+0000 7fbf1e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.324+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93956905610112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfd2130 con 0x55740bfa7490 2024-03-28T16:50:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(97 data_loggenerations_metadata [watch unwatch cookie 93956905610112] v31'213 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93956905583568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0aa0 con 0x7fc05c01f430 2024-03-28T16:50:17.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(98 notify.0 [watch unwatch cookie 93956905583568] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc06a3cc980 20 remove_watcher() i=0 2024-03-28T16:50:17.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc06a3cc980 2 removed watcher, disabling cache 2024-03-28T16:50:17.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93956905587968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe7230 con 0x55740bfa7490 2024-03-28T16:50:17.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(99 notify.1 [watch unwatch cookie 93956905587968] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc06a3cc980 20 remove_watcher() i=1 2024-03-28T16:50:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.328+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:100 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93956905591712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0aa0 con 0x55740bfb05e0 2024-03-28T16:50:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.332+0000 7fc0676e6640 1 -- 172.21.15.170:0/2788609699 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(100 notify.2 [watch unwatch cookie 93956905591712] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc05c033040 con 0x55740bfb05e0 2024-03-28T16:50:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.332+0000 7fc06a3cc980 20 remove_watcher() i=2 2024-03-28T16:50:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.332+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:101 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93956905595504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0aa0 con 0x7fc05c01f430 2024-03-28T16:50:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.332+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(101 notify.3 [watch unwatch cookie 93956905595504] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x7fc05c01f430 2024-03-28T16:50:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.332+0000 7fc06a3cc980 20 remove_watcher() i=3 2024-03-28T16:50:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.332+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:102 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93956905599648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0aa0 con 0x55740bfa7490 2024-03-28T16:50:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.336+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(102 notify.4 [watch unwatch cookie 93956905599648] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc06000a4d0 con 0x55740bfa7490 2024-03-28T16:50:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.336+0000 7fc06a3cc980 20 remove_watcher() i=4 2024-03-28T16:50:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.336+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93956905589024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0aa0 con 0x55740bfaba00 2024-03-28T16:50:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.336+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(103 notify.5 [watch unwatch cookie 93956905589024] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.336+0000 7fc06a3cc980 20 remove_watcher() i=5 2024-03-28T16:50:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.336+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:104 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93956905601904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0aa0 con 0x55740bfaba00 2024-03-28T16:50:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(104 notify.6 [watch unwatch cookie 93956905601904] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 20 remove_watcher() i=6 2024-03-28T16:50:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:105 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93956905608896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55740bfe0aa0 con 0x55740bfaba00 2024-03-28T16:50:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc067ee7640 1 -- 172.21.15.170:0/2788609699 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== osd_op_reply(105 notify.7 [watch unwatch cookie 93956905608896] v31'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fc058112070 con 0x55740bfaba00 2024-03-28T16:50:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 20 remove_watcher() i=7 2024-03-28T16:50:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bfaba00 msgr2=0x55740bfaddf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bfaba00 0x55740bfaddf0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740bfa7490 msgr2=0x55740bfa9950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740bfa7490 0x55740bfa9950 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc05c01f430 msgr2=0x7fc05c01f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fc05c01f430 0x7fc05c01f850 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55740bfb05e0 msgr2=0x55740bfb2aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55740bfb05e0 0x55740bfb2aa0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf300342b0 msgr2=0x7fbf30036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf300342b0 0x7fbf30036770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55740bf769a0 tx=0x7fc05c023040 comp rx=0 tx=0).stop 2024-03-28T16:50:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc0686e8640 1 -- 172.21.15.170:0/2788609699 reap_dead start 2024-03-28T16:50:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 msgr2=0x55740bf9c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf9c1c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc0580cfd20 tx=0x7fc0580b6f50 comp rx=0 tx=0).stop 2024-03-28T16:50:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 shutdown_connections 2024-03-28T16:50:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/2788609699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf95320 0x55740bf9c1c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 >> 172.21.15.170:0/2788609699 conn(0x55740bf916b0 msgr2=0x55740bf8ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 shutdown_connections 2024-03-28T16:50:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/2788609699 wait complete. 2024-03-28T16:50:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf400342b0 msgr2=0x7fbf40036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf400342b0 0x7fbf40036770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00bfe0 tx=0x7fc05c006000 comp rx=0 tx=0).stop 2024-03-28T16:50:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf883f0 msgr2=0x55740bf88810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf883f0 0x55740bf88810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc05801b380 tx=0x7fc0580bdae0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 shutdown_connections 2024-03-28T16:50:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf400342b0 0x7fbf40036770 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/3981757835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bf883f0 0x55740bf88810 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 >> 172.21.15.170:0/3981757835 conn(0x55740bf7d200 msgr2=0x55740bf7dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 shutdown_connections 2024-03-28T16:50:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/3981757835 wait complete. 2024-03-28T16:50:17.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bf73a60 msgr2=0x55740bf75e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55740bf73a60 0x55740bf75e80 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740be7b140 msgr2=0x55740bf718f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55740be7b140 0x55740bf718f0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55740bf77a30 msgr2=0x55740bf79e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55740bf77a30 0x55740bf79e20 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf500342b0 msgr2=0x7fbf50036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf500342b0 0x7fbf50036770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c10 tx=0x7fc05c0013c0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 msgr2=0x55740be78310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:17.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 --2- 172.21.15.170:0/381926293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55740bdec080 0x55740be78310 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc058005940 tx=0x7fc0580180e0 comp rx=0 tx=0).stop 2024-03-28T16:50:17.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc0686e8640 1 -- 172.21.15.170:0/381926293 reap_dead start 2024-03-28T16:50:17.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 shutdown_connections 2024-03-28T16:50:17.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 >> 172.21.15.170:0/381926293 conn(0x55740be726e0 msgr2=0x55740bf5c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 shutdown_connections 2024-03-28T16:50:17.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:17.340+0000 7fc06a3cc980 1 -- 172.21.15.170:0/381926293 wait complete. 2024-03-28T16:50:17.363 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:50:22.373 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:50:22.373 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-03-28T16:50:22.373 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-03-28T16:50:22.373 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-03-28T16:50:22.373 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-03-28T16:50:22.420 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:22.420 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:22.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b820c980 1 Processor -- start 2024-03-28T16:50:22.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b820c980 1 -- start start 2024-03-28T16:50:22.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b820c980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560909ab9670 0x560909abba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b820c980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5609099ff4c0 con 0x560909ab5170 2024-03-28T16:50:22.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b820c980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5609099c60f0 con 0x560909ab9670 2024-03-28T16:50:22.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b5526640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560909ab9670 0x560909abba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b5526640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x560909ab9670 0x560909abba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47420/0 (socket says 172.21.15.170:47420) 2024-03-28T16:50:22.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b5526640 1 -- 172.21.15.170:0/4089918791 learned_addr learned my addr 172.21.15.170:0/4089918791 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.420+0000 7fa0b5d27640 1 --1- 172.21.15.170:0/4089918791 >> v1:172.21.15.119:6789/0 conn(0x560909ab5170 0x560909ab5550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:49598/0 (socket says 172.21.15.170:49598) 2024-03-28T16:50:22.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.424+0000 7fa0b5526640 1 -- 172.21.15.170:0/4089918791 >> v1:172.21.15.119:6789/0 conn(0x560909ab5170 legacy=0x560909ab5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.424+0000 7fa0b5526640 1 -- 172.21.15.170:0/4089918791 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560909abcfb0 con 0x560909ab9670 2024-03-28T16:50:22.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.424+0000 7fa0b5526640 1 --2- 172.21.15.170:0/4089918791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab9670 0x560909abba60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000a4e0 tx=0x7fa0a001b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbdc85eec65ee30 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.424+0000 7fa0b4d25640 1 -- 172.21.15.170:0/4089918791 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a00aa540 con 0x560909ab9670 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.424+0000 7fa0b4d25640 1 -- 172.21.15.170:0/4089918791 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a00a6070 con 0x560909ab9670 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.424+0000 7fa0b4d25640 1 -- 172.21.15.170:0/4089918791 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a00aa850 con 0x560909ab9670 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.428+0000 7fa0b820c980 1 -- 172.21.15.170:0/4089918791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab9670 msgr2=0x560909abba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.428+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4089918791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab9670 0x560909abba60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000a4e0 tx=0x7fa0a001b090 comp rx=0 tx=0).stop 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.428+0000 7fa0b820c980 1 -- 172.21.15.170:0/4089918791 shutdown_connections 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.428+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4089918791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab9670 0x560909abba60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.428+0000 7fa0b820c980 1 -- 172.21.15.170:0/4089918791 >> 172.21.15.170:0/4089918791 conn(0x560909a28890 msgr2=0x560909a28ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.428+0000 7fa0b820c980 1 -- 172.21.15.170:0/4089918791 shutdown_connections 2024-03-28T16:50:22.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.428+0000 7fa0b820c980 1 -- 172.21.15.170:0/4089918791 wait complete. 2024-03-28T16:50:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.440+0000 7fa0b820c980 1 Processor -- start 2024-03-28T16:50:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.440+0000 7fa0b820c980 1 -- start start 2024-03-28T16:50:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909b9f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609099c3620 con 0x560909ab5170 2024-03-28T16:50:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909b9f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909b9f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47424/0 (socket says 172.21.15.170:47424) 2024-03-28T16:50:22.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 -- 172.21.15.170:0/1054405740 learned_addr learned my addr 172.21.15.170:0/1054405740 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 -- 172.21.15.170:0/1054405740 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a000a1c0 con 0x560909ab5170 2024-03-28T16:50:22.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/1054405740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909b9f0e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8001cc0 tx=0x7fa0a800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8300090ae2ddb92 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7f9fa3e07640 1 -- 172.21.15.170:0/1054405740 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8014070 con 0x560909ab5170 2024-03-28T16:50:22.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7f9fa3e07640 1 -- 172.21.15.170:0/1054405740 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a800f590 con 0x560909ab5170 2024-03-28T16:50:22.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7f9fa3e07640 1 -- 172.21.15.170:0/1054405740 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8007040 con 0x560909ab5170 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- 172.21.15.170:0/1054405740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 msgr2=0x560909b9f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 --2- 172.21.15.170:0/1054405740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909b9f0e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8001cc0 tx=0x7fa0a800b610 comp rx=0 tx=0).stop 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- 172.21.15.170:0/1054405740 shutdown_connections 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 --2- 172.21.15.170:0/1054405740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909b9f0e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- 172.21.15.170:0/1054405740 >> 172.21.15.170:0/1054405740 conn(0x560909a28890 msgr2=0x560909b9ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- 172.21.15.170:0/1054405740 shutdown_connections 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- 172.21.15.170:0/1054405740 wait complete. 2024-03-28T16:50:22.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 Processor -- start 2024-03-28T16:50:22.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- start start 2024-03-28T16:50:22.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909ba5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b820c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609099bee00 con 0x560909ab5170 2024-03-28T16:50:22.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909ba5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909ba5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47440/0 (socket says 172.21.15.170:47440) 2024-03-28T16:50:22.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.444+0000 7fa0b5d27640 1 -- 172.21.15.170:0/4045132494 learned_addr learned my addr 172.21.15.170:0/4045132494 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b5d27640 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609099ff4c0 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909ba5f30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x560909b9ff70 tx=0x7fa0a8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7f9fa2604640 1 -- 172.21.15.170:0/4045132494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8014030 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7f9fa2604640 1 -- 172.21.15.170:0/4045132494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a801c840 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560909ba6470 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7f9fa2604640 1 -- 172.21.15.170:0/4045132494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a801cb40 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560909ba68a0 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7f9fa2604640 1 -- 172.21.15.170:0/4045132494 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa0a801cca0 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7f9fa2604640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f9c0342b0 0x7f9f9c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7f9fa2604640 1 -- 172.21.15.170:0/4045132494 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa0a8006070 con 0x560909ab5170 2024-03-28T16:50:22.449 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b5526640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f9c0342b0 0x7f9f9c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909ab9670 0x560909bacbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bad0f0 con 0x560909ab9670 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b6528640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909ab9670 0x560909bacbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b5526640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f9c0342b0 0x7f9f9c036770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00ab000 tx=0x7fa0a00a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b6528640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909ab9670 0x560909bacbb0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b6528640 1 -- 172.21.15.170:0/4045132494 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909ab9670 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bad450 con 0x560909ab9670 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b6528640 1 -- 172.21.15.170:0/4045132494 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909ab9670 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909baf170 0x560909bb1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.448+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bb1aa0 con 0x560909baf170 2024-03-28T16:50:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909baf170 0x560909bb1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909baf170 0x560909bb1560 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b5d27640 1 -- 172.21.15.170:0/4045132494 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fa0a8016070 con 0x560909baf170 2024-03-28T16:50:22.452 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909badfe0 con 0x560909ab9670 2024-03-28T16:50:22.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b6528640 1 -- 172.21.15.170:0/4045132494 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909ab9670 2024-03-28T16:50:22.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560909bb2e40 0x560909bb5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bb58d0 con 0x560909bb2e40 2024-03-28T16:50:22.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b5526640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560909bb2e40 0x560909bb5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.453 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b5526640 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560909bb2e40 0x560909bb5230 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b5526640 1 -- 172.21.15.170:0/4045132494 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909bb2e40 2024-03-28T16:50:22.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b820c980 1 Processor -- start 2024-03-28T16:50:22.454 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b820c980 1 -- start start 2024-03-28T16:50:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b820c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bc3800 0x560909bc3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b820c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560909957e30 con 0x560909bc3800 2024-03-28T16:50:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.452+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bc3800 0x560909bc3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bc3800 0x560909bc3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47450/0 (socket says 172.21.15.170:47450) 2024-03-28T16:50:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5d27640 1 -- 172.21.15.170:0/774775945 learned_addr learned my addr 172.21.15.170:0/774775945 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5d27640 1 -- 172.21.15.170:0/774775945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560909bc4160 con 0x560909bc3800 2024-03-28T16:50:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bc3800 0x560909bc3c20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000f70 tx=0x7fa0a80187d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7f9f9b7fe640 1 -- 172.21.15.170:0/774775945 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8017c90 con 0x560909bc3800 2024-03-28T16:50:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7f9f9b7fe640 1 -- 172.21.15.170:0/774775945 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8017df0 con 0x560909bc3800 2024-03-28T16:50:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560909bc43b0 con 0x560909bc3800 2024-03-28T16:50:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7f9f9b7fe640 1 -- 172.21.15.170:0/774775945 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a80419d0 con 0x560909bc3800 2024-03-28T16:50:22.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560909bc4f00 con 0x560909bc3800 2024-03-28T16:50:22.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7f9f9b7fe640 1 -- 172.21.15.170:0/774775945 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa0a8041b30 con 0x560909bc3800 2024-03-28T16:50:22.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7f9f9b7fe640 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f900342b0 0x7f9f90036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7f9f9b7fe640 1 -- 172.21.15.170:0/774775945 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa0a8045050 con 0x560909bc3800 2024-03-28T16:50:22.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5526640 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f900342b0 0x7f9f90036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b820c980 1 Processor -- start 2024-03-28T16:50:22.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b820c980 1 -- start start 2024-03-28T16:50:22.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5526640 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f900342b0 0x7f9f90036770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x560909baa640 tx=0x7fa0a0005e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b820c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b820c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56090994b790 con 0x560909bd06f0 2024-03-28T16:50:22.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47458/0 (socket says 172.21.15.170:47458) 2024-03-28T16:50:22.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.456+0000 7fa0b5d27640 1 -- 172.21.15.170:0/2087809014 learned_addr learned my addr 172.21.15.170:0/2087809014 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b5d27640 1 -- 172.21.15.170:0/2087809014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560909bd10e0 con 0x560909bd06f0 2024-03-28T16:50:22.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/2087809014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd0b10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8011990 tx=0x7fa0a80412d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2fcdc628218f96 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7f9f997fa640 1 -- 172.21.15.170:0/2087809014 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a803cc90 con 0x560909bd06f0 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7f9f997fa640 1 -- 172.21.15.170:0/2087809014 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8046b40 con 0x560909bd06f0 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7f9f997fa640 1 -- 172.21.15.170:0/2087809014 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a8049800 con 0x560909bd06f0 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- 172.21.15.170:0/2087809014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 msgr2=0x560909bd0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 --2- 172.21.15.170:0/2087809014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd0b10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8011990 tx=0x7fa0a80412d0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- 172.21.15.170:0/2087809014 shutdown_connections 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 --2- 172.21.15.170:0/2087809014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd0b10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- 172.21.15.170:0/2087809014 >> 172.21.15.170:0/2087809014 conn(0x560909bcca80 msgr2=0x560909bcaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- 172.21.15.170:0/2087809014 shutdown_connections 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- 172.21.15.170:0/2087809014 wait complete. 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 Processor -- start 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- start start 2024-03-28T16:50:22.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560909bd7a60 con 0x560909bd06f0 2024-03-28T16:50:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b5d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47470/0 (socket says 172.21.15.170:47470) 2024-03-28T16:50:22.462 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 learned_addr learned my addr 172.21.15.170:0/941346605 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560909bc4f00 con 0x560909bd06f0 2024-03-28T16:50:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd7520 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80265d0 tx=0x7fa0a8028f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7f9f877fe640 1 -- 172.21.15.170:0/941346605 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a803cc90 con 0x560909bd06f0 2024-03-28T16:50:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7f9f877fe640 1 -- 172.21.15.170:0/941346605 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8046980 con 0x560909bd06f0 2024-03-28T16:50:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560909bd7c60 con 0x560909bd06f0 2024-03-28T16:50:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7f9f877fe640 1 -- 172.21.15.170:0/941346605 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0a803c580 con 0x560909bd06f0 2024-03-28T16:50:22.463 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.460+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560909bd7c60 con 0x560909bd06f0 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7f9f877fe640 1 -- 172.21.15.170:0/941346605 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa0a8026870 con 0x560909bd06f0 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7f9f877fe640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f800342b0 0x7f9f80036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7f9f877fe640 1 -- 172.21.15.170:0/941346605 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa0a8074bf0 con 0x560909bd06f0 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b5526640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f800342b0 0x7f9f80036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909be2800 0x560909be4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be5410 con 0x560909be2800 2024-03-28T16:50:22.466 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b6528640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909be2800 0x560909be4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b5526640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f800342b0 0x7f9f80036770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00aafd0 tx=0x7fa0a00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b6528640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909be2800 0x560909be4cc0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 realm 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be57e0 con 0x560909be2800 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.467 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be63c0 con 0x560909be2800 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909be6d70 0x560909be9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be9770 con 0x560909be6d70 2024-03-28T16:50:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.464+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909be6d70 0x560909be9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5d27640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909be6d70 0x560909be9160 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b820c980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b820c980 20 searching for the correct realm 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560909beb970 0x560909bede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bee500 con 0x560909beb970 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5526640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560909beb970 0x560909bede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5526640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560909beb970 0x560909bede30 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5526640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa0a00cd050 0x7fa0a00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bee500 con 0x7fa0a00cd050 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b6528640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa0a00cd050 0x7fa0a00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b6528640 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa0a00cd050 0x7fa0a00cd430 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.468+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x7fa0a00cd050 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.472+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x7fa0a00cd050 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.476+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x7fa0a00cd050 2024-03-28T16:50:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.480 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x7fa0a00cd050 2024-03-28T16:50:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909beb970 2024-03-28T16:50:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be6d70 2024-03-28T16:50:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.480+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x560909be2800 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be6060 con 0x7fa0a00cd050 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be9d70 con 0x560909be2800 2024-03-28T16:50:22.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be9d70 con 0x560909be2800 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909be9d20 con 0x7fa0a00cd050 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 zone default found 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 4 Realm: () 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 zonegroup default 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bea5c0 con 0x560909be6d70 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909beaa00 con 0x560909beb970 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf0420 con 0x560909beb970 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 started sync module instance, tier type = 2024-03-28T16:50:22.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:22.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.484+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909befc00 con 0x7fa0a00cd050 2024-03-28T16:50:22.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.488+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.488+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94596818204512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909beffb0 con 0x7fa0a00cd050 2024-03-28T16:50:22.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.488+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf1180 con 0x560909be2800 2024-03-28T16:50:22.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.488+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94596818204512] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.488+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.488+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94596818208832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf1140 con 0x560909be2800 2024-03-28T16:50:22.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.488+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf1bf0 con 0x560909beb970 2024-03-28T16:50:22.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94596818208832] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94596818212592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf1fa0 con 0x560909beb970 2024-03-28T16:50:22.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf2910 con 0x7fa0a00cd050 2024-03-28T16:50:22.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94596818216320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf3160 con 0x7fa0a00cd050 2024-03-28T16:50:22.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf3b10 con 0x560909be2800 2024-03-28T16:50:22.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.492+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94596818212592] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94596818216320] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94596818209888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf2c40 con 0x560909be2800 2024-03-28T16:50:22.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf4360 con 0x560909be6d70 2024-03-28T16:50:22.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94596818209888] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94596818222784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf2c40 con 0x560909be6d70 2024-03-28T16:50:22.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.496+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf4b60 con 0x560909be6d70 2024-03-28T16:50:22.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.500+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94596818222784] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.500+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.500+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94596818220592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf53b0 con 0x560909be6d70 2024-03-28T16:50:22.501 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.500+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf5d20 con 0x560909be6d70 2024-03-28T16:50:22.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94596818220592] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94596818227152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf5d20 con 0x560909be6d70 2024-03-28T16:50:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=0 2024-03-28T16:50:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=1 2024-03-28T16:50:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=2 2024-03-28T16:50:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=3 2024-03-28T16:50:22.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=4 2024-03-28T16:50:22.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=5 2024-03-28T16:50:22.504 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=6 2024-03-28T16:50:22.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94596818227152] v31'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 20 add_watcher() i=7 2024-03-28T16:50:22.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:22.505 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf9080 con 0x560909be2800 2024-03-28T16:50:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.504+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94596818229744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909bf62b0 con 0x560909be2800 2024-03-28T16:50:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94596818229744] v31'214 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7f9f677fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 5 note: GC not initialized 2024-03-28T16:50:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7f9f667fc640 20 reqs_thread_entry: start 2024-03-28T16:50:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 Filter name: none 2024-03-28T16:50:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7f9ed7fff640 20 reqs_thread_entry: start 2024-03-28T16:50:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:50:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1b0b0 con 0x560909be6d70 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c198a0 con 0x560909be6d70 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:50:22.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:50:22.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.508+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c19ca0 con 0x560909be6d70 2024-03-28T16:50:22.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v31'5 uv5 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b820c980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-03-28T16:50:22.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b820c980 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:50:22.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b820c980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-03-28T16:50:22.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94596818381584 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1e610 con 0x560909be6d70 2024-03-28T16:50:22.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify (1) cookie 94596818222784 notify 133143986215 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x560909bc4f00 con 0x560909be6d70 2024-03-28T16:50:22.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7f9f997fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986215 cookie 94596818222784 notifier 4732 bl.length()=522 2024-03-28T16:50:22.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(65 notify.5 [notify cookie 94596818381584 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7f9f997fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-03-28T16:50:22.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7f9f997fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:50:22.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7f9f997fa640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f9f78003ea0 con 0x560909be6d70 2024-03-28T16:50:22.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== watch-notify(notify_complete (2) cookie 94596818381584 notify 133143986215 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560909bd10e0 con 0x560909be6d70 2024-03-28T16:50:22.514 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.512+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1ab10 con 0x560909be6d70 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v31'9 uv9 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b820c980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b820c980 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b820c980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94596818381584 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1e5e0 con 0x560909beb970 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== watch-notify(notify (1) cookie 94596818212592 notify 133143986190 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7fa0a00d5070 con 0x560909beb970 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7f9f997fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986190 cookie 94596818212592 notifier 4732 bl.length()=177 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(68 notify.2 [notify cookie 94596818381584 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7f9f997fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7f9f997fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:50:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7f9f997fa640 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f9f78003ea0 con 0x560909beb970 2024-03-28T16:50:22.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== watch-notify(notify_complete (2) cookie 94596818381584 notify 133143986190 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x560909abcfb0 con 0x560909beb970 2024-03-28T16:50:22.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.518 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:50:22.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7f9f677fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:22.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.516+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94596818229744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x560909be2800 2024-03-28T16:50:22.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.520+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94596818229744] v31'215 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.520+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94596818204512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x7fa0a00cd050 2024-03-28T16:50:22.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.520+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94596818204512] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.520+0000 7fa0b820c980 20 remove_watcher() i=0 2024-03-28T16:50:22.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.520+0000 7fa0b820c980 2 removed watcher, disabling cache 2024-03-28T16:50:22.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.520+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94596818208832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x560909be2800 2024-03-28T16:50:22.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.524+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94596818208832] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.524+0000 7fa0b820c980 20 remove_watcher() i=1 2024-03-28T16:50:22.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.524+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94596818212592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x560909beb970 2024-03-28T16:50:22.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.524+0000 7fa0b5526640 1 -- 172.21.15.170:0/941346605 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94596818212592] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a00aaa00 con 0x560909beb970 2024-03-28T16:50:22.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.524+0000 7fa0b820c980 20 remove_watcher() i=2 2024-03-28T16:50:22.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.524+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94596818216320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x7fa0a00cd050 2024-03-28T16:50:22.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.524+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94596818216320] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x7fa0a00cd050 2024-03-28T16:50:22.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.528+0000 7fa0b820c980 20 remove_watcher() i=3 2024-03-28T16:50:22.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.528+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94596818209888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x560909be2800 2024-03-28T16:50:22.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.528+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94596818209888] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0b00083f0 con 0x560909be2800 2024-03-28T16:50:22.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.528+0000 7fa0b820c980 20 remove_watcher() i=4 2024-03-28T16:50:22.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.528+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94596818222784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x560909be6d70 2024-03-28T16:50:22.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.532+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94596818222784] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.532+0000 7fa0b820c980 20 remove_watcher() i=5 2024-03-28T16:50:22.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.532+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94596818220592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x560909be6d70 2024-03-28T16:50:22.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.532+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94596818220592] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.532+0000 7fa0b820c980 20 remove_watcher() i=6 2024-03-28T16:50:22.533 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.532+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94596818227152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x560909c1a2b0 con 0x560909be6d70 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b5d27640 1 -- 172.21.15.170:0/941346605 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94596818227152] v31'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa0a8016070 con 0x560909be6d70 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 20 remove_watcher() i=7 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909be6d70 msgr2=0x560909be9160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909be6d70 0x560909be9160 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909be2800 msgr2=0x560909be4cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909be2800 0x560909be4cc0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa0a00cd050 msgr2=0x7fa0a00cd430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa0a00cd050 0x7fa0a00cd430 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560909beb970 msgr2=0x560909bede30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x560909beb970 0x560909bede30 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f800342b0 msgr2=0x7f9f80036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f800342b0 0x7f9f80036770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00aafd0 tx=0x7fa0a00a6000 comp rx=0 tx=0).stop 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 msgr2=0x560909bd7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/941346605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bd06f0 0x560909bd7520 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80265d0 tx=0x7fa0a8028f00 comp rx=0 tx=0).stop 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b6528640 1 -- 172.21.15.170:0/941346605 reap_dead start 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 shutdown_connections 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 >> 172.21.15.170:0/941346605 conn(0x560909bcca80 msgr2=0x560909bca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 shutdown_connections 2024-03-28T16:50:22.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/941346605 wait complete. 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f900342b0 msgr2=0x7f9f90036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f900342b0 0x7f9f90036770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x560909baa640 tx=0x7fa0a0005e50 comp rx=0 tx=0).stop 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bc3800 msgr2=0x560909bc3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bc3800 0x560909bc3c20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8000f70 tx=0x7fa0a80187d0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 shutdown_connections 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f900342b0 0x7f9f90036770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/774775945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909bc3800 0x560909bc3c20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 >> 172.21.15.170:0/774775945 conn(0x560909bb8610 msgr2=0x560909bb9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 shutdown_connections 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/774775945 wait complete. 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909baf170 msgr2=0x560909bb1560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x560909baf170 0x560909bb1560 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909ab9670 msgr2=0x560909bacbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x560909ab9670 0x560909bacbb0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560909bb2e40 msgr2=0x560909bb5230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x560909bb2e40 0x560909bb5230 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f9c0342b0 msgr2=0x7f9f9c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9f9c0342b0 0x7f9f9c036770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa0a00ab000 tx=0x7fa0a00a5000 comp rx=0 tx=0).stop 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 msgr2=0x560909ba5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 --2- 172.21.15.170:0/4045132494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560909ab5170 0x560909ba5f30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x560909b9ff70 tx=0x7fa0a8000bd0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b6528640 1 -- 172.21.15.170:0/4045132494 reap_dead start 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 shutdown_connections 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 >> 172.21.15.170:0/4045132494 conn(0x560909a28890 msgr2=0x560909b99780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 shutdown_connections 2024-03-28T16:50:22.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.536+0000 7fa0b820c980 1 -- 172.21.15.170:0/4045132494 wait complete. 2024-03-28T16:50:22.548 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:50:22.567 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:50:22.573 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-03-28T16:50:22.573 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-03-28T16:50:22.573 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-03-28T16:50:22.574 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-03-28T16:50:22.607 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:22.607 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:22.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a803e980 1 Processor -- start 2024-03-28T16:50:22.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a803e980 1 -- start start 2024-03-28T16:50:22.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a803e980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5563bca25670 0x5563bca27a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a803e980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5563bc96d4c0 con 0x5563bca226b0 2024-03-28T16:50:22.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a803e980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5563bc9340f0 con 0x5563bca25670 2024-03-28T16:50:22.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a5358640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5563bca25670 0x5563bca27a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a5358640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5563bca25670 0x5563bca27a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47480/0 (socket says 172.21.15.170:47480) 2024-03-28T16:50:22.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a5358640 1 -- 172.21.15.170:0/672295857 learned_addr learned my addr 172.21.15.170:0/672295857 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a5b59640 1 --1- 172.21.15.170:0/672295857 >> v1:172.21.15.119:6789/0 conn(0x5563bca226b0 0x5563bca22a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:49612/0 (socket says 172.21.15.170:49612) 2024-03-28T16:50:22.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a5358640 1 -- 172.21.15.170:0/672295857 >> v1:172.21.15.119:6789/0 conn(0x5563bca226b0 legacy=0x5563bca22a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.608+0000 7f24a5358640 1 -- 172.21.15.170:0/672295857 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bca231a0 con 0x5563bca25670 2024-03-28T16:50:22.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a5358640 1 --2- 172.21.15.170:0/672295857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca25670 0x5563bca27a60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f249c00a5c0 tx=0x7f249c01b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224562b96993daed server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a4b57640 1 -- 172.21.15.170:0/672295857 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f249c0aa470 con 0x5563bca25670 2024-03-28T16:50:22.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a4b57640 1 -- 172.21.15.170:0/672295857 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c0aa5d0 con 0x5563bca25670 2024-03-28T16:50:22.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a4b57640 1 -- 172.21.15.170:0/672295857 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f249c0aa890 con 0x5563bca25670 2024-03-28T16:50:22.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a803e980 1 -- 172.21.15.170:0/672295857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca25670 msgr2=0x5563bca27a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.613 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a803e980 1 --2- 172.21.15.170:0/672295857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca25670 0x5563bca27a60 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f249c00a5c0 tx=0x7f249c01b340 comp rx=0 tx=0).stop 2024-03-28T16:50:22.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a803e980 1 -- 172.21.15.170:0/672295857 shutdown_connections 2024-03-28T16:50:22.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a803e980 1 --2- 172.21.15.170:0/672295857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca25670 0x5563bca27a60 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a803e980 1 -- 172.21.15.170:0/672295857 >> 172.21.15.170:0/672295857 conn(0x5563bca1ae60 msgr2=0x5563bca1d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a803e980 1 -- 172.21.15.170:0/672295857 shutdown_connections 2024-03-28T16:50:22.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.612+0000 7f24a803e980 1 -- 172.21.15.170:0/672295857 wait complete. 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 Processor -- start 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 -- start start 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb0aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bc931620 con 0x5563bca226b0 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb0aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb0aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47486/0 (socket says 172.21.15.170:47486) 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a5b59640 1 -- 172.21.15.170:0/1752350518 learned_addr learned my addr 172.21.15.170:0/1752350518 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a5b59640 1 -- 172.21.15.170:0/1752350518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c005df0 con 0x5563bca226b0 2024-03-28T16:50:22.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a5b59640 1 --2- 172.21.15.170:0/1752350518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb0aec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2498001cc0 tx=0x7f249800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63a37b45758f452 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f2393e07640 1 -- 172.21.15.170:0/1752350518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498014070 con 0x5563bca226b0 2024-03-28T16:50:22.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f2393e07640 1 -- 172.21.15.170:0/1752350518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249800f590 con 0x5563bca226b0 2024-03-28T16:50:22.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f2393e07640 1 -- 172.21.15.170:0/1752350518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498007040 con 0x5563bca226b0 2024-03-28T16:50:22.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 -- 172.21.15.170:0/1752350518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 msgr2=0x5563bcb0aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 --2- 172.21.15.170:0/1752350518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb0aec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2498001cc0 tx=0x7f249800b610 comp rx=0 tx=0).stop 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 -- 172.21.15.170:0/1752350518 shutdown_connections 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 --2- 172.21.15.170:0/1752350518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb0aec0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 -- 172.21.15.170:0/1752350518 >> 172.21.15.170:0/1752350518 conn(0x5563bca1ae60 msgr2=0x5563bcb06a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 -- 172.21.15.170:0/1752350518 shutdown_connections 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.628+0000 7f24a803e980 1 -- 172.21.15.170:0/1752350518 wait complete. 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 Processor -- start 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 -- start start 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb11d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bc92ce00 con 0x5563bca226b0 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb11d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb11d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47490/0 (socket says 172.21.15.170:47490) 2024-03-28T16:50:22.632 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a5b59640 1 -- 172.21.15.170:0/4239125777 learned_addr learned my addr 172.21.15.170:0/4239125777 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a5b59640 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bc96d4c0 con 0x5563bca226b0 2024-03-28T16:50:22.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a5b59640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb11d10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5563bcb0bd50 tx=0x7f2498000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f2392604640 1 -- 172.21.15.170:0/4239125777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498014030 con 0x5563bca226b0 2024-03-28T16:50:22.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f2392604640 1 -- 172.21.15.170:0/4239125777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249801c840 con 0x5563bca226b0 2024-03-28T16:50:22.633 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5563bcb12250 con 0x5563bca226b0 2024-03-28T16:50:22.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f2392604640 1 -- 172.21.15.170:0/4239125777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f249801cb40 con 0x5563bca226b0 2024-03-28T16:50:22.634 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563bcb12680 con 0x5563bca226b0 2024-03-28T16:50:22.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f2392604640 1 -- 172.21.15.170:0/4239125777 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f249801cca0 con 0x5563bca226b0 2024-03-28T16:50:22.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f2392604640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f238c0342b0 0x7f238c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f2392604640 1 -- 172.21.15.170:0/4239125777 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2498006070 con 0x5563bca226b0 2024-03-28T16:50:22.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a5358640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f238c0342b0 0x7f238c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bca25670 0x5563bcb18990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.635 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.632+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb18ed0 con 0x5563bca25670 2024-03-28T16:50:22.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a635a640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bca25670 0x5563bcb18990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a5358640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f238c0342b0 0x7f238c036770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f249c0ae000 tx=0x7f249c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a635a640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bca25670 0x5563bcb18990 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a635a640 1 -- 172.21.15.170:0/4239125777 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bca25670 2024-03-28T16:50:22.636 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb19230 con 0x5563bca25670 2024-03-28T16:50:22.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a635a640 1 -- 172.21.15.170:0/4239125777 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bca25670 2024-03-28T16:50:22.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb1af50 0x5563bcb1d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb1d880 con 0x5563bcb1af50 2024-03-28T16:50:22.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a5b59640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb1af50 0x5563bcb1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a5b59640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb1af50 0x5563bcb1d340 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a5b59640 1 -- 172.21.15.170:0/4239125777 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb1af50 2024-03-28T16:50:22.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb19d90 con 0x5563bca25670 2024-03-28T16:50:22.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a635a640 1 -- 172.21.15.170:0/4239125777 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bca25670 2024-03-28T16:50:22.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5563bcb1ec10 0x5563bcb21000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb216a0 con 0x5563bcb1ec10 2024-03-28T16:50:22.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a5358640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5563bcb1ec10 0x5563bcb21000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.636+0000 7f24a5358640 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5563bcb1ec10 0x5563bcb21000 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a5358640 1 -- 172.21.15.170:0/4239125777 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb1ec10 2024-03-28T16:50:22.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a803e980 1 Processor -- start 2024-03-28T16:50:22.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a803e980 1 -- start start 2024-03-28T16:50:22.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a803e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb2f5b0 0x5563bcb2f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a803e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bc8c5e30 con 0x5563bcb2f5b0 2024-03-28T16:50:22.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb2f5b0 0x5563bcb2f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb2f5b0 0x5563bcb2f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47498/0 (socket says 172.21.15.170:47498) 2024-03-28T16:50:22.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a5b59640 1 -- 172.21.15.170:0/1048168968 learned_addr learned my addr 172.21.15.170:0/1048168968 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a5b59640 1 -- 172.21.15.170:0/1048168968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bcb2ff10 con 0x5563bcb2f5b0 2024-03-28T16:50:22.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a5b59640 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb2f5b0 0x5563bcb2f9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2498000f70 tx=0x7f24980187d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f238b7fe640 1 -- 172.21.15.170:0/1048168968 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498017c90 con 0x5563bcb2f5b0 2024-03-28T16:50:22.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f238b7fe640 1 -- 172.21.15.170:0/1048168968 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498017df0 con 0x5563bcb2f5b0 2024-03-28T16:50:22.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5563bcb30160 con 0x5563bcb2f5b0 2024-03-28T16:50:22.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f238b7fe640 1 -- 172.21.15.170:0/1048168968 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24980419d0 con 0x5563bcb2f5b0 2024-03-28T16:50:22.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563bcb30cb0 con 0x5563bcb2f5b0 2024-03-28T16:50:22.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f238b7fe640 1 -- 172.21.15.170:0/1048168968 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f2498041b30 con 0x5563bcb2f5b0 2024-03-28T16:50:22.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f238b7fe640 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23800342b0 0x7f2380036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f238b7fe640 1 -- 172.21.15.170:0/1048168968 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2498045050 con 0x5563bcb2f5b0 2024-03-28T16:50:22.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.640+0000 7f24a5358640 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23800342b0 0x7f2380036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 Processor -- start 2024-03-28T16:50:22.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- start start 2024-03-28T16:50:22.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bcb3c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bc8b9790 con 0x5563bcb3c510 2024-03-28T16:50:22.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5358640 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23800342b0 0x7f2380036770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5563bcb163e0 tx=0x7f249c005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bcb3c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bcb3c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47510/0 (socket says 172.21.15.170:47510) 2024-03-28T16:50:22.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 -- 172.21.15.170:0/2677576125 learned_addr learned my addr 172.21.15.170:0/2677576125 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 -- 172.21.15.170:0/2677576125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bcb3cec0 con 0x5563bcb3c510 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 --2- 172.21.15.170:0/2677576125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bcb3c8f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2498011990 tx=0x7f24980412d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d48809c50c604e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f23897fa640 1 -- 172.21.15.170:0/2677576125 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498046b40 con 0x5563bcb3c510 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f23897fa640 1 -- 172.21.15.170:0/2677576125 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498059070 con 0x5563bcb3c510 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f23897fa640 1 -- 172.21.15.170:0/2677576125 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498049800 con 0x5563bcb3c510 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- 172.21.15.170:0/2677576125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 msgr2=0x5563bcb3c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 --2- 172.21.15.170:0/2677576125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bcb3c8f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2498011990 tx=0x7f24980412d0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- 172.21.15.170:0/2677576125 shutdown_connections 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 --2- 172.21.15.170:0/2677576125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bcb3c8f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- 172.21.15.170:0/2677576125 >> 172.21.15.170:0/2677576125 conn(0x5563bcb38850 msgr2=0x5563bcb368b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- 172.21.15.170:0/2677576125 shutdown_connections 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- 172.21.15.170:0/2677576125 wait complete. 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 Processor -- start 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- start start 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bca21d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a803e980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bca22290 con 0x5563bcb3c510 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bca21d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bca21d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47520/0 (socket says 172.21.15.170:47520) 2024-03-28T16:50:22.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 learned_addr learned my addr 172.21.15.170:0/115704392 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.648 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.644+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bcb30cb0 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a5b59640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bca21d50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2498026800 tx=0x7f2498028f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f23777fe640 1 -- 172.21.15.170:0/115704392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498065070 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5563bca20470 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f23777fe640 1 -- 172.21.15.170:0/115704392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498046c70 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563bca20470 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f23777fe640 1 -- 172.21.15.170:0/115704392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2498041670 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f23777fe640 1 -- 172.21.15.170:0/115704392 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f249803c450 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f23777fe640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2370034210 0x7f23700366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f23777fe640 1 -- 172.21.15.170:0/115704392 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2498073ba0 con 0x5563bcb3c510 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a5358640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2370034210 0x7f23700366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bcb508c0 0x5563bcb52d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb534d0 con 0x5563bcb508c0 2024-03-28T16:50:22.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a635a640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bcb508c0 0x5563bcb52d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a5358640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2370034210 0x7f23700366d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f249c01b380 tx=0x7f249c0c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a635a640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bcb508c0 0x5563bcb52d80 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 realm 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb538a0 con 0x5563bcb508c0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.648+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54480 con 0x5563bcb508c0 2024-03-28T16:50:22.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:22.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb54e30 0x5563bcb57220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb57830 con 0x5563bcb54e30 2024-03-28T16:50:22.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5b59640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb54e30 0x5563bcb57220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5b59640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb54e30 0x5563bcb57220 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 20 searching for the correct realm 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5563bcb59a40 0x5563bcb5bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5c5d0 con 0x5563bcb59a40 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5358640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5563bcb59a40 0x5563bcb5bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5358640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5563bcb59a40 0x5563bcb5bf00 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5358640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f249c0e0040 0x7f249c0d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5c5d0 con 0x7f249c0e0040 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a635a640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f249c0e0040 0x7f249c0d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a635a640 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f249c0e0040 0x7f249c0d3400 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.652+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x7f249c0e0040 2024-03-28T16:50:22.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.656+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x7f249c0e0040 2024-03-28T16:50:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.660+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x7f249c0e0040 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x7f249c0e0040 2024-03-28T16:50:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb59a40 2024-03-28T16:50:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.664+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb54e30 2024-03-28T16:50:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.668 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x5563bcb508c0 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb54120 con 0x7f249c0e0040 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb57e40 con 0x5563bcb508c0 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb57e40 con 0x5563bcb508c0 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb57e10 con 0x7f249c0e0040 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 zone default found 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 4 Realm: () 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 zonegroup default 2024-03-28T16:50:22.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb58690 con 0x5563bcb54e30 2024-03-28T16:50:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb58a90 con 0x5563bcb59a40 2024-03-28T16:50:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5e4e0 con 0x5563bcb59a40 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 started sync module instance, tier type = 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.668+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5dcd0 con 0x7f249c0e0040 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.672+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.672+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93886856157232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5dc50 con 0x7f249c0e0040 2024-03-28T16:50:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.672+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5f240 con 0x5563bcb508c0 2024-03-28T16:50:22.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.672+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93886856157232] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.672+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.672+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93886856161536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5dc50 con 0x5563bcb508c0 2024-03-28T16:50:22.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.672+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5f8e0 con 0x5563bcb59a40 2024-03-28T16:50:22.675 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93886856161536] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.676 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93886856164320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5fc90 con 0x5563bcb59a40 2024-03-28T16:50:22.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb60600 con 0x7f249c0e0040 2024-03-28T16:50:22.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93886856168080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb60e50 con 0x7f249c0e0040 2024-03-28T16:50:22.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb61820 con 0x5563bcb508c0 2024-03-28T16:50:22.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.676+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93886856164320] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93886856168080] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93886856172320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb60930 con 0x5563bcb508c0 2024-03-28T16:50:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb62070 con 0x5563bcb54e30 2024-03-28T16:50:22.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93886856172320] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93886856174544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb60930 con 0x5563bcb54e30 2024-03-28T16:50:22.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.680+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb62870 con 0x5563bcb54e30 2024-03-28T16:50:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.684+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93886856174544] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.684+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.684+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93886856162592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb63070 con 0x5563bcb54e30 2024-03-28T16:50:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.684+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb639e0 con 0x5563bcb54e30 2024-03-28T16:50:22.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93886856162592] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93886856178912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb639e0 con 0x5563bcb54e30 2024-03-28T16:50:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=0 2024-03-28T16:50:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=1 2024-03-28T16:50:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=2 2024-03-28T16:50:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=3 2024-03-28T16:50:22.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=4 2024-03-28T16:50:22.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=5 2024-03-28T16:50:22.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=6 2024-03-28T16:50:22.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93886856178912] v31'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 20 add_watcher() i=7 2024-03-28T16:50:22.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:22.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb66da0 con 0x5563bcb508c0 2024-03-28T16:50:22.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.688+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93886856181520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb642b0 con 0x5563bcb508c0 2024-03-28T16:50:22.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93886856181520] v31'216 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f23577fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:22.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:22.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:22.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 5 note: GC not initialized 2024-03-28T16:50:22.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f23567fc640 20 reqs_thread_entry: start 2024-03-28T16:50:22.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:22.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 20 Filter name: none 2024-03-28T16:50:22.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f22c7fff640 20 reqs_thread_entry: start 2024-03-28T16:50:22.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-03-28T16:50:22.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.696 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb88d60 con 0x7f249c0e0040 2024-03-28T16:50:22.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(62 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:50:22.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-03-28T16:50:22.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-03-28T16:50:22.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:22.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb8ca30 con 0x5563bcb54e30 2024-03-28T16:50:22.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.692+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 fud [create,call,writefull 0~181] v31'6 uv5 ondisk = -17 ((17) File exists)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.701 INFO:teuthology.orchestra.run.smithi170.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-03-28T16:50:22.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f23577fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:22.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93886856181520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb895f0 con 0x5563bcb508c0 2024-03-28T16:50:22.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 93886856181520] v31'217 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93886856157232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb636e0 con 0x7f249c0e0040 2024-03-28T16:50:22.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 93886856157232] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f24a803e980 20 remove_watcher() i=0 2024-03-28T16:50:22.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f24a803e980 2 removed watcher, disabling cache 2024-03-28T16:50:22.702 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.696+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93886856161536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5d830 con 0x5563bcb508c0 2024-03-28T16:50:22.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.700+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 93886856161536] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.700+0000 7f24a803e980 20 remove_watcher() i=1 2024-03-28T16:50:22.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.700+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93886856164320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5d830 con 0x5563bcb59a40 2024-03-28T16:50:22.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.700+0000 7f24a5358640 1 -- 172.21.15.170:0/115704392 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 93886856164320] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f249c0b0020 con 0x5563bcb59a40 2024-03-28T16:50:22.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.700+0000 7f24a803e980 20 remove_watcher() i=2 2024-03-28T16:50:22.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.700+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93886856168080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5e900 con 0x7f249c0e0040 2024-03-28T16:50:22.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.704+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 93886856168080] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x7f249c0e0040 2024-03-28T16:50:22.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.704+0000 7f24a803e980 20 remove_watcher() i=3 2024-03-28T16:50:22.705 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.704+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93886856172320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5e900 con 0x5563bcb508c0 2024-03-28T16:50:22.706 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.704+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 93886856172320] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f24a000a4d0 con 0x5563bcb508c0 2024-03-28T16:50:22.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.704+0000 7f24a803e980 20 remove_watcher() i=4 2024-03-28T16:50:22.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.704+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93886856174544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb61320 con 0x5563bcb54e30 2024-03-28T16:50:22.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.708+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 93886856174544] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.708+0000 7f24a803e980 20 remove_watcher() i=5 2024-03-28T16:50:22.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.708+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93886856162592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb5e900 con 0x5563bcb54e30 2024-03-28T16:50:22.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.708+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 93886856162592] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.708+0000 7f24a803e980 20 remove_watcher() i=6 2024-03-28T16:50:22.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.708+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93886856178912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5563bcb61320 con 0x5563bcb54e30 2024-03-28T16:50:22.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a5b59640 1 -- 172.21.15.170:0/115704392 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 93886856178912] v31'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f2498016070 con 0x5563bcb54e30 2024-03-28T16:50:22.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 20 remove_watcher() i=7 2024-03-28T16:50:22.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb54e30 msgr2=0x5563bcb57220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb54e30 0x5563bcb57220 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bcb508c0 msgr2=0x5563bcb52d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bcb508c0 0x5563bcb52d80 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f249c0e0040 msgr2=0x7f249c0d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f249c0e0040 0x7f249c0d3400 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5563bcb59a40 msgr2=0x5563bcb5bf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5563bcb59a40 0x5563bcb5bf00 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2370034210 msgr2=0x7f23700366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2370034210 0x7f23700366d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f249c01b380 tx=0x7f249c0c1040 comp rx=0 tx=0).stop 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 msgr2=0x5563bca21d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/115704392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb3c510 0x5563bca21d50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2498026800 tx=0x7f2498028f00 comp rx=0 tx=0).stop 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 reap_dead start 2024-03-28T16:50:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a635a640 1 -- 172.21.15.170:0/115704392 reap_dead start 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 shutdown_connections 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 >> 172.21.15.170:0/115704392 conn(0x5563bcb38850 msgr2=0x5563bcb35eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 shutdown_connections 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/115704392 wait complete. 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23800342b0 msgr2=0x7f2380036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23800342b0 0x7f2380036770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5563bcb163e0 tx=0x7f249c005ad0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb2f5b0 msgr2=0x5563bcb2f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb2f5b0 0x5563bcb2f9d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2498000f70 tx=0x7f24980187d0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 shutdown_connections 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23800342b0 0x7f2380036770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/1048168968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bcb2f5b0 0x5563bcb2f9d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 >> 172.21.15.170:0/1048168968 conn(0x5563bcb243c0 msgr2=0x5563bcb24ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 shutdown_connections 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/1048168968 wait complete. 2024-03-28T16:50:22.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb1af50 msgr2=0x5563bcb1d340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5563bcb1af50 0x5563bcb1d340 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bca25670 msgr2=0x5563bcb18990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5563bca25670 0x5563bcb18990 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5563bcb1ec10 msgr2=0x5563bcb21000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5563bcb1ec10 0x5563bcb21000 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f238c0342b0 msgr2=0x7f238c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f238c0342b0 0x7f238c036770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f249c0ae000 tx=0x7f249c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 msgr2=0x5563bcb11d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 --2- 172.21.15.170:0/4239125777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5563bca226b0 0x5563bcb11d10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x5563bcb0bd50 tx=0x7f2498000bd0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a635a640 1 -- 172.21.15.170:0/4239125777 reap_dead start 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 shutdown_connections 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 >> 172.21.15.170:0/4239125777 conn(0x5563bca1ae60 msgr2=0x5563bcb05560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 shutdown_connections 2024-03-28T16:50:22.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.712+0000 7f24a803e980 1 -- 172.21.15.170:0/4239125777 wait complete. 2024-03-28T16:50:22.725 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-03-28T16:50:22.730 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:50:22.730 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-03-28T16:50:22.734 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:50:22.734 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-03-28T16:50:22.735 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-03-28T16:50:22.735 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-03-28T16:50:22.735 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-03-28T16:50:22.801 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:22.802 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:22.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.800+0000 7fa6cc7ad980 1 Processor -- start 2024-03-28T16:50:22.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6cc7ad980 1 -- start start 2024-03-28T16:50:22.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6cc7ad980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x562bcd141760 0x562bcd143b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6cc7ad980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562bcd0874c0 con 0x562bcd0af8c0 2024-03-28T16:50:22.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6cc7ad980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x562bcd04e0f0 con 0x562bcd141760 2024-03-28T16:50:22.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6c9ac7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x562bcd141760 0x562bcd143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6c9ac7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x562bcd141760 0x562bcd143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47526/0 (socket says 172.21.15.170:47526) 2024-03-28T16:50:22.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6ca2c8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x562bcd0af8c0 0x562bcd140040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:49622/0 (socket says 172.21.15.170:49622) 2024-03-28T16:50:22.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/667816582 learned_addr learned my addr 172.21.15.170:0/667816582 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/667816582 >> v1:172.21.15.119:6789/0 conn(0x562bcd0af8c0 legacy=0x562bcd140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/667816582 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcd13ba60 con 0x562bcd141760 2024-03-28T16:50:22.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/667816582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd141760 0x562bcd143b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400a5c0 tx=0x7fa6b401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af119a809d2af70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.804+0000 7fa6c92c6640 1 -- 172.21.15.170:0/667816582 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6b40aa470 con 0x562bcd141760 2024-03-28T16:50:22.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6c92c6640 1 -- 172.21.15.170:0/667816582 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b40aa5d0 con 0x562bcd141760 2024-03-28T16:50:22.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6c92c6640 1 -- 172.21.15.170:0/667816582 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6b40aa890 con 0x562bcd141760 2024-03-28T16:50:22.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/667816582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd141760 msgr2=0x562bcd143b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/667816582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd141760 0x562bcd143b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400a5c0 tx=0x7fa6b401b340 comp rx=0 tx=0).stop 2024-03-28T16:50:22.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/667816582 shutdown_connections 2024-03-28T16:50:22.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/667816582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd141760 0x562bcd143b50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/667816582 >> 172.21.15.170:0/667816582 conn(0x562bcd135720 msgr2=0x562bcd137b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/667816582 shutdown_connections 2024-03-28T16:50:22.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.808+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/667816582 wait complete. 2024-03-28T16:50:22.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 Processor -- start 2024-03-28T16:50:22.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- start start 2024-03-28T16:50:22.823 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd227090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcd04b620 con 0x562bcd0af8c0 2024-03-28T16:50:22.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd227090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd227090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47534/0 (socket says 172.21.15.170:47534) 2024-03-28T16:50:22.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/114337626 learned_addr learned my addr 172.21.15.170:0/114337626 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/114337626 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4005df0 con 0x562bcd0af8c0 2024-03-28T16:50:22.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/114337626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd227090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fb0 tx=0x7fa6bc001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd0e2cb42707408 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa5b8608640 1 -- 172.21.15.170:0/114337626 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc013420 con 0x562bcd0af8c0 2024-03-28T16:50:22.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa5b8608640 1 -- 172.21.15.170:0/114337626 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc013580 con 0x562bcd0af8c0 2024-03-28T16:50:22.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa5b8608640 1 -- 172.21.15.170:0/114337626 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc013860 con 0x562bcd0af8c0 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/114337626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 msgr2=0x562bcd227090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/114337626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd227090 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fb0 tx=0x7fa6bc001940 comp rx=0 tx=0).stop 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/114337626 shutdown_connections 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/114337626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd227090 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/114337626 >> 172.21.15.170:0/114337626 conn(0x562bcd135720 msgr2=0x562bcd222c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/114337626 shutdown_connections 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/114337626 wait complete. 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 Processor -- start 2024-03-28T16:50:22.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- start start 2024-03-28T16:50:22.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd22dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6cc7ad980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcd046e00 con 0x562bcd0af8c0 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd22dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd22dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47544/0 (socket says 172.21.15.170:47544) 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.824+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/2623225502 learned_addr learned my addr 172.21.15.170:0/2623225502 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcd0874c0 con 0x562bcd0af8c0 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd22dee0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x562bcd227f20 tx=0x7fa6bc000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa5b6e05640 1 -- 172.21.15.170:0/2623225502 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc0037a0 con 0x562bcd0af8c0 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa5b6e05640 1 -- 172.21.15.170:0/2623225502 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc003900 con 0x562bcd0af8c0 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa5b6e05640 1 -- 172.21.15.170:0/2623225502 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc003ba0 con 0x562bcd0af8c0 2024-03-28T16:50:22.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562bcd22e420 con 0x562bcd0af8c0 2024-03-28T16:50:22.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bcd22e850 con 0x562bcd0af8c0 2024-03-28T16:50:22.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa5b6e05640 1 -- 172.21.15.170:0/2623225502 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa6bc028460 con 0x562bcd0af8c0 2024-03-28T16:50:22.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa5b6e05640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5b00342b0 0x7fa5b0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa5b6e05640 1 -- 172.21.15.170:0/2623225502 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa6bc0494a0 con 0x562bcd0af8c0 2024-03-28T16:50:22.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5b00342b0 0x7fa5b0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd141760 0x562bcd234b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2350a0 con 0x562bcd141760 2024-03-28T16:50:22.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6caac9640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd141760 0x562bcd234b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5b00342b0 0x7fa5b0036770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40ae000 tx=0x7fa6b40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6caac9640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd141760 0x562bcd234b60 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6caac9640 1 -- 172.21.15.170:0/2623225502 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd141760 2024-03-28T16:50:22.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd235400 con 0x562bcd141760 2024-03-28T16:50:22.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.828+0000 7fa6caac9640 1 -- 172.21.15.170:0/2623225502 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd141760 2024-03-28T16:50:22.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd237120 0x562bcd239510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd239a50 con 0x562bcd237120 2024-03-28T16:50:22.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd237120 0x562bcd239510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd237120 0x562bcd239510 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/2623225502 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd237120 2024-03-28T16:50:22.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd235f60 con 0x562bcd141760 2024-03-28T16:50:22.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6caac9640 1 -- 172.21.15.170:0/2623225502 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd141760 2024-03-28T16:50:22.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x562bcd23ade0 0x562bcd23d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd23d820 con 0x562bcd23ade0 2024-03-28T16:50:22.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x562bcd23ade0 0x562bcd23d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x562bcd23ade0 0x562bcd23d1d0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/2623225502 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd23ade0 2024-03-28T16:50:22.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 Processor -- start 2024-03-28T16:50:22.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 -- start start 2024-03-28T16:50:22.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd24b730 0x562bcd24bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.832+0000 7fa6cc7ad980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bccfdfe30 con 0x562bcd24b730 2024-03-28T16:50:22.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd24b730 0x562bcd24bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd24b730 0x562bcd24bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47558/0 (socket says 172.21.15.170:47558) 2024-03-28T16:50:22.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3745547165 learned_addr learned my addr 172.21.15.170:0/3745547165 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3745547165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcd24c090 con 0x562bcd24b730 2024-03-28T16:50:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd24b730 0x562bcd24bb50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0479a0 tx=0x7fa6bc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa5affff640 1 -- 172.21.15.170:0/3745547165 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc039070 con 0x562bcd24b730 2024-03-28T16:50:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa5affff640 1 -- 172.21.15.170:0/3745547165 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc012c70 con 0x562bcd24b730 2024-03-28T16:50:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562bcd24c2e0 con 0x562bcd24b730 2024-03-28T16:50:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa5affff640 1 -- 172.21.15.170:0/3745547165 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc007a30 con 0x562bcd24b730 2024-03-28T16:50:22.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bcd24ce30 con 0x562bcd24b730 2024-03-28T16:50:22.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa5affff640 1 -- 172.21.15.170:0/3745547165 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa6bc007b90 con 0x562bcd24b730 2024-03-28T16:50:22.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa5affff640 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5a40342b0 0x7fa5a4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa5affff640 1 -- 172.21.15.170:0/3745547165 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa6bc03fbc0 con 0x562bcd24b730 2024-03-28T16:50:22.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5a40342b0 0x7fa5a4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6cc7ad980 1 Processor -- start 2024-03-28T16:50:22.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6cc7ad980 1 -- start start 2024-03-28T16:50:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5a40342b0 0x7fa5a4036770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x562bcd2362c0 tx=0x7fa6b4005ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6cc7ad980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd258950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6cc7ad980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bccfd3790 con 0x562bcd258550 2024-03-28T16:50:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd258950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd258950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47560/0 (socket says 172.21.15.170:47560) 2024-03-28T16:50:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.836+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/2876026962 learned_addr learned my addr 172.21.15.170:0/2876026962 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/2876026962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcd258f20 con 0x562bcd258550 2024-03-28T16:50:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/2876026962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd258950 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc012fd0 tx=0x7fa6bc007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d6438e5e81dc2e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa5adffb640 1 -- 172.21.15.170:0/2876026962 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc035480 con 0x562bcd258550 2024-03-28T16:50:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa5adffb640 1 -- 172.21.15.170:0/2876026962 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc049840 con 0x562bcd258550 2024-03-28T16:50:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa5adffb640 1 -- 172.21.15.170:0/2876026962 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc0123c0 con 0x562bcd258550 2024-03-28T16:50:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2876026962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 msgr2=0x562bcd258950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2876026962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd258950 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc012fd0 tx=0x7fa6bc007e40 comp rx=0 tx=0).stop 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2876026962 shutdown_connections 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2876026962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd258950 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2876026962 >> 172.21.15.170:0/2876026962 conn(0x562bcd254890 msgr2=0x562bcd2528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2876026962 shutdown_connections 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2876026962 wait complete. 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 Processor -- start 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- start start 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd25f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcd25f8f0 con 0x562bcd258550 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd25f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6ca2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd25f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47568/0 (socket says 172.21.15.170:47568) 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 learned_addr learned my addr 172.21.15.170:0/3303437863 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:22.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcd24c090 con 0x562bcd258550 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd25f3b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc046770 tx=0x7fa6bc04cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa59bfff640 1 -- 172.21.15.170:0/3303437863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc063070 con 0x562bcd258550 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562bcd24c090 con 0x562bcd258550 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa59bfff640 1 -- 172.21.15.170:0/3303437863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc035480 con 0x562bcd258550 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa59bfff640 1 -- 172.21.15.170:0/3303437863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6bc035cd0 con 0x562bcd258550 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.840+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bcd24c090 con 0x562bcd258550 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa59bfff640 1 -- 172.21.15.170:0/3303437863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa6bc008a50 con 0x562bcd258550 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa59bfff640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5940342b0 0x7fa594036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa59bfff640 1 -- 172.21.15.170:0/3303437863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa6bc068bc0 con 0x562bcd258550 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5940342b0 0x7fa594036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd26a600 0x562bcd26cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26d210 con 0x562bcd26a600 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6caac9640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd26a600 0x562bcd26cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5940342b0 0x7fa594036770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6b401b380 tx=0x7fa6b40c1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6caac9640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd26a600 0x562bcd26cac0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 realm 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26d5e0 con 0x562bcd26a600 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26de60 con 0x562bcd26a600 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd26eed0 0x562bcd2712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2718d0 con 0x562bcd26eed0 2024-03-28T16:50:22.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd26eed0 0x562bcd2712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.844+0000 7fa6ca2c8640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd26eed0 0x562bcd2712c0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6cc7ad980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6cc7ad980 20 searching for the correct realm 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x562bcd273ab0 0x562bcd275f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd276640 con 0x562bcd273ab0 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x562bcd273ab0 0x562bcd275f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x562bcd273ab0 0x562bcd275f70 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6c9ac7640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa6b40e0040 0x7fa6b40d3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd276640 con 0x7fa6b40e0040 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa6b40e0040 0x7fa6b40d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa6b40e0040 0x7fa6b40d3400 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.848+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x7fa6b40e0040 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x7fa6b40e0040 2024-03-28T16:50:22.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.852+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x7fa6b40e0040 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.856+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x7fa6b40e0040 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd273ab0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26eed0 2024-03-28T16:50:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x562bcd26a600 2024-03-28T16:50:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd26e1c0 con 0x7fa6b40e0040 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd271eb0 con 0x562bcd26a600 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd271eb0 con 0x562bcd26a600 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd271e80 con 0x7fa6b40e0040 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 zone default found 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 4 Realm: () 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 zonegroup default 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.860+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd272700 con 0x562bcd26eed0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd272b00 con 0x562bcd273ab0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd278550 con 0x562bcd273ab0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 20 started sync module instance, tier type = 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd277d40 con 0x7fa6b40e0040 2024-03-28T16:50:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94746125498528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd277cc0 con 0x7fa6b40e0040 2024-03-28T16:50:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2792b0 con 0x562bcd26a600 2024-03-28T16:50:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.864+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94746125498528] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.868+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.868+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94746125502832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd277cc0 con 0x562bcd26a600 2024-03-28T16:50:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.868+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd279970 con 0x562bcd273ab0 2024-03-28T16:50:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.868+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94746125502832] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.868+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.868+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94746125505616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd279d20 con 0x562bcd273ab0 2024-03-28T16:50:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.868+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27a690 con 0x7fa6b40e0040 2024-03-28T16:50:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94746125509376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27ad80 con 0x7fa6b40e0040 2024-03-28T16:50:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27b820 con 0x562bcd26a600 2024-03-28T16:50:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94746125505616] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94746125509376] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94746125513472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27b7e0 con 0x562bcd26a600 2024-03-28T16:50:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.872+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd278f00 con 0x562bcd26eed0 2024-03-28T16:50:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94746125513472] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94746125510624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27c070 con 0x562bcd26eed0 2024-03-28T16:50:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27c870 con 0x562bcd26eed0 2024-03-28T16:50:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94746125510624] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94746125515728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27d070 con 0x562bcd26eed0 2024-03-28T16:50:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.876+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27d9e0 con 0x562bcd26eed0 2024-03-28T16:50:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94746125522240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27de00 con 0x562bcd26eed0 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=0 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=1 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=2 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=3 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=4 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=5 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94746125515728] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=6 2024-03-28T16:50:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94746125522240] v31'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 20 add_watcher() i=7 2024-03-28T16:50:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27cce0 con 0x562bcd26a600 2024-03-28T16:50:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.880+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94746125523296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27e740 con 0x562bcd26a600 2024-03-28T16:50:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.884+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94746125523296] v31'218 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.884+0000 7fa580fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.884+0000 7fa6cc7ad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.884+0000 7fa6cc7ad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.884+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2893f0 con 0x562bcd26a600 2024-03-28T16:50:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.884+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.884+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd289830 con 0x562bcd26eed0 2024-03-28T16:50:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.888+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.888+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd289830 con 0x562bcd273ab0 2024-03-28T16:50:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.888+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.888+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28a200 con 0x562bcd26a600 2024-03-28T16:50:22.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.888+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'219 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.888+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28a580 con 0x7fa6b40e0040 2024-03-28T16:50:22.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.892+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'68 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.892+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28aa20 con 0x562bcd273ab0 2024-03-28T16:50:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.892+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'45 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.892+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2875a0 con 0x562bcd26a600 2024-03-28T16:50:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.892+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.892+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2875a0 con 0x562bcd26eed0 2024-03-28T16:50:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.896+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'46 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.896+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2875a0 con 0x7fa6b40e0040 2024-03-28T16:50:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.896+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.896+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2875a0 con 0x562bcd273ab0 2024-03-28T16:50:22.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.896+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.896+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28b760 con 0x7fa6b40e0040 2024-03-28T16:50:22.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.900+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'69 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.900+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28b760 con 0x562bcd26eed0 2024-03-28T16:50:22.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.900+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'30 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.900+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28bc90 con 0x7fa6b40e0040 2024-03-28T16:50:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.900+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.900+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28bc90 con 0x562bcd26eed0 2024-03-28T16:50:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.904+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'52 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.904+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2874a0 con 0x562bcd26a600 2024-03-28T16:50:22.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.904+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'220 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.904+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28b760 con 0x562bcd273ab0 2024-03-28T16:50:22.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.908+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'62 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.908+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28b760 con 0x7fa6b40e0040 2024-03-28T16:50:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.908+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'70 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.908+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28b760 con 0x562bcd273ab0 2024-03-28T16:50:22.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.908+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.908+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28b760 con 0x562bcd26a600 2024-03-28T16:50:22.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.912+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.912+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd26a600 2024-03-28T16:50:22.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.912+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'221 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.912+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd273ab0 2024-03-28T16:50:22.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.912+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'63 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.912+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x7fa6b40e0040 2024-03-28T16:50:22.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.916+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'36 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.916+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd273ab0 2024-03-28T16:50:22.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.916+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'29 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.916+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd273ab0 2024-03-28T16:50:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.916+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.916+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd273ab0 2024-03-28T16:50:22.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.920+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'22 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.920+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd273ab0 2024-03-28T16:50:22.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.920+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'46 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.920+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd26eed0 2024-03-28T16:50:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.920+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'30 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.920+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd273ab0 2024-03-28T16:50:22.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.924+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.924+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd26eed0 2024-03-28T16:50:22.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.924+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'47 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.924+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd273ab0 2024-03-28T16:50:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.928+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'64 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.928+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x562bcd26eed0 2024-03-28T16:50:22.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.928+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'53 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.928+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd28c320 con 0x7fa6b40e0040 2024-03-28T16:50:22.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa4fb7fe640 20 reqs_thread_entry: start 2024-03-28T16:50:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 20 Filter name: none 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa4f97fa640 20 reqs_thread_entry: start 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 20 rados->read ofs=0 len=0 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd285a60 con 0x7fa6b40e0040 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(94 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd285f60 con 0x562bcd26a600 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(95 bud.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-03-28T16:50:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-03-28T16:50:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify cookie 94746125555136 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd285a40 con 0x562bcd26eed0 2024-03-28T16:50:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== watch-notify(notify (1) cookie 94746125510624 notify 133143986217 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x562bcd258f20 con 0x562bcd26eed0 2024-03-28T16:50:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa5adffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986217 cookie 94746125510624 notifier 4746 bl.length()=170 2024-03-28T16:50:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa5adffb640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fa590003d60 con 0x562bcd26eed0 2024-03-28T16:50:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.932+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(96 notify.5 [notify cookie 94746125555136 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== watch-notify(notify_complete (2) cookie 94746125555136 notify 133143986217 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562bcd22e850 con 0x562bcd26eed0 2024-03-28T16:50:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(97 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:98 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd286890 con 0x562bcd26a600 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(98 NX5QOQKC6BH2IDN8HC7A [delete] v31'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6cc7ad980 10 removing email index: 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6cc7ad980 10 removing user buckets index 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6cc7ad980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:99 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94746125558688 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2855c0 con 0x562bcd26eed0 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== watch-notify(notify (1) cookie 94746125515728 notify 133143986218 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x562bcd22e850 con 0x562bcd26eed0 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(99 notify.6 [notify cookie 94746125558688 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa5ad7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986218 cookie 94746125515728 notifier 4746 bl.length()=160 2024-03-28T16:50:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa5ad7fa640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fa5000032c0 con 0x562bcd26eed0 2024-03-28T16:50:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== watch-notify(notify_complete (2) cookie 94746125558688 notify 133143986218 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7fa6bc02d070 con 0x562bcd26eed0 2024-03-28T16:50:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:101 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd2855c0 con 0x562bcd26a600 2024-03-28T16:50:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.936+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(100 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(101 bud.buckets [delete] v31'6 uv4 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6cc7ad980 10 removing user index: bud 2024-03-28T16:50:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6cc7ad980 10 removing default.rgw.meta+users.uid+bud from cache 2024-03-28T16:50:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6cc7ad980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-03-28T16:50:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [notify cookie 94746125556080 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd286890 con 0x562bcd26eed0 2024-03-28T16:50:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== watch-notify(notify (1) cookie 94746125510624 notify 133143986219 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x7fa6bc02d070 con 0x562bcd26eed0 2024-03-28T16:50:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa5ad7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986219 cookie 94746125510624 notifier 4746 bl.length()=152 2024-03-28T16:50:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(102 notify.5 [notify cookie 94746125556080 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa5ad7fa640 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7fa5000032c0 con 0x562bcd26eed0 2024-03-28T16:50:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(103 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== watch-notify(notify_complete (2) cookie 94746125556080 notify 133143986219 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562bcd22e420 con 0x562bcd26eed0 2024-03-28T16:50:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:104 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd285970 con 0x7fa6b40e0040 2024-03-28T16:50:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(104 bud [call,call,delete] v31'3 uv3 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa580fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.940+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94746125523296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd285970 con 0x562bcd26a600 2024-03-28T16:50:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.944+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94746125523296] v31'222 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.944+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94746125498528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27d960 con 0x7fa6b40e0040 2024-03-28T16:50:22.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.944+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94746125498528] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.944+0000 7fa6cc7ad980 20 remove_watcher() i=0 2024-03-28T16:50:22.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.944+0000 7fa6cc7ad980 2 removed watcher, disabling cache 2024-03-28T16:50:22.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.944+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94746125502832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27d960 con 0x562bcd26a600 2024-03-28T16:50:22.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94746125502832] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6cc7ad980 20 remove_watcher() i=1 2024-03-28T16:50:22.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94746125505616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd278970 con 0x562bcd273ab0 2024-03-28T16:50:22.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6c9ac7640 1 -- 172.21.15.170:0/3303437863 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94746125505616] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6b40b0020 con 0x562bcd273ab0 2024-03-28T16:50:22.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6cc7ad980 20 remove_watcher() i=2 2024-03-28T16:50:22.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94746125509376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27d960 con 0x7fa6b40e0040 2024-03-28T16:50:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94746125509376] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x7fa6b40e0040 2024-03-28T16:50:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6cc7ad980 20 remove_watcher() i=3 2024-03-28T16:50:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.948+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94746125513472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd278970 con 0x562bcd26a600 2024-03-28T16:50:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.952+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94746125513472] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6c40083f0 con 0x562bcd26a600 2024-03-28T16:50:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.952+0000 7fa6cc7ad980 20 remove_watcher() i=4 2024-03-28T16:50:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.952+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94746125510624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27a300 con 0x562bcd26eed0 2024-03-28T16:50:22.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.952+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94746125510624] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.952+0000 7fa6cc7ad980 20 remove_watcher() i=5 2024-03-28T16:50:22.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.952+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94746125515728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27d960 con 0x562bcd26eed0 2024-03-28T16:50:22.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94746125515728] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 20 remove_watcher() i=6 2024-03-28T16:50:22.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94746125522240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x562bcd27b300 con 0x562bcd26eed0 2024-03-28T16:50:22.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6ca2c8640 1 -- 172.21.15.170:0/3303437863 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94746125522240] v31'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fa6bc024020 con 0x562bcd26eed0 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 20 remove_watcher() i=7 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd26eed0 msgr2=0x562bcd2712c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd26eed0 0x562bcd2712c0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd26a600 msgr2=0x562bcd26cac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd26a600 0x562bcd26cac0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa6b40e0040 msgr2=0x7fa6b40d3400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fa6b40e0040 0x7fa6b40d3400 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x562bcd273ab0 msgr2=0x562bcd275f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x562bcd273ab0 0x562bcd275f70 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5940342b0 msgr2=0x7fa594036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5940342b0 0x7fa594036770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6b401b380 tx=0x7fa6b40c1040 comp rx=0 tx=0).stop 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6caac9640 1 -- 172.21.15.170:0/3303437863 reap_dead start 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 msgr2=0x562bcd25f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.956+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd25f3b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc046770 tx=0x7fa6bc04cee0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 shutdown_connections 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3303437863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd258550 0x562bcd25f3b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 >> 172.21.15.170:0/3303437863 conn(0x562bcd254890 msgr2=0x562bcd2528c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 shutdown_connections 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3303437863 wait complete. 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5a40342b0 msgr2=0x7fa5a4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5a40342b0 0x7fa5a4036770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x562bcd2362c0 tx=0x7fa6b4005ad0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd24b730 msgr2=0x562bcd24bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd24b730 0x562bcd24bb50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0479a0 tx=0x7fa6bc011b00 comp rx=0 tx=0).stop 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 shutdown_connections 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5a40342b0 0x7fa5a4036770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/3745547165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd24b730 0x562bcd24bb50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 >> 172.21.15.170:0/3745547165 conn(0x562bcd240540 msgr2=0x562bcd241050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 shutdown_connections 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/3745547165 wait complete. 2024-03-28T16:50:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd237120 msgr2=0x562bcd239510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x562bcd237120 0x562bcd239510 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd141760 msgr2=0x562bcd234b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x562bcd141760 0x562bcd234b60 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x562bcd23ade0 msgr2=0x562bcd23d1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x562bcd23ade0 0x562bcd23d1d0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5b00342b0 msgr2=0x7fa5b0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5b00342b0 0x7fa5b0036770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40ae000 tx=0x7fa6b40a6000 comp rx=0 tx=0).stop 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 msgr2=0x562bcd22dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 --2- 172.21.15.170:0/2623225502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bcd0af8c0 0x562bcd22dee0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x562bcd227f20 tx=0x7fa6bc000f50 comp rx=0 tx=0).stop 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6caac9640 1 -- 172.21.15.170:0/2623225502 reap_dead start 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 shutdown_connections 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 >> 172.21.15.170:0/2623225502 conn(0x562bcd135720 msgr2=0x562bcd221730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 shutdown_connections 2024-03-28T16:50:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:22.960+0000 7fa6cc7ad980 1 -- 172.21.15.170:0/2623225502 wait complete. 2024-03-28T16:50:22.971 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-03-28T16:50:22.971 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-03-28T16:50:22.971 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-03-28T16:50:23.048 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:23.048 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:23.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d2be0980 1 Processor -- start 2024-03-28T16:50:23.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d2be0980 1 -- start start 2024-03-28T16:50:23.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d2be0980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556654249140 0x556654249520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d2be0980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5566541934c0 con 0x556654249a60 2024-03-28T16:50:23.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d2be0980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55665415a0f0 con 0x556654249140 2024-03-28T16:50:23.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d06fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556654249140 0x556654249520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d06fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x556654249140 0x556654249520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47584/0 (socket says 172.21.15.170:47584) 2024-03-28T16:50:23.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d06fb640 1 -- 172.21.15.170:0/1858907060 learned_addr learned my addr 172.21.15.170:0/1858907060 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:23.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d06fb640 1 -- 172.21.15.170:0/1858907060 >> v1:172.21.15.119:6789/0 conn(0x556654249a60 legacy=0x55665424a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.048+0000 7f65d06fb640 1 -- 172.21.15.170:0/1858907060 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665424ac40 con 0x556654249140 2024-03-28T16:50:23.052 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d06fb640 1 --2- 172.21.15.170:0/1858907060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654249520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65c800a4e0 tx=0x7f65c801b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ee2c7b459f7500 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65cf6f9640 1 -- 172.21.15.170:0/1858907060 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80aa540 con 0x556654249140 2024-03-28T16:50:23.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65cf6f9640 1 -- 172.21.15.170:0/1858907060 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80a6070 con 0x556654249140 2024-03-28T16:50:23.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65cf6f9640 1 -- 172.21.15.170:0/1858907060 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80aa850 con 0x556654249140 2024-03-28T16:50:23.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d2be0980 1 -- 172.21.15.170:0/1858907060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 msgr2=0x556654249520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d2be0980 1 --2- 172.21.15.170:0/1858907060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654249520 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65c800a4e0 tx=0x7f65c801b090 comp rx=0 tx=0).stop 2024-03-28T16:50:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d2be0980 1 -- 172.21.15.170:0/1858907060 shutdown_connections 2024-03-28T16:50:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d2be0980 1 --2- 172.21.15.170:0/1858907060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654249520 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d2be0980 1 -- 172.21.15.170:0/1858907060 >> 172.21.15.170:0/1858907060 conn(0x5566541bc860 msgr2=0x5566541bcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d2be0980 1 -- 172.21.15.170:0/1858907060 shutdown_connections 2024-03-28T16:50:23.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.052+0000 7f65d2be0980 1 -- 172.21.15.170:0/1858907060 wait complete. 2024-03-28T16:50:23.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 Processor -- start 2024-03-28T16:50:23.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- start start 2024-03-28T16:50:23.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654333610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556654157620 con 0x556654249140 2024-03-28T16:50:23.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654333610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654333610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47586/0 (socket says 172.21.15.170:47586) 2024-03-28T16:50:23.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 -- 172.21.15.170:0/1362589382 learned_addr learned my addr 172.21.15.170:0/1362589382 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:23.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 -- 172.21.15.170:0/1362589382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c800a1c0 con 0x556654249140 2024-03-28T16:50:23.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 --2- 172.21.15.170:0/1362589382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654333610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f65c8017ca0 tx=0x7f65c80033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a90c4df02b376e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f64be60c640 1 -- 172.21.15.170:0/1362589382 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80aa780 con 0x556654249140 2024-03-28T16:50:23.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f64be60c640 1 -- 172.21.15.170:0/1362589382 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80aa8e0 con 0x556654249140 2024-03-28T16:50:23.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f64be60c640 1 -- 172.21.15.170:0/1362589382 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80aab80 con 0x556654249140 2024-03-28T16:50:23.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- 172.21.15.170:0/1362589382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 msgr2=0x556654333610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 --2- 172.21.15.170:0/1362589382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654333610 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f65c8017ca0 tx=0x7f65c80033b0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- 172.21.15.170:0/1362589382 shutdown_connections 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 --2- 172.21.15.170:0/1362589382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x556654333610 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- 172.21.15.170:0/1362589382 >> 172.21.15.170:0/1362589382 conn(0x5566541bc860 msgr2=0x55665432f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- 172.21.15.170:0/1362589382 shutdown_connections 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- 172.21.15.170:0/1362589382 wait complete. 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 Processor -- start 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- start start 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x55665433a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d2be0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556654152e00 con 0x556654249140 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x55665433a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x55665433a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47592/0 (socket says 172.21.15.170:47592) 2024-03-28T16:50:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.068+0000 7f65d06fb640 1 -- 172.21.15.170:0/392499926 learned_addr learned my addr 172.21.15.170:0/392499926 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d06fb640 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55665424ac40 con 0x556654249140 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d06fb640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x55665433a360 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f65c801aec0 tx=0x7f65c8017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f64bce09640 1 -- 172.21.15.170:0/392499926 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80c4070 con 0x556654249140 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55665433a8a0 con 0x556654249140 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f64bce09640 1 -- 172.21.15.170:0/392499926 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80b5650 con 0x556654249140 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f64bce09640 1 -- 172.21.15.170:0/392499926 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80b5910 con 0x556654249140 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55665433acd0 con 0x556654249140 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f64bce09640 1 -- 172.21.15.170:0/392499926 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f65c80a8070 con 0x556654249140 2024-03-28T16:50:23.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f64bce09640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64b80342b0 0x7f64b8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f64bce09640 1 -- 172.21.15.170:0/392499926 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f65c80e3710 con 0x556654249140 2024-03-28T16:50:23.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65cfefa640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64b80342b0 0x7f64b8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654249a60 0x556654340f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543414d0 con 0x556654249a60 2024-03-28T16:50:23.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d0efc640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654249a60 0x556654340f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65cfefa640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64b80342b0 0x7f64b8036770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f65c4001e20 tx=0x7f65c40013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d0efc640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654249a60 0x556654340f90 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d0efc640 1 -- 172.21.15.170:0/392499926 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654249a60 2024-03-28T16:50:23.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654341830 con 0x556654249a60 2024-03-28T16:50:23.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d0efc640 1 -- 172.21.15.170:0/392499926 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f65c00083f0 con 0x556654249a60 2024-03-28T16:50:23.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556654343550 0x556654345940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654345e80 con 0x556654343550 2024-03-28T16:50:23.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.072+0000 7f65d06fb640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556654343550 0x556654345940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d06fb640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556654343550 0x556654345940 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d06fb640 1 -- 172.21.15.170:0/392499926 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f65c80d1790 con 0x556654343550 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654342390 con 0x556654249a60 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d0efc640 1 -- 172.21.15.170:0/392499926 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f65c00083f0 con 0x556654249a60 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5566543471f0 0x5566543495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654349c80 con 0x5566543471f0 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65cfefa640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5566543471f0 0x5566543495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65cfefa640 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5566543471f0 0x5566543495e0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65cfefa640 1 -- 172.21.15.170:0/392499926 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f65c4010630 con 0x5566543471f0 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d2be0980 1 Processor -- start 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d2be0980 1 -- start start 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d2be0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654357bb0 0x556654357fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d2be0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566540ebe30 con 0x556654357bb0 2024-03-28T16:50:23.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654357bb0 0x556654357fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654357bb0 0x556654357fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47600/0 (socket says 172.21.15.170:47600) 2024-03-28T16:50:23.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d06fb640 1 -- 172.21.15.170:0/416539630 learned_addr learned my addr 172.21.15.170:0/416539630 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:23.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.076+0000 7f65d06fb640 1 -- 172.21.15.170:0/416539630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556654358510 con 0x556654357bb0 2024-03-28T16:50:23.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d06fb640 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654357bb0 0x556654357fd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f65c80c7e30 tx=0x7f65c80c9e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64b5ffb640 1 -- 172.21.15.170:0/416539630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80a9e10 con 0x556654357bb0 2024-03-28T16:50:23.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64b5ffb640 1 -- 172.21.15.170:0/416539630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80d5040 con 0x556654357bb0 2024-03-28T16:50:23.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556654358760 con 0x556654357bb0 2024-03-28T16:50:23.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64b5ffb640 1 -- 172.21.15.170:0/416539630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80e3be0 con 0x556654357bb0 2024-03-28T16:50:23.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566543592b0 con 0x556654357bb0 2024-03-28T16:50:23.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64b5ffb640 1 -- 172.21.15.170:0/416539630 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f65c80c1ac0 con 0x556654357bb0 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64b5ffb640 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a80342b0 0x7f64a8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64b5ffb640 1 -- 172.21.15.170:0/416539630 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f65c80edba0 con 0x556654357bb0 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65cfefa640 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a80342b0 0x7f64a8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d2be0980 1 Processor -- start 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d2be0980 1 -- start start 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65cfefa640 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a80342b0 0x7f64a8036770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55665433e9c0 tx=0x7f65c4006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d2be0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x556654364ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d2be0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566540df790 con 0x556654364ac0 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x556654364ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x556654364ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47606/0 (socket says 172.21.15.170:47606) 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d06fb640 1 -- 172.21.15.170:0/3479169939 learned_addr learned my addr 172.21.15.170:0/3479169939 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:23.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d06fb640 1 -- 172.21.15.170:0/3479169939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566543654b0 con 0x556654364ac0 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f65d06fb640 1 --2- 172.21.15.170:0/3479169939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x556654364ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f65c80a9f90 tx=0x7f65c80dc860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa156154bb00743b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64af7fe640 1 -- 172.21.15.170:0/3479169939 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80b5df0 con 0x556654364ac0 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64af7fe640 1 -- 172.21.15.170:0/3479169939 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80c1dc0 con 0x556654364ac0 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.080+0000 7f64af7fe640 1 -- 172.21.15.170:0/3479169939 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80dc570 con 0x556654364ac0 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/3479169939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 msgr2=0x556654364ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 --2- 172.21.15.170:0/3479169939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x556654364ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f65c80a9f90 tx=0x7f65c80dc860 comp rx=0 tx=0).stop 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/3479169939 shutdown_connections 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 --2- 172.21.15.170:0/3479169939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x556654364ee0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/3479169939 >> 172.21.15.170:0/3479169939 conn(0x556654360e00 msgr2=0x55665435ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/3479169939 shutdown_connections 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/3479169939 wait complete. 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 Processor -- start 2024-03-28T16:50:23.085 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- start start 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x55665436b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55665436be40 con 0x556654364ac0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x55665436b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d06fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x55665436b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47610/0 (socket says 172.21.15.170:47610) 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 learned_addr learned my addr 172.21.15.170:0/420342365 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566543592b0 con 0x556654364ac0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d06fb640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x55665436b900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ac010 tx=0x7f65c80ebf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f64adffb640 1 -- 172.21.15.170:0/420342365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c8101070 con 0x556654364ac0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5566543592b0 con 0x556654364ac0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f64adffb640 1 -- 172.21.15.170:0/420342365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80b5df0 con 0x556654364ac0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f64adffb640 1 -- 172.21.15.170:0/420342365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c80ddc50 con 0x556654364ac0 2024-03-28T16:50:23.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55665436c280 con 0x556654364ac0 2024-03-28T16:50:23.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f64adffb640 1 -- 172.21.15.170:0/420342365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f65c8105070 con 0x556654364ac0 2024-03-28T16:50:23.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f64adffb640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a00342b0 0x7f64a0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f64adffb640 1 -- 172.21.15.170:0/420342365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f65c80ccde0 con 0x556654364ac0 2024-03-28T16:50:23.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65cfefa640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a00342b0 0x7f64a0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654376bd0 0x556654379090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543797e0 con 0x556654376bd0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.084+0000 7f65d0efc640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654376bd0 0x556654379090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65cfefa640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a00342b0 0x7f64a0036770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f65c4010fd0 tx=0x7f65c4011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d0efc640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654376bd0 0x556654379090 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 realm 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654379bb0 con 0x556654376bd0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a760 con 0x556654376bd0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55665437b110 0x55665437d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437dba0 con 0x55665437b110 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d06fb640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55665437b110 0x55665437d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d06fb640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55665437b110 0x55665437d500 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f65c80d1790 con 0x55665437b110 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:23.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 20 searching for the correct realm 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55665437fd60 0x556654382220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543828a0 con 0x55665437fd60 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65cfefa640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55665437fd60 0x556654382220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65cfefa640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55665437fd60 0x556654382220 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65cfefa640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65c400f3d0 0x7f65c400f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543828a0 con 0x7f65c400f3d0 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.088+0000 7f65d0efc640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65c400f3d0 0x7f65c400f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:23.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65c400f3d0 0x7f65c400f7f0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:23.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c80d1790 con 0x55665437b110 2024-03-28T16:50:23.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x7f65c400f3d0 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.092+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x7f65c400f3d0 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x7f65c400f3d0 2024-03-28T16:50:23.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.096+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x7f65c400f3d0 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437fd60 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x55665437b110 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.106 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x556654376bd0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437a400 con 0x7f65c400f3d0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437e160 con 0x556654376bd0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437e160 con 0x556654376bd0 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437e110 con 0x7f65c400f3d0 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 zone default found 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 4 Realm: () 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 zonegroup default 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.100+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437e9b0 con 0x55665437b110 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665437ede0 con 0x55665437fd60 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543847e0 con 0x55665437fd60 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 20 started sync module instance, tier type = 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654383fa0 con 0x7f65c400f3d0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93897987996416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654384350 con 0x7f65c400f3d0 2024-03-28T16:50:23.109 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654385540 con 0x556654376bd0 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.104+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 93897987996416] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93897988000768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543854c0 con 0x556654376bd0 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654385f90 con 0x55665437fd60 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93897988000768] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93897988001824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654385440 con 0x55665437fd60 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654386c80 con 0x7f65c400f3d0 2024-03-28T16:50:23.111 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93897988008176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543874d0 con 0x7f65c400f3d0 2024-03-28T16:50:23.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.108+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654387e80 con 0x556654376bd0 2024-03-28T16:50:23.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 93897988001824] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 93897988008176] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93897988006720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654386fe0 con 0x556654376bd0 2024-03-28T16:50:23.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543886d0 con 0x55665437b110 2024-03-28T16:50:23.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93897988006720] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93897988014640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654386fe0 con 0x55665437b110 2024-03-28T16:50:23.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.112+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654388ed0 con 0x55665437b110 2024-03-28T16:50:23.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.116+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93897988014640] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.116+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.116+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93897988012448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543896d0 con 0x55665437b110 2024-03-28T16:50:23.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.116+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665438a040 con 0x55665437b110 2024-03-28T16:50:23.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.116+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93897988012448] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93897988019008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665438a040 con 0x55665437b110 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=0 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=1 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=2 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=3 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=4 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=5 2024-03-28T16:50:23.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=6 2024-03-28T16:50:23.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93897988019008] v31'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 add_watcher() i=7 2024-03-28T16:50:23.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:23.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665438d390 con 0x556654376bd0 2024-03-28T16:50:23.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93897988033968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665438a1b0 con 0x556654376bd0 2024-03-28T16:50:23.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93897988033968] v31'223 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f64867e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:23.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:23.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:23.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.120+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654395a00 con 0x556654376bd0 2024-03-28T16:50:23.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.124+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.124+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654395e40 con 0x55665437b110 2024-03-28T16:50:23.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.124+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.124+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654395e40 con 0x55665437fd60 2024-03-28T16:50:23.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665438c220 con 0x556654376bd0 2024-03-28T16:50:23.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'224 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665438c220 con 0x7f65c400f3d0 2024-03-28T16:50:23.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'71 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55665438c220 con 0x55665437fd60 2024-03-28T16:50:23.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'47 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.128+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654397550 con 0x556654376bd0 2024-03-28T16:50:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.132+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'29 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.132+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654397550 con 0x55665437b110 2024-03-28T16:50:23.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.132+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'48 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.132+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654397aa0 con 0x7f65c400f3d0 2024-03-28T16:50:23.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.136+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.136+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654397aa0 con 0x55665437fd60 2024-03-28T16:50:23.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.136+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.136+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654397fd0 con 0x7f65c400f3d0 2024-03-28T16:50:23.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.136+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'72 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.136+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654397fd0 con 0x55665437b110 2024-03-28T16:50:23.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.140+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'31 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.140+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654398500 con 0x7f65c400f3d0 2024-03-28T16:50:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.140+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.140+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654398500 con 0x55665437b110 2024-03-28T16:50:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.140+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'54 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.140+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543989a0 con 0x556654376bd0 2024-03-28T16:50:23.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.144+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'225 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.144+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543989a0 con 0x55665437fd60 2024-03-28T16:50:23.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.144+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'65 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.144+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654398ed0 con 0x7f65c400f3d0 2024-03-28T16:50:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.144+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'73 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.144+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654398ed0 con 0x55665437fd60 2024-03-28T16:50:23.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.148+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.147 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.148+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x556654376bd0 2024-03-28T16:50:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.148+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.148+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x556654376bd0 2024-03-28T16:50:23.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.152+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'226 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.152+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437fd60 2024-03-28T16:50:23.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.152+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'66 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.152+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x7f65c400f3d0 2024-03-28T16:50:23.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.152+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'37 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.152+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437fd60 2024-03-28T16:50:23.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.156+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'30 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.156+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437fd60 2024-03-28T16:50:23.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.156+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.156+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437fd60 2024-03-28T16:50:23.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.156+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'23 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.156+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437fd60 2024-03-28T16:50:23.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.160+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'48 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.160+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437b110 2024-03-28T16:50:23.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.160+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'31 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.160+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437fd60 2024-03-28T16:50:23.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.160+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.160+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437b110 2024-03-28T16:50:23.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.164+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'49 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.164+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437fd60 2024-03-28T16:50:23.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.164+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'67 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.166 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.164+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x55665437b110 2024-03-28T16:50:23.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.168+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'55 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.168+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543990a0 con 0x7f65c400f3d0 2024-03-28T16:50:23.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f64857e2640 20 reqs_thread_entry: start 2024-03-28T16:50:23.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:23.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 20 Filter name: none 2024-03-28T16:50:23.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f63f6ffd640 20 reqs_thread_entry: start 2024-03-28T16:50:23.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:50:23.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543940b0 con 0x55665437b110 2024-03-28T16:50:23.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(94 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 20 rados_obj.operate() r=0 bl.length=346 2024-03-28T16:50:23.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-03-28T16:50:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:50:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654394080 con 0x55665437b110 2024-03-28T16:50:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(95 fud.buckets [call out=186b] v0'0 uv9 ondisk = 0) v8 ==== 155+0+186 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 : miss 2024-03-28T16:50:23.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.172+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:96 8.f 8:f580a709:root::.bucket.meta.myfoo7%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.10:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543b5220 con 0x55665437fd60 2024-03-28T16:50:23.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(96 .bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 [call out=48b,stat out=16b,read 0~355 out=355b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 334+0+637 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 20 rados_obj.operate() r=0 bl.length=355 2024-03-28T16:50:23.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 info.flags=0x17 2024-03-28T16:50:23.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 to cache LRU end 2024-03-28T16:50:23.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:23.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:50:23.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 2024-03-28T16:50:23.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:97 4.0 4:024013e5:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432810 con 0x55665437fd60 2024-03-28T16:50:23.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(97 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:98 4.f 4:f161ffc7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432890 con 0x556654376bd0 2024-03-28T16:50:23.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(98 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:99 4.1d 4:bbf0e2f7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432890 con 0x55665437fd60 2024-03-28T16:50:23.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(99 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:100 4.16 4:6a255352:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432890 con 0x55665437fd60 2024-03-28T16:50:23.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(100 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 4.26 4:64d736ec:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432890 con 0x55665437b110 2024-03-28T16:50:23.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(101 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.176+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:102 4.6 4:6104770f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432890 con 0x7f65c400f3d0 2024-03-28T16:50:23.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(102 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5 [call out=378b] v0'0 uv7 ondisk = 0) v8 ==== 195+0+378 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 10 cls_bucket_list_unordered: got fourteen 2024-03-28T16:50:23.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:103 4.5 4:a0031e4f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432890 con 0x55665437fd60 2024-03-28T16:50:23.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(103 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:104 4.3 4:c06cfb55:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432820 con 0x556654376bd0 2024-03-28T16:50:23.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(104 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 4.2c 4:34fe866c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432820 con 0x556654376bd0 2024-03-28T16:50:23.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(105 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:106 4.10 4:082a7bbd:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432820 con 0x55665437fd60 2024-03-28T16:50:23.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 31 ==== osd_op_reply(106 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:107 4.13 4:c8f6006a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654432820 con 0x7f65c400f3d0 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(107 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 196+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 20 get_obj_state: octx=0x7f65c0008c30 obj=myfoo7:fourteen state=0x556654430878 s->prefetch_data=0 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:108 3.16 3:69f58a22:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.10_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654431680 con 0x7f65c400f3d0 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(108 b88c2392-6ef8-4660-b080-19480b748fc0.4221.10_fourteen [getxattrs out=858b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 239+0+874 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 10 manifest: total_size = 8 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.10821529145840220275 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 20 get_obj_state: octx=0x7f65c0008c30 obj=myfoo7:fourteen state=0x556654430878 s->prefetch_data=0 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 20 prepare_atomic_modification: state is not atomic. state=0x556654430878 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5 2024-03-28T16:50:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.180+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:109 4.6 4:6104770f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=188b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654431ef0 con 0x7f65c400f3d0 2024-03-28T16:50:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(109 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5 [stat,call,call] v31'8 uv8 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:110 3.16 3:69f58a22:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.10_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654431ef0 con 0x7f65c400f3d0 2024-03-28T16:50:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(110 b88c2392-6ef8-4660-b080-19480b748fc0.4221.10_fourteen [call] v31'4 uv4 ondisk = 0) v8 ==== 197+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:111 4.6 4:6104770f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=265b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654431ef0 con 0x7f65c400f3d0 2024-03-28T16:50:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 20 list_objects_ordered: starting attempt 1 2024-03-28T16:50:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-28T16:50:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:112 4.0 4:024013e5:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654431ef0 con 0x55665437fd60 2024-03-28T16:50:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:113 4.f 4:f161ffc7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543bd420 con 0x556654376bd0 2024-03-28T16:50:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:114 4.1d 4:bbf0e2f7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654431ef0 con 0x55665437fd60 2024-03-28T16:50:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 32 ==== osd_op_reply(112 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:115 4.16 4:6a255352:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543bd420 con 0x55665437fd60 2024-03-28T16:50:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(113 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:116 4.26 4:64d736ec:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543babf0 con 0x55665437b110 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:117 4.6 4:6104770f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430630 con 0x7f65c400f3d0 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:118 4.5 4:a0031e4f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543c5520 con 0x55665437fd60 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:119 4.3 4:c06cfb55:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543c8000 con 0x556654376bd0 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:120 4.2c 4:34fe866c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543ca8a0 con 0x556654376bd0 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:121 4.10 4:082a7bbd:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cd3c0 con 0x55665437fd60 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 33 ==== osd_op_reply(114 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:122 4.13 4:c8f6006a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543d01f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 34 ==== osd_op_reply(115 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.184+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 35 ==== osd_op_reply(118 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 36 ==== osd_op_reply(121 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(119 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(120 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(111 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5 [stat,call,call] v31'9 uv9 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f64aeffd640 20 handle_completion(): completion ok for obj=fourteen 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 25 ==== osd_op_reply(122 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 196+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(116 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 26 ==== osd_op_reply(117 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5 [call out=124b] v0'0 uv9 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[b88c2392-6ef8-4660-b080-19480b748fc0.4221.10])"; uploads.size()=0; is_truncated=0 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:123 4.0 4:024013e5:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cd340 con 0x55665437fd60 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:124 4.f 4:f161ffc7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543be940 con 0x556654376bd0 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:125 4.1d 4:bbf0e2f7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cc200 con 0x55665437fd60 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:126 4.16 4:6a255352:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cc5a0 con 0x55665437fd60 2024-03-28T16:50:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:127 4.26 4:64d736ec:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cce80 con 0x55665437b110 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:128 4.6 4:6104770f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cd340 con 0x7f65c400f3d0 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:129 4.5 4:a0031e4f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543be940 con 0x55665437fd60 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:130 4.3 4:c06cfb55:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543be940 con 0x556654376bd0 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:131 4.2c 4:34fe866c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cce80 con 0x556654376bd0 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:132 4.10 4:082a7bbd:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543ce8b0 con 0x55665437fd60 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:133 4.13 4:c8f6006a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cd340 con 0x7f65c400f3d0 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 37 ==== osd_op_reply(123 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 38 ==== osd_op_reply(125 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 39 ==== osd_op_reply(126 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 40 ==== osd_op_reply(132 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 41 ==== osd_op_reply(129 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 27 ==== osd_op_reply(128 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5 [call out=124b] v0'0 uv9 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(124 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(131 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(130 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 28 ==== osd_op_reply(133 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 196+0+85 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(127 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.188+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:134 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430a60 con 0x55665437b110 2024-03-28T16:50:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(134 fud.buckets [call] v31'10 uv10 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 10 cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-03-28T16:50:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:135 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430a60 con 0x7f65c400f3d0 2024-03-28T16:50:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 29 ==== osd_op_reply(135 myfoo7 [call out=48b,read 0~168 out=168b] v0'0 uv46 ondisk = 0) v8 ==== 192+0+216 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 20 rados_obj.operate() r=0 bl.length=168 2024-03-28T16:50:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 10 cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x11 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 10 adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 10 removing default.rgw.meta+root+myfoo7 from cache 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:136 7.e 7:7759931f:::notify.3:head [notify cookie 93897988285776 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430a60 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 30 ==== watch-notify(notify (1) cookie 93897988008176 notify 133143986192 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x7f65c004c040 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f64aeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 93897988008176 notifier 4753 bl.length()=150 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 31 ==== osd_op_reply(136 notify.3 [notify cookie 93897988285776 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f64aeffd640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f64040032c0 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 32 ==== osd_op_reply(137 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 33 ==== watch-notify(notify_complete (2) cookie 93897988285776 notify 133143986192 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f65c003c000 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:138 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430df0 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 34 ==== osd_op_reply(138 myfoo7 [call,call,delete] v31'47 uv47 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 from cache 2024-03-28T16:50:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10, ofs0, ns] 2024-03-28T16:50:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.192+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:139 7.9 7:93e5b521:::notify.7:head [notify cookie 93897988285776 in=220b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430a10 con 0x55665437b110 2024-03-28T16:50:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== watch-notify(notify (1) cookie 93897988019008 notify 133143986220 ret 0) v3 ==== 250+0+0 (crc 0 0 0) 0x5566543654b0 con 0x55665437b110 2024-03-28T16:50:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f64af7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986220 cookie 93897988019008 notifier 4753 bl.length()=208 2024-03-28T16:50:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f64af7fe640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:140 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f6494003d60 con 0x55665437b110 2024-03-28T16:50:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(139 notify.7 [notify cookie 93897988285776 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(140 notify.7 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== watch-notify(notify_complete (2) cookie 93897988285776 notify 133143986220 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f65c80e7030 con 0x55665437b110 2024-03-28T16:50:23.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:141 8.f 8:f580a709:root::.bucket.meta.myfoo7%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.10:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430df0 con 0x55665437fd60 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 42 ==== osd_op_reply(141 .bucket.meta.myfoo7:b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 [call,call,delete] v31'2 uv2 ondisk = 0) v8 ==== 292+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:142 4.0 4:024013e5:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430a10 con 0x55665437fd60 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:143 4.f 4:f161ffc7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cce80 con 0x556654376bd0 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:144 4.1d 4:bbf0e2f7:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430d70 con 0x55665437fd60 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:145 4.16 4:6a255352:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543ba3f0 con 0x55665437fd60 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:146 4.26 4:64d736ec:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543c9800 con 0x55665437b110 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:147 4.6 4:6104770f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430a10 con 0x7f65c400f3d0 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:148 4.5 4:a0031e4f:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430d70 con 0x55665437fd60 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:149 4.3 4:c06cfb55:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cce80 con 0x556654376bd0 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:150 4.2c 4:34fe866c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543ba3f0 con 0x556654376bd0 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:151 4.10 4:082a7bbd:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543c9800 con 0x55665437fd60 2024-03-28T16:50:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.196+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:152 4.13 4:c8f6006a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654430d70 con 0x7f65c400f3d0 2024-03-28T16:50:23.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 35 ==== osd_op_reply(147 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.5 [delete] v31'10 uv10 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(146 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.4 [delete] v31'4 uv4 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 43 ==== osd_op_reply(144 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.2 [delete] v31'3 uv3 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 44 ==== osd_op_reply(148 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.6 [delete] v31'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 36 ==== osd_op_reply(143 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.1 [delete] v31'6 uv6 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 45 ==== osd_op_reply(142 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.0 [delete] v31'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 46 ==== osd_op_reply(151 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.9 [delete] v31'6 uv6 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 37 ==== osd_op_reply(150 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.8 [delete] v31'4 uv4 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 47 ==== osd_op_reply(145 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.3 [delete] v31'8 uv8 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 38 ==== osd_op_reply(149 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.7 [delete] v31'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 36 ==== osd_op_reply(152 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.10.10 [delete] v31'6 uv6 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d2be0980 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 : miss 2024-03-28T16:50:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d2be0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.200+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:153 6.13 6:cf916ccc:::pubsub..bucket.myfoo7%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.10:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cd4b0 con 0x55665437b110 2024-03-28T16:50:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== osd_op_reply(153 pubsub..bucket.myfoo7/b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 210+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 info.flags=0x0 2024-03-28T16:50:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 10 adding default.rgw.log++pubsub..bucket.myfoo7/b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 to cache LRU end 2024-03-28T16:50:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 10 removing default.rgw.log++pubsub..bucket.myfoo7/b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 from cache 2024-03-28T16:50:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.log:pubsub..bucket.myfoo7/b88c2392-6ef8-4660-b080-19480b748fc0.4221.10, ofs0, ns] 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:154 7.d 7:b4812045:::notify.4:head [notify cookie 93897988704752 in=217b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cce40 con 0x556654376bd0 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 39 ==== watch-notify(notify (1) cookie 93897988006720 notify 133143986197 ret 0) v3 ==== 247+0+0 (crc 0 0 0) 0x7f65c003c000 con 0x556654376bd0 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f64aeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986197 cookie 93897988006720 notifier 4753 bl.length()=205 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f64aeffd640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:155 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f64040032c0 con 0x556654376bd0 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 40 ==== osd_op_reply(154 notify.4 [notify cookie 93897988704752 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 41 ==== osd_op_reply(155 notify.4 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 42 ==== watch-notify(notify_complete (2) cookie 93897988704752 notify 133143986197 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f65c0030040 con 0x556654376bd0 2024-03-28T16:50:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:156 6.13 6:cf916ccc:::pubsub..bucket.myfoo7%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cc9a0 con 0x55665437b110 2024-03-28T16:50:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(156 pubsub..bucket.myfoo7/b88c2392-6ef8-4660-b080-19480b748fc0.4221.10 [delete] v31'32 uv29 ondisk = -2 ((2) No such file or directory)) v8 ==== 210+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.204+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:157 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cc870 con 0x55665437b110 2024-03-28T16:50:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(157 fud.buckets [call] v31'11 uv11 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65d2be0980 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-03-28T16:50:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65d2be0980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:50:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:158 7.15 7:a93a5511:::notify.2:head [notify cookie 93897988062304 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cc870 con 0x55665437fd60 2024-03-28T16:50:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 48 ==== watch-notify(notify (1) cookie 93897988001824 notify 133143986191 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x7f65c4050060 con 0x55665437fd60 2024-03-28T16:50:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f64af7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986191 cookie 93897988001824 notifier 4753 bl.length()=170 2024-03-28T16:50:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f64af7fe640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:159 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f6494003d60 con 0x55665437fd60 2024-03-28T16:50:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 49 ==== osd_op_reply(158 notify.2 [notify cookie 93897988062304 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 50 ==== watch-notify(notify_complete (2) cookie 93897988062304 notify 133143986191 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f65c4044040 con 0x55665437fd60 2024-03-28T16:50:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 51 ==== osd_op_reply(159 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.208+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:160 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566544309f0 con 0x55665437b110 2024-03-28T16:50:23.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(160 p5YnriCv1nAtykxBrupQ [delete] v31'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d2be0980 10 removing email index: 2024-03-28T16:50:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d2be0980 10 removing user buckets index 2024-03-28T16:50:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d2be0980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-03-28T16:50:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93897988062304 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cc870 con 0x55665437b110 2024-03-28T16:50:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== watch-notify(notify (1) cookie 93897988012448 notify 133143986221 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x7f65c80e7030 con 0x55665437b110 2024-03-28T16:50:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f64aeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986221 cookie 93897988012448 notifier 4753 bl.length()=160 2024-03-28T16:50:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f64aeffd640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:162 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f64040032c0 con 0x55665437b110 2024-03-28T16:50:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 41 ==== osd_op_reply(161 notify.6 [notify cookie 93897988062304 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 42 ==== osd_op_reply(162 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 43 ==== watch-notify(notify_complete (2) cookie 93897988062304 notify 133143986221 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x556654358510 con 0x55665437b110 2024-03-28T16:50:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.212+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:163 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566544309f0 con 0x55665437b110 2024-03-28T16:50:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 44 ==== osd_op_reply(163 fud.buckets [delete] v31'12 uv12 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d2be0980 10 removing user index: fud 2024-03-28T16:50:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d2be0980 10 removing default.rgw.meta+users.uid+fud from cache 2024-03-28T16:50:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d2be0980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-03-28T16:50:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify cookie 93897988062304 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5566543cc870 con 0x55665437b110 2024-03-28T16:50:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 45 ==== watch-notify(notify (1) cookie 93897988014640 notify 133143986222 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x556654358510 con 0x55665437b110 2024-03-28T16:50:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f64af7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986222 cookie 93897988014640 notifier 4753 bl.length()=152 2024-03-28T16:50:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f64af7fe640 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:165 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f6494003d60 con 0x55665437b110 2024-03-28T16:50:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 46 ==== osd_op_reply(164 notify.5 [notify cookie 93897988062304 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 47 ==== osd_op_reply(165 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 48 ==== watch-notify(notify_complete (2) cookie 93897988062304 notify 133143986222 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55665433acd0 con 0x55665437b110 2024-03-28T16:50:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.216+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:166 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393c60 con 0x55665437b110 2024-03-28T16:50:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 49 ==== osd_op_reply(166 fud [call,call,delete] v31'7 uv7 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f64867e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:167 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93897988033968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x556654376bd0 2024-03-28T16:50:23.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 43 ==== osd_op_reply(167 data_loggenerations_metadata [watch unwatch cookie 93897988033968] v31'227 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:168 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93897987996416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x7f65c400f3d0 2024-03-28T16:50:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 37 ==== osd_op_reply(168 notify.0 [watch unwatch cookie 93897987996416] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d2be0980 20 remove_watcher() i=0 2024-03-28T16:50:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d2be0980 2 removed watcher, disabling cache 2024-03-28T16:50:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.220+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:169 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93897988000768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x556654376bd0 2024-03-28T16:50:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.224+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 44 ==== osd_op_reply(169 notify.1 [watch unwatch cookie 93897988000768] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.224+0000 7f65d2be0980 20 remove_watcher() i=1 2024-03-28T16:50:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.224+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:170 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93897988001824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x55665437fd60 2024-03-28T16:50:23.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65cfefa640 1 -- 172.21.15.170:0/420342365 <== osd.3 v2:172.21.15.170:6816/2102412426 52 ==== osd_op_reply(170 notify.2 [watch unwatch cookie 93897988001824] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c4032040 con 0x55665437fd60 2024-03-28T16:50:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d2be0980 20 remove_watcher() i=2 2024-03-28T16:50:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:171 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93897988008176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x7f65c400f3d0 2024-03-28T16:50:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.2 v2:172.21.15.170:6800/447751618 38 ==== osd_op_reply(171 notify.3 [watch unwatch cookie 93897988008176] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x7f65c400f3d0 2024-03-28T16:50:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d2be0980 20 remove_watcher() i=3 2024-03-28T16:50:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:172 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93897988006720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x556654376bd0 2024-03-28T16:50:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 <== osd.1 v2:172.21.15.170:6805/11349688 45 ==== osd_op_reply(172 notify.4 [watch unwatch cookie 93897988006720] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c00083f0 con 0x556654376bd0 2024-03-28T16:50:23.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d2be0980 20 remove_watcher() i=4 2024-03-28T16:50:23.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.228+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:173 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93897988014640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x55665437b110 2024-03-28T16:50:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.232+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 50 ==== osd_op_reply(173 notify.5 [watch unwatch cookie 93897988014640] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.232+0000 7f65d2be0980 20 remove_watcher() i=5 2024-03-28T16:50:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.232+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:174 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93897988012448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x55665437b110 2024-03-28T16:50:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.232+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 51 ==== osd_op_reply(174 notify.6 [watch unwatch cookie 93897988012448] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.232+0000 7f65d2be0980 20 remove_watcher() i=6 2024-03-28T16:50:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.232+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:175 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93897988019008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x556654393be0 con 0x55665437b110 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d06fb640 1 -- 172.21.15.170:0/420342365 <== osd.0 v2:172.21.15.119:6800/2028815515 52 ==== osd_op_reply(175 notify.7 [watch unwatch cookie 93897988019008] v31'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f65c811b040 con 0x55665437b110 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 20 remove_watcher() i=7 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55665437b110 msgr2=0x55665437d500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55665437b110 0x55665437d500 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654376bd0 msgr2=0x556654379090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654376bd0 0x556654379090 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65c400f3d0 msgr2=0x7f65c400f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f65c400f3d0 0x7f65c400f7f0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55665437fd60 msgr2=0x556654382220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55665437fd60 0x556654382220 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a00342b0 msgr2=0x7f64a0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a00342b0 0x7f64a0036770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f65c4010fd0 tx=0x7f65c4011000 comp rx=0 tx=0).stop 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 msgr2=0x55665436b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d0efc640 1 -- 172.21.15.170:0/420342365 reap_dead start 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x55665436b900 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ac010 tx=0x7f65c80ebf60 comp rx=0 tx=0).stop 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 shutdown_connections 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/420342365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654364ac0 0x55665436b900 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 >> 172.21.15.170:0/420342365 conn(0x556654360e00 msgr2=0x55665435e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 shutdown_connections 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/420342365 wait complete. 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a80342b0 msgr2=0x7f64a8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a80342b0 0x7f64a8036770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55665433e9c0 tx=0x7f65c4006000 comp rx=0 tx=0).stop 2024-03-28T16:50:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654357bb0 msgr2=0x556654357fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654357bb0 0x556654357fd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f65c80c7e30 tx=0x7f65c80c9e00 comp rx=0 tx=0).stop 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 shutdown_connections 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64a80342b0 0x7f64a8036770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/416539630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654357bb0 0x556654357fd0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 >> 172.21.15.170:0/416539630 conn(0x55665434c9c0 msgr2=0x55665434d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 shutdown_connections 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/416539630 wait complete. 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556654343550 msgr2=0x556654345940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x556654343550 0x556654345940 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654249a60 msgr2=0x556654340f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x556654249a60 0x556654340f90 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5566543471f0 msgr2=0x5566543495e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x5566543471f0 0x5566543495e0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64b80342b0 msgr2=0x7f64b8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f64b80342b0 0x7f64b8036770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f65c4001e20 tx=0x7f65c40013a0 comp rx=0 tx=0).stop 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 msgr2=0x55665433a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 --2- 172.21.15.170:0/392499926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556654249140 0x55665433a360 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f65c801aec0 tx=0x7f65c8017e30 comp rx=0 tx=0).stop 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d0efc640 1 -- 172.21.15.170:0/392499926 reap_dead start 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 shutdown_connections 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 >> 172.21.15.170:0/392499926 conn(0x5566541bc860 msgr2=0x55665432dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 shutdown_connections 2024-03-28T16:50:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:23.236+0000 7f65d2be0980 1 -- 172.21.15.170:0/392499926 wait complete. 2024-03-28T16:50:28.248 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-03-28T16:50:28.248 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-03-28T16:50:28.248 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-03-28T16:50:28.287 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:28.287 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:28.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff904f29980 1 Processor -- start 2024-03-28T16:50:28.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff904f29980 1 -- start start 2024-03-28T16:50:28.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff904f29980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c539365130 0x55c539365510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff904f29980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c5392af520 con 0x55c539365a50 2024-03-28T16:50:28.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff904f29980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c5392760f0 con 0x55c539365130 2024-03-28T16:50:28.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff902a44640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c539365130 0x55c539365510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff902a44640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c539365130 0x55c539365510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47618/0 (socket says 172.21.15.170:47618) 2024-03-28T16:50:28.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff902a44640 1 -- 172.21.15.170:0/3157979538 learned_addr learned my addr 172.21.15.170:0/3157979538 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff902a44640 1 -- 172.21.15.170:0/3157979538 >> v1:172.21.15.119:6789/0 conn(0x55c539365a50 legacy=0x55c539366400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.288+0000 7ff902a44640 1 -- 172.21.15.170:0/3157979538 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c539366c30 con 0x55c539365130 2024-03-28T16:50:28.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff902a44640 1 --2- 172.21.15.170:0/3157979538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539365510 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400a5c0 tx=0x7ff8f401b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ab853070c023c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff901a42640 1 -- 172.21.15.170:0/3157979538 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40aa470 con 0x55c539365130 2024-03-28T16:50:28.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff901a42640 1 -- 172.21.15.170:0/3157979538 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40aa5d0 con 0x55c539365130 2024-03-28T16:50:28.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff901a42640 1 -- 172.21.15.170:0/3157979538 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40aa890 con 0x55c539365130 2024-03-28T16:50:28.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff904f29980 1 -- 172.21.15.170:0/3157979538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 msgr2=0x55c539365510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff904f29980 1 --2- 172.21.15.170:0/3157979538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539365510 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400a5c0 tx=0x7ff8f401b340 comp rx=0 tx=0).stop 2024-03-28T16:50:28.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff904f29980 1 -- 172.21.15.170:0/3157979538 shutdown_connections 2024-03-28T16:50:28.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff904f29980 1 --2- 172.21.15.170:0/3157979538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539365510 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff904f29980 1 -- 172.21.15.170:0/3157979538 >> 172.21.15.170:0/3157979538 conn(0x55c5392d8850 msgr2=0x55c5392d8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff904f29980 1 -- 172.21.15.170:0/3157979538 shutdown_connections 2024-03-28T16:50:28.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.292+0000 7ff904f29980 1 -- 172.21.15.170:0/3157979538 wait complete. 2024-03-28T16:50:28.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff904f29980 1 Processor -- start 2024-03-28T16:50:28.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff904f29980 1 -- start start 2024-03-28T16:50:28.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff904f29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c53944f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff904f29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c539273620 con 0x55c539365130 2024-03-28T16:50:28.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c53944f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c53944f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47622/0 (socket says 172.21.15.170:47622) 2024-03-28T16:50:28.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff902a44640 1 -- 172.21.15.170:0/1444700215 learned_addr learned my addr 172.21.15.170:0/1444700215 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff902a44640 1 -- 172.21.15.170:0/1444700215 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4005df0 con 0x55c539365130 2024-03-28T16:50:28.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff902a44640 1 --2- 172.21.15.170:0/1444700215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c53944f610 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40ac040 tx=0x7ff8f4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5ef21bff56add2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff7f0e09640 1 -- 172.21.15.170:0/1444700215 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40aa930 con 0x55c539365130 2024-03-28T16:50:28.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff7f0e09640 1 -- 172.21.15.170:0/1444700215 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40aaa90 con 0x55c539365130 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.308+0000 7ff7f0e09640 1 -- 172.21.15.170:0/1444700215 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40aad50 con 0x55c539365130 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- 172.21.15.170:0/1444700215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 msgr2=0x55c53944f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 --2- 172.21.15.170:0/1444700215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c53944f610 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40ac040 tx=0x7ff8f4003030 comp rx=0 tx=0).stop 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- 172.21.15.170:0/1444700215 shutdown_connections 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 --2- 172.21.15.170:0/1444700215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c53944f610 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- 172.21.15.170:0/1444700215 >> 172.21.15.170:0/1444700215 conn(0x55c5392d8850 msgr2=0x55c53944b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- 172.21.15.170:0/1444700215 shutdown_connections 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- 172.21.15.170:0/1444700215 wait complete. 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 Processor -- start 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- start start 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539456360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c53926ee00 con 0x55c539365130 2024-03-28T16:50:28.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539456360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539456360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47634/0 (socket says 172.21.15.170:47634) 2024-03-28T16:50:28.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff902a44640 1 -- 172.21.15.170:0/1448914647 learned_addr learned my addr 172.21.15.170:0/1448914647 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff902a44640 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c539366c30 con 0x55c539365130 2024-03-28T16:50:28.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff902a44640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539456360 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8f401b380 tx=0x7ff8f40180e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff7ef606640 1 -- 172.21.15.170:0/1448914647 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40abee0 con 0x55c539365130 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5394568a0 con 0x55c539365130 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff7ef606640 1 -- 172.21.15.170:0/1448914647 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40c6690 con 0x55c539365130 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff7ef606640 1 -- 172.21.15.170:0/1448914647 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40c6930 con 0x55c539365130 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.312+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c539456cd0 con 0x55c539365130 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff7ef606640 1 -- 172.21.15.170:0/1448914647 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff8f40c6ba0 con 0x55c539365130 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff7ef606640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7e80342b0 0x7ff7e8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff7ef606640 1 -- 172.21.15.170:0/1448914647 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff8f40e4a40 con 0x55c539365130 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff902243640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7e80342b0 0x7ff7e8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539365a50 0x55c53945cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53945d4d0 con 0x55c539365a50 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff903245640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539365a50 0x55c53945cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff902243640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7e80342b0 0x7ff7e8036770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0012f0 tx=0x7ff8ec0066b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff903245640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539365a50 0x55c53945cf90 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff903245640 1 -- 172.21.15.170:0/1448914647 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539365a50 2024-03-28T16:50:28.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53945d830 con 0x55c539365a50 2024-03-28T16:50:28.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff903245640 1 -- 172.21.15.170:0/1448914647 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539365a50 2024-03-28T16:50:28.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c53945f1c0 0x55c5394615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539461d70 con 0x55c53945f1c0 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff902a44640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c53945f1c0 0x55c5394615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff902a44640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c53945f1c0 0x55c5394615e0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff902a44640 1 -- 172.21.15.170:0/1448914647 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7ff8f40cd020 con 0x55c53945f1c0 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.316+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53945e030 con 0x55c539365a50 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff903245640 1 -- 172.21.15.170:0/1448914647 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539365a50 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c539463190 0x55c539465580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539465c40 con 0x55c539463190 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902243640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c539463190 0x55c539465580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902243640 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c539463190 0x55c539465580 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902243640 1 -- 172.21.15.170:0/1448914647 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7ff8ec015190 con 0x55c539463190 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 Processor -- start 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 -- start start 2024-03-28T16:50:28.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539473b70 0x55c539473f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c539207e30 con 0x55c539473b70 2024-03-28T16:50:28.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539473b70 0x55c539473f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539473b70 0x55c539473f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47648/0 (socket says 172.21.15.170:47648) 2024-03-28T16:50:28.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902a44640 1 -- 172.21.15.170:0/741865420 learned_addr learned my addr 172.21.15.170:0/741865420 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902a44640 1 -- 172.21.15.170:0/741865420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5394744d0 con 0x55c539473b70 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff902a44640 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539473b70 0x55c539473f90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40bc560 tx=0x7ff8f40bdf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff7ece01640 1 -- 172.21.15.170:0/741865420 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40bc920 con 0x55c539473b70 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff7ece01640 1 -- 172.21.15.170:0/741865420 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40bca80 con 0x55c539473b70 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c539474720 con 0x55c539473b70 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff7ece01640 1 -- 172.21.15.170:0/741865420 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40b33f0 con 0x55c539473b70 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.320+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c539475270 con 0x55c539473b70 2024-03-28T16:50:28.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff7ece01640 1 -- 172.21.15.170:0/741865420 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff8f40ea070 con 0x55c539473b70 2024-03-28T16:50:28.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff7ece01640 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7dc0342b0 0x7ff7dc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff7ece01640 1 -- 172.21.15.170:0/741865420 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff8f40e60e0 con 0x55c539473b70 2024-03-28T16:50:28.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff902243640 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7dc0342b0 0x7ff7dc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 Processor -- start 2024-03-28T16:50:28.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 -- start start 2024-03-28T16:50:28.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff902243640 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7dc0342b0 0x7ff7dc036770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55c539462100 tx=0x7ff8ec011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539480d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5391fb790 con 0x55c539480900 2024-03-28T16:50:28.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539480d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539480d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47660/0 (socket says 172.21.15.170:47660) 2024-03-28T16:50:28.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff902a44640 1 -- 172.21.15.170:0/899891314 learned_addr learned my addr 172.21.15.170:0/899891314 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff902a44640 1 -- 172.21.15.170:0/899891314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5394812f0 con 0x55c539480900 2024-03-28T16:50:28.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff902a44640 1 --2- 172.21.15.170:0/899891314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539480d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40b7000 tx=0x7ff8f40dcba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222b5a030f983437 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff7e67fc640 1 -- 172.21.15.170:0/899891314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40fb040 con 0x55c539480900 2024-03-28T16:50:28.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff7e67fc640 1 -- 172.21.15.170:0/899891314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40dc5d0 con 0x55c539480900 2024-03-28T16:50:28.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff7e67fc640 1 -- 172.21.15.170:0/899891314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40bf200 con 0x55c539480900 2024-03-28T16:50:28.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 -- 172.21.15.170:0/899891314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 msgr2=0x55c539480d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 --2- 172.21.15.170:0/899891314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539480d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40b7000 tx=0x7ff8f40dcba0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 -- 172.21.15.170:0/899891314 shutdown_connections 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 --2- 172.21.15.170:0/899891314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539480d20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 -- 172.21.15.170:0/899891314 >> 172.21.15.170:0/899891314 conn(0x55c53947ccd0 msgr2=0x55c53947ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 -- 172.21.15.170:0/899891314 shutdown_connections 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 -- 172.21.15.170:0/899891314 wait complete. 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.324+0000 7ff904f29980 1 Processor -- start 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 1 -- start start 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539487830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c539487d70 con 0x55c539480900 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539487830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff902a44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539487830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47666/0 (socket says 172.21.15.170:47666) 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 learned_addr learned my addr 172.21.15.170:0/1025327253 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5394744d0 con 0x55c539480900 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff902a44640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539487830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40c6b70 tx=0x7ff8f40d8f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff7e4ff9640 1 -- 172.21.15.170:0/1025327253 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40c7cb0 con 0x55c539480900 2024-03-28T16:50:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5394744d0 con 0x55c539480900 2024-03-28T16:50:28.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5394744d0 con 0x55c539480900 2024-03-28T16:50:28.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff7e4ff9640 1 -- 172.21.15.170:0/1025327253 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40de420 con 0x55c539480900 2024-03-28T16:50:28.330 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff7e4ff9640 1 -- 172.21.15.170:0/1025327253 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f40bfde0 con 0x55c539480900 2024-03-28T16:50:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff7e4ff9640 1 -- 172.21.15.170:0/1025327253 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7ff8f40dea50 con 0x55c539480900 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff7e4ff9640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7cc0342b0 0x7ff7cc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff7e4ff9640 1 -- 172.21.15.170:0/1025327253 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff8f4118910 con 0x55c539480900 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff902243640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7cc0342b0 0x7ff7cc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539492a40 0x55c539494f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539495650 con 0x55c539492a40 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff903245640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539492a40 0x55c539494f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff902243640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7cc0342b0 0x7ff7cc036770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec015610 tx=0x7ff8ec012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.328+0000 7ff903245640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539492a40 0x55c539494f00 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 realm 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539495a20 con 0x55c539492a40 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539496600 con 0x55c539492a40 2024-03-28T16:50:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c539496fb0 0x55c5394993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394999b0 con 0x55c539496fb0 2024-03-28T16:50:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902a44640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c539496fb0 0x55c5394993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902a44640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c539496fb0 0x55c5394993a0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 20 searching for the correct realm 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c53949bb90 0x55c53949e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53949e720 con 0x55c53949bb90 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902243640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c53949bb90 0x55c53949e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902243640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c53949bb90 0x55c53949e050 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902243640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff8ec013040 0x7ff8ec0243c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53949e720 con 0x7ff8ec013040 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.332+0000 7ff903245640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff8ec013040 0x7ff8ec0243c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff8ec013040 0x7ff8ec0243c0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x7ff8ec013040 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.336+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x7ff8ec013040 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.340+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x7ff8ec013040 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.347 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x7ff8ec013040 2024-03-28T16:50:28.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.344+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c53949bb90 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539496fb0 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x55c539492a40 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394962a0 con 0x7ff8ec013040 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:28.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539499f90 con 0x55c539492a40 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539499f90 con 0x55c539492a40 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c539499f60 con 0x7ff8ec013040 2024-03-28T16:50:28.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 zone default found 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 4 Realm: () 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 zonegroup default 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53949a7e0 con 0x55c539496fb0 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53949ac40 con 0x55c53949bb90 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.348+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53949a480 con 0x55c53949bb90 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 20 started sync module instance, tier type = 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53949fe20 con 0x7ff8ec013040 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94305558067584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53949fda0 con 0x7ff8ec013040 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a1460 con 0x55c539492a40 2024-03-28T16:50:28.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.352+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94305558067584] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94305558072096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a13e0 con 0x55c539492a40 2024-03-28T16:50:28.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a1be0 con 0x55c53949bb90 2024-03-28T16:50:28.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94305558072096] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94305558073424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a1f90 con 0x55c53949bb90 2024-03-28T16:50:28.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a2900 con 0x7ff8ec013040 2024-03-28T16:50:28.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94305558078832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a2ff0 con 0x7ff8ec013040 2024-03-28T16:50:28.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.356+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a3ac0 con 0x55c539492a40 2024-03-28T16:50:28.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.360+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94305558073424] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.360+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94305558078832] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.360+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.360+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94305558082976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a3a80 con 0x55c539492a40 2024-03-28T16:50:28.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.360+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a2bc0 con 0x55c539496fb0 2024-03-28T16:50:28.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.360+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94305558082976] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.364+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.364+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94305558074544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a4310 con 0x55c539496fb0 2024-03-28T16:50:28.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.364+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a4b10 con 0x55c539496fb0 2024-03-28T16:50:28.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.364+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94305558074544] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.364+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.364+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94305558085232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a5310 con 0x55c539496fb0 2024-03-28T16:50:28.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.364+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a5c80 con 0x55c539496fb0 2024-03-28T16:50:28.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94305558085232] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94305558091744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a60a0 con 0x55c539496fb0 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=0 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=1 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=2 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=3 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=4 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=5 2024-03-28T16:50:28.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=6 2024-03-28T16:50:28.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94305558091744] v31'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 20 add_watcher() i=7 2024-03-28T16:50:28.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:28.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a9390 con 0x55c539492a40 2024-03-28T16:50:28.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.368+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94305558093168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a6630 con 0x55c539492a40 2024-03-28T16:50:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94305558093168] v31'228 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff7b97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 5 note: GC not initialized 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff733fff640 20 reqs_thread_entry: start 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 20 Filter name: none 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff731ffb640 20 reqs_thread_entry: start 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394cb350 con 0x55c539492a40 2024-03-28T16:50:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:50:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:50:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:28.374 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.372+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394cd870 con 0x55c539496fb0 2024-03-28T16:50:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.376+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 usage.10 [call] v31'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.376+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394cd870 con 0x55c539496fb0 2024-03-28T16:50:28.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.376+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(64 usage.10 [call] v31'34 uv33 ondisk = -61 ((61) No data available)) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff7b97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:28.379 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94305558093168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c53948c780 con 0x55c539492a40 2024-03-28T16:50:28.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94305558093168] v31'229 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.380 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94305558067584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a5d70 con 0x7ff8ec013040 2024-03-28T16:50:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94305558067584] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff904f29980 20 remove_watcher() i=0 2024-03-28T16:50:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff904f29980 2 removed watcher, disabling cache 2024-03-28T16:50:28.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.380+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94305558072096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a5d70 con 0x55c539492a40 2024-03-28T16:50:28.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.384+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94305558072096] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.384+0000 7ff904f29980 20 remove_watcher() i=1 2024-03-28T16:50:28.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.384+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94305558073424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a0b20 con 0x55c53949bb90 2024-03-28T16:50:28.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.384+0000 7ff902243640 1 -- 172.21.15.170:0/1025327253 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94305558073424] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8ec015190 con 0x55c53949bb90 2024-03-28T16:50:28.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.384+0000 7ff904f29980 20 remove_watcher() i=2 2024-03-28T16:50:28.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.384+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94305558078832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a5d70 con 0x7ff8ec013040 2024-03-28T16:50:28.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.388+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94305558078832] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x7ff8ec013040 2024-03-28T16:50:28.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.388+0000 7ff904f29980 20 remove_watcher() i=3 2024-03-28T16:50:28.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.388+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94305558082976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a2570 con 0x55c539492a40 2024-03-28T16:50:28.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.388+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94305558082976] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8fc0083f0 con 0x55c539492a40 2024-03-28T16:50:28.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.388+0000 7ff904f29980 20 remove_watcher() i=4 2024-03-28T16:50:28.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.388+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94305558074544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a5d70 con 0x55c539496fb0 2024-03-28T16:50:28.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.392+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94305558074544] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.392+0000 7ff904f29980 20 remove_watcher() i=5 2024-03-28T16:50:28.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.392+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94305558085232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a2570 con 0x55c539496fb0 2024-03-28T16:50:28.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.392+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94305558085232] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.392+0000 7ff904f29980 20 remove_watcher() i=6 2024-03-28T16:50:28.393 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.392+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94305558091744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c5394a5d70 con 0x55c539496fb0 2024-03-28T16:50:28.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff902a44640 1 -- 172.21.15.170:0/1025327253 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94305558091744] v31'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7ff8f4107020 con 0x55c539496fb0 2024-03-28T16:50:28.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 20 remove_watcher() i=7 2024-03-28T16:50:28.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c539496fb0 msgr2=0x55c5394993a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c539496fb0 0x55c5394993a0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539492a40 msgr2=0x55c539494f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539492a40 0x55c539494f00 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff8ec013040 msgr2=0x7ff8ec0243c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7ff8ec013040 0x7ff8ec0243c0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c53949bb90 msgr2=0x55c53949e050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c53949bb90 0x55c53949e050 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7cc0342b0 msgr2=0x7ff7cc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7cc0342b0 0x7ff7cc036770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec015610 tx=0x7ff8ec012000 comp rx=0 tx=0).stop 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff903245640 1 -- 172.21.15.170:0/1025327253 reap_dead start 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 msgr2=0x55c539487830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539487830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40c6b70 tx=0x7ff8f40d8f60 comp rx=0 tx=0).stop 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 shutdown_connections 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1025327253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539480900 0x55c539487830 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 >> 172.21.15.170:0/1025327253 conn(0x55c53947ccd0 msgr2=0x55c53947a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 shutdown_connections 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1025327253 wait complete. 2024-03-28T16:50:28.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7dc0342b0 msgr2=0x7ff7dc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7dc0342b0 0x7ff7dc036770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55c539462100 tx=0x7ff8ec011000 comp rx=0 tx=0).stop 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539473b70 msgr2=0x55c539473f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539473b70 0x55c539473f90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40bc560 tx=0x7ff8f40bdf70 comp rx=0 tx=0).stop 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 shutdown_connections 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7dc0342b0 0x7ff7dc036770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/741865420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539473b70 0x55c539473f90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 >> 172.21.15.170:0/741865420 conn(0x55c539468980 msgr2=0x55c539469490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 shutdown_connections 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/741865420 wait complete. 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c53945f1c0 msgr2=0x55c5394615e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c53945f1c0 0x55c5394615e0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539365a50 msgr2=0x55c53945cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c539365a50 0x55c53945cf90 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c539463190 msgr2=0x55c539465580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c539463190 0x55c539465580 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7e80342b0 msgr2=0x7ff7e8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7ff7e80342b0 0x7ff7e8036770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0012f0 tx=0x7ff8ec0066b0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 msgr2=0x55c539456360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 --2- 172.21.15.170:0/1448914647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c539365130 0x55c539456360 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8f401b380 tx=0x7ff8f40180e0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff903245640 1 -- 172.21.15.170:0/1448914647 reap_dead start 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 shutdown_connections 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 >> 172.21.15.170:0/1448914647 conn(0x55c5392d8850 msgr2=0x55c539449cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 shutdown_connections 2024-03-28T16:50:28.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.396+0000 7ff904f29980 1 -- 172.21.15.170:0/1448914647 wait complete. 2024-03-28T16:50:28.408 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-03-28T16:50:28.408 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-03-28T16:50:28.409 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-03-28T16:50:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:28.486 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961d12d980 1 Processor -- start 2024-03-28T16:50:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961d12d980 1 -- start start 2024-03-28T16:50:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961d12d980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b667661070 0x55b6676f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961d12d980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55b667639520 con 0x55b6676f2530 2024-03-28T16:50:28.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961d12d980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55b6676000f0 con 0x55b667661070 2024-03-28T16:50:28.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961ac48640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b667661070 0x55b6676f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961ac48640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55b667661070 0x55b6676f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47678/0 (socket says 172.21.15.170:47678) 2024-03-28T16:50:28.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961a447640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55b6676f2530 0x55b6676f4920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:49642/0 (socket says 172.21.15.170:49642) 2024-03-28T16:50:28.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961ac48640 1 -- 172.21.15.170:0/3204608248 learned_addr learned my addr 172.21.15.170:0/3204608248 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961ac48640 1 -- 172.21.15.170:0/3204608248 >> v1:172.21.15.119:6789/0 conn(0x55b6676f2530 legacy=0x55b6676f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.488+0000 7f961ac48640 1 -- 172.21.15.170:0/3204608248 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6676f7050 con 0x55b667661070 2024-03-28T16:50:28.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961ac48640 1 --2- 172.21.15.170:0/3204608248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6676f1ff0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f961400a4e0 tx=0x7f961401b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9bcae2b1dc46ac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f9619c46640 1 -- 172.21.15.170:0/3204608248 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140aa540 con 0x55b667661070 2024-03-28T16:50:28.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f9619c46640 1 -- 172.21.15.170:0/3204608248 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140a6070 con 0x55b667661070 2024-03-28T16:50:28.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f9619c46640 1 -- 172.21.15.170:0/3204608248 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140aa850 con 0x55b667661070 2024-03-28T16:50:28.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961d12d980 1 -- 172.21.15.170:0/3204608248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 msgr2=0x55b6676f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961d12d980 1 --2- 172.21.15.170:0/3204608248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6676f1ff0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f961400a4e0 tx=0x7f961401b090 comp rx=0 tx=0).stop 2024-03-28T16:50:28.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961d12d980 1 -- 172.21.15.170:0/3204608248 shutdown_connections 2024-03-28T16:50:28.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961d12d980 1 --2- 172.21.15.170:0/3204608248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6676f1ff0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961d12d980 1 -- 172.21.15.170:0/3204608248 >> 172.21.15.170:0/3204608248 conn(0x55b6676e76d0 msgr2=0x55b6676e9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961d12d980 1 -- 172.21.15.170:0/3204608248 shutdown_connections 2024-03-28T16:50:28.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.492+0000 7f961d12d980 1 -- 172.21.15.170:0/3204608248 wait complete. 2024-03-28T16:50:28.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961d12d980 1 Processor -- start 2024-03-28T16:50:28.508 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961d12d980 1 -- start start 2024-03-28T16:50:28.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961d12d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961d12d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6675fd620 con 0x55b667661070 2024-03-28T16:50:28.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677d9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47692/0 (socket says 172.21.15.170:47692) 2024-03-28T16:50:28.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961ac48640 1 -- 172.21.15.170:0/440811858 learned_addr learned my addr 172.21.15.170:0/440811858 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961ac48640 1 -- 172.21.15.170:0/440811858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961400a1c0 con 0x55b667661070 2024-03-28T16:50:28.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f961ac48640 1 --2- 172.21.15.170:0/440811858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677d9160 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9614017ca0 tx=0x7f96140033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa13859e47088df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f9508e09640 1 -- 172.21.15.170:0/440811858 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140aa750 con 0x55b667661070 2024-03-28T16:50:28.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f9508e09640 1 -- 172.21.15.170:0/440811858 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140aa8b0 con 0x55b667661070 2024-03-28T16:50:28.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.508+0000 7f9508e09640 1 -- 172.21.15.170:0/440811858 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140aab70 con 0x55b667661070 2024-03-28T16:50:28.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- 172.21.15.170:0/440811858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 msgr2=0x55b6677d9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 --2- 172.21.15.170:0/440811858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677d9160 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9614017ca0 tx=0x7f96140033b0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- 172.21.15.170:0/440811858 shutdown_connections 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 --2- 172.21.15.170:0/440811858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677d9160 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- 172.21.15.170:0/440811858 >> 172.21.15.170:0/440811858 conn(0x55b6676e76d0 msgr2=0x55b6677d4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- 172.21.15.170:0/440811858 shutdown_connections 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- 172.21.15.170:0/440811858 wait complete. 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 Processor -- start 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- start start 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677e0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6675f8e00 con 0x55b667661070 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677e0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47694/0 (socket says 172.21.15.170:47694) 2024-03-28T16:50:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961ac48640 1 -- 172.21.15.170:0/362091622 learned_addr learned my addr 172.21.15.170:0/362091622 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.515 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961ac48640 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6676f7050 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961ac48640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677e0050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9614000f90 tx=0x7f9614017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f9507606640 1 -- 172.21.15.170:0/362091622 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140c4070 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f9507606640 1 -- 172.21.15.170:0/362091622 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140b5780 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f9507606640 1 -- 172.21.15.170:0/362091622 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140b5aa0 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b6677e0590 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.512+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6677e09c0 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f9507606640 1 -- 172.21.15.170:0/362091622 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f96140a8070 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f9507606640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9500034300 0x7f95000367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f9507606640 1 -- 172.21.15.170:0/362091622 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f96140e3d90 con 0x55b667661070 2024-03-28T16:50:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961a447640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9500034300 0x7f95000367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b6676f2530 0x55b6677e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6677e71c0 con 0x55b6676f2530 2024-03-28T16:50:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961b449640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b6676f2530 0x55b6677e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961a447640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9500034300 0x7f95000367c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9610001e50 tx=0x7f96100013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961b449640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b6676f2530 0x55b6677e6c80 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961b449640 1 -- 172.21.15.170:0/362091622 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b6676f2530 2024-03-28T16:50:28.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6677e7520 con 0x55b6676f2530 2024-03-28T16:50:28.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961b449640 1 -- 172.21.15.170:0/362091622 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f960c0083f0 con 0x55b6676f2530 2024-03-28T16:50:28.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b6677e8e30 0x55b6677eb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6677eb9f0 con 0x55b6677e8e30 2024-03-28T16:50:28.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961ac48640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b6677e8e30 0x55b6677eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961ac48640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b6677e8e30 0x55b6677eb260 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961ac48640 1 -- 172.21.15.170:0/362091622 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f96140b5c00 con 0x55b6677e8e30 2024-03-28T16:50:28.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.516+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6677e7d20 con 0x55b6676f2530 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961b449640 1 -- 172.21.15.170:0/362091622 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f960c0083f0 con 0x55b6676f2530 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b6677ece10 0x55b6677ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6677ef8f0 con 0x55b6677ece10 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961a447640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b6677ece10 0x55b6677ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961a447640 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b6677ece10 0x55b6677ef200 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961a447640 1 -- 172.21.15.170:0/362091622 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f961000f700 con 0x55b6677ece10 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 Processor -- start 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 -- start start 2024-03-28T16:50:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b6677fd810 0x55b6677fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b667591e30 con 0x55b6677fd810 2024-03-28T16:50:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b6677fd810 0x55b6677fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b6677fd810 0x55b6677fdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47698/0 (socket says 172.21.15.170:47698) 2024-03-28T16:50:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961ac48640 1 -- 172.21.15.170:0/2895079183 learned_addr learned my addr 172.21.15.170:0/2895079183 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961ac48640 1 -- 172.21.15.170:0/2895079183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6677fe170 con 0x55b6677fd810 2024-03-28T16:50:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961ac48640 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b6677fd810 0x55b6677fdc30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f96140e2290 tx=0x7f96140a9e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f9504e01640 1 -- 172.21.15.170:0/2895079183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9614005250 con 0x55b6677fd810 2024-03-28T16:50:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b6677fe3c0 con 0x55b6677fd810 2024-03-28T16:50:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f9504e01640 1 -- 172.21.15.170:0/2895079183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140de040 con 0x55b6677fd810 2024-03-28T16:50:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f9504e01640 1 -- 172.21.15.170:0/2895079183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140054e0 con 0x55b6677fd810 2024-03-28T16:50:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.520+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6677fef10 con 0x55b6677fd810 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f9504e01640 1 -- 172.21.15.170:0/2895079183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f96140d23f0 con 0x55b6677fd810 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f9504e01640 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94f4034260 0x7f94f4036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f9504e01640 1 -- 172.21.15.170:0/2895079183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f96140e4340 con 0x55b6677fd810 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961a447640 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94f4034260 0x7f94f4036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961d12d980 1 Processor -- start 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961d12d980 1 -- start start 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961a447640 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94f4034260 0x7f94f4036720 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55b6677e8080 tx=0x7f961000f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961d12d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b66780ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961d12d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b667585790 con 0x55b66780a710 2024-03-28T16:50:28.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b66780ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b66780ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47702/0 (socket says 172.21.15.170:47702) 2024-03-28T16:50:28.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961ac48640 1 -- 172.21.15.170:0/2876850535 learned_addr learned my addr 172.21.15.170:0/2876850535 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961ac48640 1 -- 172.21.15.170:0/2876850535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66780b100 con 0x55b66780a710 2024-03-28T16:50:28.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.524+0000 7f961ac48640 1 --2- 172.21.15.170:0/2876850535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b66780ab30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f96140e4130 tx=0x7f96140e4220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61229923485ed8ff server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f94fe7fc640 1 -- 172.21.15.170:0/2876850535 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140f9050 con 0x55b66780a710 2024-03-28T16:50:28.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f94fe7fc640 1 -- 172.21.15.170:0/2876850535 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140fa040 con 0x55b66780a710 2024-03-28T16:50:28.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f94fe7fc640 1 -- 172.21.15.170:0/2876850535 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140d26a0 con 0x55b66780a710 2024-03-28T16:50:28.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- 172.21.15.170:0/2876850535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 msgr2=0x55b66780ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 --2- 172.21.15.170:0/2876850535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b66780ab30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f96140e4130 tx=0x7f96140e4220 comp rx=0 tx=0).stop 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- 172.21.15.170:0/2876850535 shutdown_connections 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 --2- 172.21.15.170:0/2876850535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b66780ab30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- 172.21.15.170:0/2876850535 >> 172.21.15.170:0/2876850535 conn(0x55b667806a50 msgr2=0x55b667804ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- 172.21.15.170:0/2876850535 shutdown_connections 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- 172.21.15.170:0/2876850535 wait complete. 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 Processor -- start 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- start start 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b667811580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b667811ac0 con 0x55b66780a710 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b667811580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961ac48640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b667811580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47706/0 (socket says 172.21.15.170:47706) 2024-03-28T16:50:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 learned_addr learned my addr 172.21.15.170:0/3266171696 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6677fef10 con 0x55b66780a710 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961ac48640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b667811580 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f96140bb000 tx=0x7f96140baf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f94fcff9640 1 -- 172.21.15.170:0/3266171696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140d27d0 con 0x55b66780a710 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b667811cc0 con 0x55b66780a710 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f94fcff9640 1 -- 172.21.15.170:0/3266171696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961410e070 con 0x55b66780a710 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f94fcff9640 1 -- 172.21.15.170:0/3266171696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96140d86d0 con 0x55b66780a710 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.528+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b667811cc0 con 0x55b66780a710 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.531 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f94fcff9640 1 -- 172.21.15.170:0/3266171696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9614110030 con 0x55b66780a710 2024-03-28T16:50:28.534 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f94fcff9640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94e40342b0 0x7f94e4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f94fcff9640 1 -- 172.21.15.170:0/3266171696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f961411b710 con 0x55b66780a710 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b66781c9c0 0x55b66781ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66781f5d0 con 0x55b66781c9c0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961a447640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94e40342b0 0x7f94e4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961b449640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b66781c9c0 0x55b66781ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961b449640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b66781c9c0 0x55b66781ee80 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961a447640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94e40342b0 0x7f94e4036770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55b6677efc90 tx=0x7f9610010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 realm 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.535 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66781f9a0 con 0x55b66781c9c0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667820550 con 0x55b66781c9c0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:28.536 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b667820f00 0x55b6678232f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667823870 con 0x55b667820f00 2024-03-28T16:50:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961ac48640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b667820f00 0x55b6678232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.532+0000 7f961ac48640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b667820f00 0x55b6678232f0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961d12d980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961d12d980 20 searching for the correct realm 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b667825770 0x55b667827ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667828270 con 0x55b667825770 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961a447640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b667825770 0x55b667827ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961a447640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b667825770 0x55b667827ba0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961a447640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f961001fbe0 0x7f961003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667828270 con 0x7f961001fbe0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f961001fbe0 0x7f961003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f961001fbe0 0x7f961003a3c0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.536+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x7f961001fbe0 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x7f961001fbe0 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.540+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x7f961001fbe0 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.548 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.544+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.553 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x7f961001fbe0 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667825770 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b667820f00 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x55b66781c9c0 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678201f0 con 0x7f961001fbe0 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678240e0 con 0x55b66781c9c0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.548+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678240e0 con 0x55b66781c9c0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.555 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b6678240e0 con 0x7f961001fbe0 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 zone default found 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 4 Realm: () 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 zonegroup default 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.556 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667824930 con 0x55b667820f00 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667824d10 con 0x55b667825770 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782a210 con 0x55b667825770 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 started sync module instance, tier type = 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:28.557 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667829970 con 0x7f961001fbe0 2024-03-28T16:50:28.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94241909019856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667829d20 con 0x7f961001fbe0 2024-03-28T16:50:28.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.552+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782af70 con 0x55b66781c9c0 2024-03-28T16:50:28.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.556+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94241909019856] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.556+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.556+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94241909024304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782af30 con 0x55b66781c9c0 2024-03-28T16:50:28.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.556+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782b9f0 con 0x55b667825770 2024-03-28T16:50:28.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.556+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94241909024304] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94241909028016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782bda0 con 0x55b667825770 2024-03-28T16:50:28.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782c710 con 0x7f961001fbe0 2024-03-28T16:50:28.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94241909031808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782ce00 con 0x7f961001fbe0 2024-03-28T16:50:28.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782d8d0 con 0x55b66781c9c0 2024-03-28T16:50:28.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94241909028016] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94241909031808] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94241909035952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782d890 con 0x55b66781c9c0 2024-03-28T16:50:28.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.560+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782dc80 con 0x55b667820f00 2024-03-28T16:50:28.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.564+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94241909035952] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.564+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.564+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94241909033056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782e030 con 0x55b667820f00 2024-03-28T16:50:28.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.564+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782e890 con 0x55b667820f00 2024-03-28T16:50:28.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94241909033056] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94241909040016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782ab50 con 0x55b667820f00 2024-03-28T16:50:28.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782f640 con 0x55b667820f00 2024-03-28T16:50:28.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94241909043584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782fae0 con 0x55b667820f00 2024-03-28T16:50:28.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 20 add_watcher() i=0 2024-03-28T16:50:28.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 20 add_watcher() i=1 2024-03-28T16:50:28.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 20 add_watcher() i=2 2024-03-28T16:50:28.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 20 add_watcher() i=3 2024-03-28T16:50:28.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 20 add_watcher() i=4 2024-03-28T16:50:28.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 20 add_watcher() i=5 2024-03-28T16:50:28.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94241909040016] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.568+0000 7f961d12d980 20 add_watcher() i=6 2024-03-28T16:50:28.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94241909043584] v31'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 20 add_watcher() i=7 2024-03-28T16:50:28.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:28.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782ea70 con 0x55b66781c9c0 2024-03-28T16:50:28.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94241909044800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782ea70 con 0x55b66781c9c0 2024-03-28T16:50:28.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94241909044800] v31'230 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f94d1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 5 note: GC not initialized 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f94d0fe1640 20 reqs_thread_entry: start 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 20 Filter name: none 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f94c9ffb640 20 reqs_thread_entry: start 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667854550 con 0x55b66781c9c0 2024-03-28T16:50:28.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:28.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.572+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667854550 con 0x55b667820f00 2024-03-28T16:50:28.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 152+0+15 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f94d1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:28.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94241909044800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667854550 con 0x55b66781c9c0 2024-03-28T16:50:28.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94241909044800] v31'231 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.577 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94241909019856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667854550 con 0x7f961001fbe0 2024-03-28T16:50:28.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94241909019856] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961d12d980 20 remove_watcher() i=0 2024-03-28T16:50:28.578 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961d12d980 2 removed watcher, disabling cache 2024-03-28T16:50:28.579 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.576+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94241909024304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667854550 con 0x55b66781c9c0 2024-03-28T16:50:28.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.580+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94241909024304] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.580+0000 7f961d12d980 20 remove_watcher() i=1 2024-03-28T16:50:28.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.580+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94241909028016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b667854550 con 0x55b667825770 2024-03-28T16:50:28.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.580+0000 7f961a447640 1 -- 172.21.15.170:0/3266171696 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94241909028016] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9610033040 con 0x55b667825770 2024-03-28T16:50:28.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.580+0000 7f961d12d980 20 remove_watcher() i=2 2024-03-28T16:50:28.583 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.580+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94241909031808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782a630 con 0x7f961001fbe0 2024-03-28T16:50:28.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.584+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94241909031808] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x7f961001fbe0 2024-03-28T16:50:28.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.584+0000 7f961d12d980 20 remove_watcher() i=3 2024-03-28T16:50:28.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.584+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94241909035952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782b4b0 con 0x55b66781c9c0 2024-03-28T16:50:28.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.584+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94241909035952] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f960c0083f0 con 0x55b66781c9c0 2024-03-28T16:50:28.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.584+0000 7f961d12d980 20 remove_watcher() i=4 2024-03-28T16:50:28.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.584+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94241909033056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782c380 con 0x55b667820f00 2024-03-28T16:50:28.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.588+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94241909033056] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.588+0000 7f961d12d980 20 remove_watcher() i=5 2024-03-28T16:50:28.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.588+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94241909040016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782a630 con 0x55b667820f00 2024-03-28T16:50:28.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.588+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94241909040016] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.590 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.588+0000 7f961d12d980 20 remove_watcher() i=6 2024-03-28T16:50:28.591 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.588+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94241909043584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55b66782b4b0 con 0x55b667820f00 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961ac48640 1 -- 172.21.15.170:0/3266171696 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94241909043584] v31'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f96140fc070 con 0x55b667820f00 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 20 remove_watcher() i=7 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b667820f00 msgr2=0x55b6678232f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b667820f00 0x55b6678232f0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b66781c9c0 msgr2=0x55b66781ee80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b66781c9c0 0x55b66781ee80 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f961001fbe0 msgr2=0x7f961003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f961001fbe0 0x7f961003a3c0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b667825770 msgr2=0x55b667827ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55b667825770 0x55b667827ba0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94e40342b0 msgr2=0x7f94e4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94e40342b0 0x7f94e4036770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55b6677efc90 tx=0x7f9610010000 comp rx=0 tx=0).stop 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 msgr2=0x55b667811580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961b449640 1 -- 172.21.15.170:0/3266171696 reap_dead start 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b667811580 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f96140bb000 tx=0x7f96140baf90 comp rx=0 tx=0).stop 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 shutdown_connections 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/3266171696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b66780a710 0x55b667811580 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 >> 172.21.15.170:0/3266171696 conn(0x55b667806a50 msgr2=0x55b667804a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 shutdown_connections 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/3266171696 wait complete. 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94f4034260 msgr2=0x7f94f4036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94f4034260 0x7f94f4036720 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55b6677e8080 tx=0x7f961000f000 comp rx=0 tx=0).stop 2024-03-28T16:50:28.596 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b6677fd810 msgr2=0x55b6677fdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b6677fd810 0x55b6677fdc30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f96140e2290 tx=0x7f96140a9e90 comp rx=0 tx=0).stop 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 shutdown_connections 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f94f4034260 0x7f94f4036720 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/2895079183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b6677fd810 0x55b6677fdc30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 >> 172.21.15.170:0/2895079183 conn(0x55b6677f2620 msgr2=0x55b6677f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 shutdown_connections 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/2895079183 wait complete. 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b6677e8e30 msgr2=0x55b6677eb260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55b6677e8e30 0x55b6677eb260 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b6676f2530 msgr2=0x55b6677e6c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55b6676f2530 0x55b6677e6c80 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.597 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b6677ece10 msgr2=0x55b6677ef200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55b6677ece10 0x55b6677ef200 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9500034300 msgr2=0x7f95000367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9500034300 0x7f95000367c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9610001e50 tx=0x7f96100013b0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 msgr2=0x55b6677e0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 --2- 172.21.15.170:0/362091622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b667661070 0x55b6677e0050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9614000f90 tx=0x7f9614017e30 comp rx=0 tx=0).stop 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961b449640 1 -- 172.21.15.170:0/362091622 reap_dead start 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 shutdown_connections 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 >> 172.21.15.170:0/362091622 conn(0x55b6676e76d0 msgr2=0x55b6677d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 shutdown_connections 2024-03-28T16:50:28.598 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.592+0000 7f961d12d980 1 -- 172.21.15.170:0/362091622 wait complete. 2024-03-28T16:50:28.605 INFO:teuthology.orchestra.run.smithi170.stdout:{"entries":[],"summary":[]} 2024-03-28T16:50:28.605 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-03-28T16:50:28.605 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-03-28T16:50:28.605 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-03-28T16:50:28.605 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-03-28T16:50:28.684 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:28.684 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:28.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db7201980 1 Processor -- start 2024-03-28T16:50:28.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db7201980 1 -- start start 2024-03-28T16:50:28.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db7201980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55eeb49eee10 0x55eeb49ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db7201980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55eeb493c4c0 con 0x55eeb49ef730 2024-03-28T16:50:28.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db7201980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55eeb49030f0 con 0x55eeb49eee10 2024-03-28T16:50:28.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db4d1c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55eeb49eee10 0x55eeb49ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db4d1c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55eeb49eee10 0x55eeb49ef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47712/0 (socket says 172.21.15.170:47712) 2024-03-28T16:50:28.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db4d1c640 1 -- 172.21.15.170:0/1558218669 learned_addr learned my addr 172.21.15.170:0/1558218669 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db451b640 1 --1- 172.21.15.170:0/1558218669 >> v1:172.21.15.119:6789/0 conn(0x55eeb49ef730 0x55eeb49f0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:49650/0 (socket says 172.21.15.170:49650) 2024-03-28T16:50:28.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db4d1c640 1 -- 172.21.15.170:0/1558218669 >> v1:172.21.15.119:6789/0 conn(0x55eeb49ef730 legacy=0x55eeb49f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.684+0000 7f0db4d1c640 1 -- 172.21.15.170:0/1558218669 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb49f0970 con 0x55eeb49eee10 2024-03-28T16:50:28.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/1558218669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49ef1f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00a4e0 tx=0x7f0d9c01b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec31709192f9afd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db3d1a640 1 -- 172.21.15.170:0/1558218669 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0aa540 con 0x55eeb49eee10 2024-03-28T16:50:28.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db3d1a640 1 -- 172.21.15.170:0/1558218669 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0a6070 con 0x55eeb49eee10 2024-03-28T16:50:28.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db3d1a640 1 -- 172.21.15.170:0/1558218669 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0aa850 con 0x55eeb49eee10 2024-03-28T16:50:28.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db7201980 1 -- 172.21.15.170:0/1558218669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 msgr2=0x55eeb49ef1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db7201980 1 --2- 172.21.15.170:0/1558218669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49ef1f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00a4e0 tx=0x7f0d9c01b090 comp rx=0 tx=0).stop 2024-03-28T16:50:28.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db7201980 1 -- 172.21.15.170:0/1558218669 shutdown_connections 2024-03-28T16:50:28.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db7201980 1 --2- 172.21.15.170:0/1558218669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49ef1f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db7201980 1 -- 172.21.15.170:0/1558218669 >> 172.21.15.170:0/1558218669 conn(0x55eeb49ea6e0 msgr2=0x55eeb49ecb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db7201980 1 -- 172.21.15.170:0/1558218669 shutdown_connections 2024-03-28T16:50:28.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.688+0000 7f0db7201980 1 -- 172.21.15.170:0/1558218669 wait complete. 2024-03-28T16:50:28.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 Processor -- start 2024-03-28T16:50:28.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- start start 2024-03-28T16:50:28.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.707 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb4900620 con 0x55eeb49eee10 2024-03-28T16:50:28.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47718/0 (socket says 172.21.15.170:47718) 2024-03-28T16:50:28.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db4d1c640 1 -- 172.21.15.170:0/323891174 learned_addr learned my addr 172.21.15.170:0/323891174 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db4d1c640 1 -- 172.21.15.170:0/323891174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c00a1c0 con 0x55eeb49eee10 2024-03-28T16:50:28.708 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/323891174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49f8d20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c018cc0 tx=0x7f0d9c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85af428ed3ba7d5c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0ca2e0d640 1 -- 172.21.15.170:0/323891174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0aa700 con 0x55eeb49eee10 2024-03-28T16:50:28.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0ca2e0d640 1 -- 172.21.15.170:0/323891174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0aa860 con 0x55eeb49eee10 2024-03-28T16:50:28.709 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0ca2e0d640 1 -- 172.21.15.170:0/323891174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0aab20 con 0x55eeb49eee10 2024-03-28T16:50:28.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- 172.21.15.170:0/323891174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 msgr2=0x55eeb49f8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 --2- 172.21.15.170:0/323891174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49f8d20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c018cc0 tx=0x7f0d9c0033b0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- 172.21.15.170:0/323891174 shutdown_connections 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 --2- 172.21.15.170:0/323891174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb49f8d20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- 172.21.15.170:0/323891174 >> 172.21.15.170:0/323891174 conn(0x55eeb49ea6e0 msgr2=0x55eeb4ad5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- 172.21.15.170:0/323891174 shutdown_connections 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- 172.21.15.170:0/323891174 wait complete. 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 Processor -- start 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- start start 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb4ae33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db7201980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb48fbe00 con 0x55eeb49eee10 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.708+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb4ae33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb4ae33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47726/0 (socket says 172.21.15.170:47726) 2024-03-28T16:50:28.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db4d1c640 1 -- 172.21.15.170:0/757558440 learned_addr learned my addr 172.21.15.170:0/757558440 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db4d1c640 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb49f0970 con 0x55eeb49eee10 2024-03-28T16:50:28.712 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb4ae33f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c018cc0 tx=0x7f0d9c017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0ca160a640 1 -- 172.21.15.170:0/757558440 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0c4070 con 0x55eeb49eee10 2024-03-28T16:50:28.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eeb4ae3930 con 0x55eeb49eee10 2024-03-28T16:50:28.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0ca160a640 1 -- 172.21.15.170:0/757558440 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0b5650 con 0x55eeb49eee10 2024-03-28T16:50:28.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0ca160a640 1 -- 172.21.15.170:0/757558440 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0b58f0 con 0x55eeb49eee10 2024-03-28T16:50:28.713 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb4ae3d60 con 0x55eeb49eee10 2024-03-28T16:50:28.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0ca160a640 1 -- 172.21.15.170:0/757558440 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0d9c0a8070 con 0x55eeb49eee10 2024-03-28T16:50:28.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0ca160a640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c9c034300 0x7f0c9c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0ca160a640 1 -- 172.21.15.170:0/757558440 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0d9c0e3750 con 0x55eeb49eee10 2024-03-28T16:50:28.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb49ef730 0x55eeb4ae9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db451b640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c9c034300 0x7f0c9c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4aea510 con 0x55eeb49ef730 2024-03-28T16:50:28.714 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db551d640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb49ef730 0x55eeb4ae9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db451b640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c9c034300 0x7f0c9c0367c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000f70 tx=0x7f0da4001380 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db551d640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb49ef730 0x55eeb4ae9fd0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db551d640 1 -- 172.21.15.170:0/757558440 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb49ef730 2024-03-28T16:50:28.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.712+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4aea870 con 0x55eeb49ef730 2024-03-28T16:50:28.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db551d640 1 -- 172.21.15.170:0/757558440 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb49ef730 2024-03-28T16:50:28.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4aec200 0x55eeb4aee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4aeeda0 con 0x55eeb4aec200 2024-03-28T16:50:28.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4aec200 0x55eeb4aee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4aec200 0x55eeb4aee610 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db4d1c640 1 -- 172.21.15.170:0/757558440 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4aec200 2024-03-28T16:50:28.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4aeb070 con 0x55eeb49ef730 2024-03-28T16:50:28.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db551d640 1 -- 172.21.15.170:0/757558440 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb49ef730 2024-03-28T16:50:28.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55eeb4af01c0 0x55eeb4af25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4af2c70 con 0x55eeb4af01c0 2024-03-28T16:50:28.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db451b640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55eeb4af01c0 0x55eeb4af25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db451b640 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55eeb4af01c0 0x55eeb4af25b0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db451b640 1 -- 172.21.15.170:0/757558440 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f0da400f6a0 con 0x55eeb4af01c0 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 Processor -- start 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 -- start start 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b00ba0 0x55eeb4b00fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.716+0000 7f0db7201980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb4894e30 con 0x55eeb4b00ba0 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b00ba0 0x55eeb4b00fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b00ba0 0x55eeb4b00fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47738/0 (socket says 172.21.15.170:47738) 2024-03-28T16:50:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db4d1c640 1 -- 172.21.15.170:0/1297369645 learned_addr learned my addr 172.21.15.170:0/1297369645 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db4d1c640 1 -- 172.21.15.170:0/1297369645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb4b01500 con 0x55eeb4b00ba0 2024-03-28T16:50:28.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b00ba0 0x55eeb4b00fc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0d18d0 tx=0x7f0d9c0a95f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0c9a7fc640 1 -- 172.21.15.170:0/1297369645 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0a9a90 con 0x55eeb4b00ba0 2024-03-28T16:50:28.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eeb4b01750 con 0x55eeb4b00ba0 2024-03-28T16:50:28.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0c9a7fc640 1 -- 172.21.15.170:0/1297369645 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0e3a70 con 0x55eeb4b00ba0 2024-03-28T16:50:28.720 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0c9a7fc640 1 -- 172.21.15.170:0/1297369645 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0e3d10 con 0x55eeb4b00ba0 2024-03-28T16:50:28.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb4b022a0 con 0x55eeb4b00ba0 2024-03-28T16:50:28.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0c9a7fc640 1 -- 172.21.15.170:0/1297369645 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0d9c005020 con 0x55eeb4b00ba0 2024-03-28T16:50:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0c9a7fc640 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c8c0342b0 0x7f0c8c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0c9a7fc640 1 -- 172.21.15.170:0/1297369645 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0d9c0cabd0 con 0x55eeb4b00ba0 2024-03-28T16:50:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db451b640 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c8c0342b0 0x7f0c8c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db7201980 1 Processor -- start 2024-03-28T16:50:28.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db7201980 1 -- start start 2024-03-28T16:50:28.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db451b640 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c8c0342b0 0x7f0c8c036770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55eeb4aef130 tx=0x7f0da400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db7201980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b0df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.720+0000 7f0db7201980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb4888790 con 0x55eeb4b0daf0 2024-03-28T16:50:28.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b0df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b0df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47744/0 (socket says 172.21.15.170:47744) 2024-03-28T16:50:28.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 -- 172.21.15.170:0/654171484 learned_addr learned my addr 172.21.15.170:0/654171484 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 -- 172.21.15.170:0/654171484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb4b0e4e0 con 0x55eeb4b0daf0 2024-03-28T16:50:28.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/654171484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b0df10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0ae040 tx=0x7f0d9c0d3a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e949f2aac6912637 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0c8bfff640 1 -- 172.21.15.170:0/654171484 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0d3b40 con 0x55eeb4b0daf0 2024-03-28T16:50:28.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0c8bfff640 1 -- 172.21.15.170:0/654171484 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0d3450 con 0x55eeb4b0daf0 2024-03-28T16:50:28.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0c8bfff640 1 -- 172.21.15.170:0/654171484 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c004e30 con 0x55eeb4b0daf0 2024-03-28T16:50:28.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- 172.21.15.170:0/654171484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 msgr2=0x55eeb4b0df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 --2- 172.21.15.170:0/654171484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b0df10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0ae040 tx=0x7f0d9c0d3a00 comp rx=0 tx=0).stop 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- 172.21.15.170:0/654171484 shutdown_connections 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 --2- 172.21.15.170:0/654171484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b0df10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- 172.21.15.170:0/654171484 >> 172.21.15.170:0/654171484 conn(0x55eeb4b09e80 msgr2=0x55eeb4b07ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- 172.21.15.170:0/654171484 shutdown_connections 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- 172.21.15.170:0/654171484 wait complete. 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 Processor -- start 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- start start 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b149a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb4b14ee0 con 0x55eeb4b0daf0 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b149a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47752/0 (socket says 172.21.15.170:47752) 2024-03-28T16:50:28.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 learned_addr learned my addr 172.21.15.170:0/3065344744 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb4b022a0 con 0x55eeb4b0daf0 2024-03-28T16:50:28.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b149a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0d3a40 tx=0x7f0d9c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0c8a7fc640 1 -- 172.21.15.170:0/3065344744 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0c8c60 con 0x55eeb4b0daf0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55eeb4b022a0 con 0x55eeb4b0daf0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0c8a7fc640 1 -- 172.21.15.170:0/3065344744 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0c8dc0 con 0x55eeb4b0daf0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0c8a7fc640 1 -- 172.21.15.170:0/3065344744 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0be1e0 con 0x55eeb4b0daf0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.724+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb4b152c0 con 0x55eeb4b0daf0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0c8a7fc640 1 -- 172.21.15.170:0/3065344744 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0d9c0fe050 con 0x55eeb4b0daf0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0c8a7fc640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c800342b0 0x7f0c80036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0c8a7fc640 1 -- 172.21.15.170:0/3065344744 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0d9c0f2070 con 0x55eeb4b0daf0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db451b640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c800342b0 0x7f0c80036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb4b1fc00 0x55eeb4b220c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b22810 con 0x55eeb4b1fc00 2024-03-28T16:50:28.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db551d640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb4b1fc00 0x55eeb4b220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db451b640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c800342b0 0x7f0c80036770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c010 tx=0x7f0da4010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.731 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db551d640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb4b1fc00 0x55eeb4b220c0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 realm 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b22be0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b237c0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:28.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4b24170 0x55eeb4b26560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.728+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b26b70 con 0x55eeb4b24170 2024-03-28T16:50:28.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4b24170 0x55eeb4b26560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.733 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db4d1c640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4b24170 0x55eeb4b26560 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db7201980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:28.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db7201980 20 searching for the correct realm 2024-03-28T16:50:28.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55eeb4b28d70 0x55eeb4b2b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2b900 con 0x55eeb4b28d70 2024-03-28T16:50:28.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db451b640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55eeb4b28d70 0x55eeb4b2b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db451b640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55eeb4b28d70 0x55eeb4b2b230 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db451b640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0da401fc60 0x7f0da403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2b900 con 0x7f0da401fc60 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db551d640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0da401fc60 0x7f0da403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db551d640 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0da401fc60 0x7f0da403a3c0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.732+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x7f0da401fc60 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.736+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x7f0da401fc60 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.740+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x7f0da401fc60 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.745 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x7f0da401fc60 2024-03-28T16:50:28.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b28d70 2024-03-28T16:50:28.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.746 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b24170 2024-03-28T16:50:28.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.744+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x55eeb4b1fc00 2024-03-28T16:50:28.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b23460 con 0x7f0da401fc60 2024-03-28T16:50:28.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:28.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b27170 con 0x55eeb4b1fc00 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b27170 con 0x55eeb4b1fc00 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b27120 con 0x7f0da401fc60 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 zone default found 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 4 Realm: () 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 zonegroup default 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b279c0 con 0x55eeb4b24170 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2d720 con 0x55eeb4b28d70 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2da80 con 0x55eeb4b28d70 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 started sync module instance, tier type = 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:28.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.748+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2d000 con 0x7f0da401fc60 2024-03-28T16:50:28.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.752+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.752+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94483722193760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2d3b0 con 0x7f0da401fc60 2024-03-28T16:50:28.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.752+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2cfc0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.752+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94483722193760] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.752+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.752+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94483722198496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2e770 con 0x55eeb4b1fc00 2024-03-28T16:50:28.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.752+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2f0e0 con 0x55eeb4b28d70 2024-03-28T16:50:28.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94483722198496] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94483722202176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2f440 con 0x55eeb4b28d70 2024-03-28T16:50:28.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b2fdb0 con 0x7f0da401fc60 2024-03-28T16:50:28.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94483722205728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b30460 con 0x7f0da401fc60 2024-03-28T16:50:28.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b30f30 con 0x55eeb4b1fc00 2024-03-28T16:50:28.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94483722202176] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.756+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94483722205728] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.760+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.760+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94483722209808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b30ef0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.760+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b30030 con 0x55eeb4b24170 2024-03-28T16:50:28.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.760+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94483722209808] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.760+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.760+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94483722199552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b31780 con 0x55eeb4b24170 2024-03-28T16:50:28.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.760+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b31f80 con 0x55eeb4b24170 2024-03-28T16:50:28.763 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94483722199552] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94483722212064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b327d0 con 0x55eeb4b24170 2024-03-28T16:50:28.764 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b33140 con 0x55eeb4b24170 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94483722218656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b33560 con 0x55eeb4b24170 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 20 add_watcher() i=0 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 20 add_watcher() i=1 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 20 add_watcher() i=2 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 20 add_watcher() i=3 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 20 add_watcher() i=4 2024-03-28T16:50:28.766 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 20 add_watcher() i=5 2024-03-28T16:50:28.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94483722212064] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.767 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.764+0000 7f0db7201980 20 add_watcher() i=6 2024-03-28T16:50:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94483722218656] v31'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db7201980 20 add_watcher() i=7 2024-03-28T16:50:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db7201980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b32440 con 0x55eeb4b1fc00 2024-03-28T16:50:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.768 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94483722219712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b32440 con 0x55eeb4b1fc00 2024-03-28T16:50:28.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94483722219712] v31'232 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0c6afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:28.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db7201980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:28.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db7201980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:28.770 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.768+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3eb20 con 0x55eeb4b1fc00 2024-03-28T16:50:28.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.772+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.772 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.772+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3eb60 con 0x55eeb4b24170 2024-03-28T16:50:28.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.772+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.774 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.772+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3efa0 con 0x55eeb4b28d70 2024-03-28T16:50:28.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.776+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.775 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.776+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3f970 con 0x55eeb4b1fc00 2024-03-28T16:50:28.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.776+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'233 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.776 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.776+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3ccd0 con 0x7f0da401fc60 2024-03-28T16:50:28.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.776+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'74 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.778 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.776+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3ccd0 con 0x55eeb4b28d70 2024-03-28T16:50:28.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.780+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'49 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.780+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b404e0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.780+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.780+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b405f0 con 0x55eeb4b24170 2024-03-28T16:50:28.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.784+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'50 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.783 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.784+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b40c40 con 0x7f0da401fc60 2024-03-28T16:50:28.785 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.784+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.784+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b40c40 con 0x55eeb4b28d70 2024-03-28T16:50:28.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.788+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.788+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41170 con 0x7f0da401fc60 2024-03-28T16:50:28.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.788+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'75 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.788+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41170 con 0x55eeb4b24170 2024-03-28T16:50:28.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.788+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'35 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.788+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b416a0 con 0x7f0da401fc60 2024-03-28T16:50:28.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.792+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.792 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.792+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b416a0 con 0x55eeb4b24170 2024-03-28T16:50:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.792+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'56 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.794 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.792+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41970 con 0x55eeb4b1fc00 2024-03-28T16:50:28.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.796+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'234 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.795 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.796+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41970 con 0x55eeb4b28d70 2024-03-28T16:50:28.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.796+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'68 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.797 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.796+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41970 con 0x7f0da401fc60 2024-03-28T16:50:28.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.796+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'76 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.798 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.796+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41970 con 0x55eeb4b28d70 2024-03-28T16:50:28.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.800+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.800 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.800+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41a00 con 0x55eeb4b1fc00 2024-03-28T16:50:28.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.800+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.802 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.800+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41a00 con 0x55eeb4b1fc00 2024-03-28T16:50:28.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.804+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'235 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.803 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.804+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41a00 con 0x55eeb4b28d70 2024-03-28T16:50:28.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.804+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'69 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.804+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b41a00 con 0x7f0da401fc60 2024-03-28T16:50:28.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'38 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b28d70 2024-03-28T16:50:28.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'31 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b28d70 2024-03-28T16:50:28.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b28d70 2024-03-28T16:50:28.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'24 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.810 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.808+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b28d70 2024-03-28T16:50:28.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.812+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'50 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.812+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b24170 2024-03-28T16:50:28.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.812+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'32 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.812+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b28d70 2024-03-28T16:50:28.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.816+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.816+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b24170 2024-03-28T16:50:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.816+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'51 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.816+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b28d70 2024-03-28T16:50:28.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.820+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'70 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.819 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.820+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x55eeb4b24170 2024-03-28T16:50:28.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.820+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'57 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.821 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.820+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b419e0 con 0x7f0da401fc60 2024-03-28T16:50:28.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0c69fe3640 20 reqs_thread_entry: start 2024-03-28T16:50:28.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:28.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 20 Filter name: none 2024-03-28T16:50:28.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-03-28T16:50:28.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.824 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0bdb7fe640 20 reqs_thread_entry: start 2024-03-28T16:50:28.825 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3aff0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv17 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 20 rados_obj.operate() r=0 bl.length=357 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b5df90 con 0x7f0da401fc60 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv45 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 : miss 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:96 8.6 8:6086b0d0:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3c1b0 con 0x55eeb4b24170 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.824+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(96 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 20 rados_obj.operate() r=0 bl.length=352 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 info.flags=0x17 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 10 adding default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 to cache LRU end 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 4.20 4:07e02a4a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb0f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(97 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:98 4.18 4:183216b6:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb0f0 con 0x7f0da401fc60 2024-03-28T16:50:28.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(98 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:99 4.3f 4:fd732f54:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb0f0 con 0x7f0da401fc60 2024-03-28T16:50:28.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(99 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.829 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:100 4.14 4:29764f6c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb0f0 con 0x55eeb4b28d70 2024-03-28T16:50:28.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(100 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:101 4.2a 4:56fe9d0e:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb0f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(101 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4 [call out=424b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+424 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-03-28T16:50:28.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:102 4.35 4:aea2193a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb620 con 0x55eeb4b1fc00 2024-03-28T16:50:28.830 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(102 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:103 4.e 4:71889ca4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb620 con 0x7f0da401fc60 2024-03-28T16:50:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(103 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:104 4.15 4:ab6fc15d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb620 con 0x55eeb4b28d70 2024-03-28T16:50:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(104 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.828+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:105 4.2b 4:d49ebdda:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb620 con 0x55eeb4b1fc00 2024-03-28T16:50:28.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(105 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:106 4.22 4:461b5392:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb620 con 0x55eeb4b1fc00 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(106 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:107 4.32 4:4c963a44:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdb620 con 0x55eeb4b1fc00 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(107 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 20 get_obj_state: octx=0x7f0dac008c30 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55eeb4bd94a8 s->prefetch_data=0 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:108 3.15 3:a8185fbe:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.7_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bda510 con 0x7f0da401fc60 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(108 b88c2392-6ef8-4660-b080-19480b748fc0.4221.7_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=875b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+891 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 10 manifest: total_size = 6 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 20 get_obj_state: setting s->obj_tag to b88c2392-6ef8-4660-b080-19480b748fc0.4221.202659804594569045 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 20 get_obj_state: octx=0x7f0dac008c30 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55eeb4bd94a8 s->prefetch_data=0 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 20 prepare_atomic_modification: state is not atomic. state=0x55eeb4bd94a8 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 20 bucket index object: :.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4 2024-03-28T16:50:28.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:109 4.2a 4:56fe9d0e:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b62500 con 0x55eeb4b1fc00 2024-03-28T16:50:28.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(109 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4 [stat,call,call] v31'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.832+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:110 3.15 3:a8185fbe:::b88c2392-6ef8-4660-b080-19480b748fc0.4221.7_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b604e0 con 0x7f0da401fc60 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(110 b88c2392-6ef8-4660-b080-19480b748fc0.4221.7_15f6f8dc036519d7fe15b39338f6e5db [call] v31'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:111 4.2a 4:56fe9d0e:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b62500 con 0x55eeb4b1fc00 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 20 list_objects_ordered: starting attempt 1 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:112 4.20 4:07e02a4a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bda6f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:113 4.18 4:183216b6:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b62500 con 0x7f0da401fc60 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:114 4.3f 4:fd732f54:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b68d40 con 0x7f0da401fc60 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:115 4.14 4:29764f6c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6b760 con 0x55eeb4b28d70 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:116 4.2a 4:56fe9d0e:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6e260 con 0x55eeb4b1fc00 2024-03-28T16:50:28.836 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:117 4.35 4:aea2193a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b68d40 con 0x55eeb4b1fc00 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:118 4.e 4:71889ca4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b732a0 con 0x7f0da401fc60 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:119 4.15 4:ab6fc15d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b75d10 con 0x55eeb4b28d70 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:120 4.2b 4:d49ebdda:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b78710 con 0x55eeb4b1fc00 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:121 4.22 4:461b5392:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b7b150 con 0x55eeb4b1fc00 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:122 4.32 4:4c963a44:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b7db10 con 0x55eeb4b1fc00 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(112 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 25 ==== osd_op_reply(113 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 26 ==== osd_op_reply(114 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(115 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 27 ==== osd_op_reply(118 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(119 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 36 ==== osd_op_reply(117 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 37 ==== osd_op_reply(120 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 38 ==== osd_op_reply(121 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 39 ==== osd_op_reply(122 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 40 ==== osd_op_reply(111 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4 [stat,call,call] v31'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0c8bfff640 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 41 ==== osd_op_reply(116 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[b88c2392-6ef8-4660-b080-19480b748fc0.4221.7])"; uploads.size()=0; is_truncated=0 2024-03-28T16:50:28.841 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:123 4.20 4:07e02a4a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b787a0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:124 4.18 4:183216b6:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b73300 con 0x7f0da401fc60 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:125 4.3f 4:fd732f54:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b79b80 con 0x7f0da401fc60 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:126 4.14 4:29764f6c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6a640 con 0x55eeb4b28d70 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:127 4.2a 4:56fe9d0e:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6aa80 con 0x55eeb4b1fc00 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:128 4.35 4:aea2193a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b787a0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:129 4.e 4:71889ca4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6b330 con 0x7f0da401fc60 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:130 4.15 4:ab6fc15d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6aa80 con 0x55eeb4b28d70 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:131 4.2b 4:d49ebdda:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b7dab0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:132 4.22 4:461b5392:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b787a0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:133 4.32 4:4c963a44:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b73300 con 0x55eeb4b1fc00 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(126 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 31 ==== osd_op_reply(130 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 28 ==== osd_op_reply(124 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 29 ==== osd_op_reply(125 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 30 ==== osd_op_reply(129 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 42 ==== osd_op_reply(123 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 43 ==== osd_op_reply(127 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 44 ==== osd_op_reply(128 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 45 ==== osd_op_reply(132 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 46 ==== osd_op_reply(131 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 47 ==== osd_op_reply(133 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.836+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:134 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6aa80 con 0x7f0da401fc60 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 31 ==== osd_op_reply(134 foo.buckets [call] v31'48 uv48 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:135 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b73300 con 0x7f0da401fc60 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 32 ==== osd_op_reply(135 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv42 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 20 rados_obj.operate() r=0 bl.length=165 2024-03-28T16:50:28.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-03-28T16:50:28.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-03-28T16:50:28.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 10 removing default.rgw.meta+root+myfoo from cache 2024-03-28T16:50:28.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-03-28T16:50:28.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:136 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94483722903520 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6db60 con 0x55eeb4b24170 2024-03-28T16:50:28.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== watch-notify(notify (1) cookie 94483722212064 notify 133143986223 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x55eeb4b0e4e0 con 0x55eeb4b24170 2024-03-28T16:50:28.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0c8b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986223 cookie 94483722212064 notifier 4774 bl.length()=149 2024-03-28T16:50:28.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(136 notify.6 [notify cookie 94483722903520 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.840+0000 7f0c8b7fe640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:137 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f0be80032c0 con 0x55eeb4b24170 2024-03-28T16:50:28.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== watch-notify(notify_complete (2) cookie 94483722903520 notify 133143986223 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f0d9c0e7070 con 0x55eeb4b24170 2024-03-28T16:50:28.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(137 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c0b47c0 con 0x55eeb4b24170 2024-03-28T16:50:28.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:138 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b78190 con 0x7f0da401fc60 2024-03-28T16:50:28.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 33 ==== osd_op_reply(138 myfoo [call,call,delete] v31'49 uv49 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db7201980 10 removing default.rgw.meta+root+.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 from cache 2024-03-28T16:50:28.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db7201980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7, ofs0, ns] 2024-03-28T16:50:28.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:139 7.9 7:93e5b521:::notify.7:head [notify cookie 94483722502544 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6db60 con 0x55eeb4b24170 2024-03-28T16:50:28.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== watch-notify(notify (1) cookie 94483722218656 notify 133143986224 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x7f0d9c0e7070 con 0x55eeb4b24170 2024-03-28T16:50:28.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0c8b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986224 cookie 94483722218656 notifier 4774 bl.length()=206 2024-03-28T16:50:28.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(139 notify.7 [notify cookie 94483722502544 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.844+0000 7f0c8b7fe640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:140 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f0be80032c0 con 0x55eeb4b24170 2024-03-28T16:50:28.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== watch-notify(notify_complete (2) cookie 94483722502544 notify 133143986224 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55eeb4b01500 con 0x55eeb4b24170 2024-03-28T16:50:28.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(140 notify.7 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:141 8.6 8:6086b0d0:root::.bucket.meta.myfoo%3ab88c2392-6ef8-4660-b080-19480b748fc0.4221.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6db60 con 0x55eeb4b24170 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(141 .bucket.meta.myfoo:b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 [call,call,delete] v31'2 uv2 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:142 4.20 4:07e02a4a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6db60 con 0x55eeb4b1fc00 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:143 4.18 4:183216b6:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9fe0 con 0x7f0da401fc60 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:144 4.3f 4:fd732f54:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6af20 con 0x7f0da401fc60 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:145 4.14 4:29764f6c:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6b300 con 0x55eeb4b28d70 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:146 4.2a 4:56fe9d0e:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9740 con 0x55eeb4b1fc00 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:147 4.35 4:aea2193a:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6db60 con 0x55eeb4b1fc00 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:148 4.e 4:71889ca4:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b7a860 con 0x7f0da401fc60 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:149 4.15 4:ab6fc15d:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9740 con 0x55eeb4b28d70 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:150 4.2b 4:d49ebdda:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6db60 con 0x55eeb4b1fc00 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:151 4.22 4:461b5392:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6af20 con 0x55eeb4b1fc00 2024-03-28T16:50:28.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.848+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:152 4.32 4:4c963a44:::.dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b69170 con 0x55eeb4b1fc00 2024-03-28T16:50:28.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 48 ==== osd_op_reply(146 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.4 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 34 ==== osd_op_reply(144 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.2 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 35 ==== osd_op_reply(148 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.6 [delete] v31'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 36 ==== osd_op_reply(143 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.1 [delete] v31'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 49 ==== osd_op_reply(142 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.0 [delete] v31'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 50 ==== osd_op_reply(150 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.8 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 51 ==== osd_op_reply(151 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.9 [delete] v31'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 32 ==== osd_op_reply(149 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.7 [delete] v31'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 52 ==== osd_op_reply(152 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.10 [delete] v31'4 uv4 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 53 ==== osd_op_reply(147 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.5 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 33 ==== osd_op_reply(145 .dir.b88c2392-6ef8-4660-b080-19480b748fc0.4221.7.3 [delete] v31'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 : miss 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 20 rados->read ofs=0 len=0 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:153 6.f 6:f5d4135a:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9fe0 con 0x7f0da401fc60 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 37 ==== osd_op_reply(153 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 info.flags=0x0 2024-03-28T16:50:28.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 10 adding default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 to cache LRU end 2024-03-28T16:50:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 10 removing default.rgw.log++pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 from cache 2024-03-28T16:50:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.log:pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.7, ofs0, ns] 2024-03-28T16:50:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:154 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94483722461008 in=215b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6a7c0 con 0x55eeb4b24170 2024-03-28T16:50:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== watch-notify(notify (1) cookie 94483722212064 notify 133143986225 ret 0) v3 ==== 245+0+0 (crc 0 0 0) 0x55eeb4b01500 con 0x55eeb4b24170 2024-03-28T16:50:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0c8bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986225 cookie 94483722212064 notifier 4774 bl.length()=203 2024-03-28T16:50:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== osd_op_reply(154 notify.6 [notify cookie 94483722461008 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0c8bfff640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:155 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f0c78003d60 con 0x55eeb4b24170 2024-03-28T16:50:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.852+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(155 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.856+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== watch-notify(notify_complete (2) cookie 94483722461008 notify 133143986225 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55eeb4ae3d60 con 0x55eeb4b24170 2024-03-28T16:50:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.856+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:156 6.f 6:f5d4135a:::pubsub..bucket.myfoo%2fb88c2392-6ef8-4660-b080-19480b748fc0.4221.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9fe0 con 0x7f0da401fc60 2024-03-28T16:50:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.856+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 38 ==== osd_op_reply(156 pubsub..bucket.myfoo/b88c2392-6ef8-4660-b080-19480b748fc0.4221.7 [delete] v31'35 uv27 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.856+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:157 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9fe0 con 0x7f0da401fc60 2024-03-28T16:50:28.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 39 ==== osd_op_reply(157 foo.buckets [call] v31'50 uv50 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db7201980 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-03-28T16:50:28.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db7201980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-03-28T16:50:28.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:158 7.e 7:7759931f:::notify.3:head [notify cookie 94483722461008 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3b390 con 0x7f0da401fc60 2024-03-28T16:50:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 40 ==== watch-notify(notify (1) cookie 94483722205728 notify 133143986193 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x7f0dac051060 con 0x7f0da401fc60 2024-03-28T16:50:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0c8b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986193 cookie 94483722205728 notifier 4774 bl.length()=170 2024-03-28T16:50:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 41 ==== osd_op_reply(158 notify.3 [notify cookie 94483722461008 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0c8b7fe640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:159 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f0be8002e60 con 0x7f0da401fc60 2024-03-28T16:50:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 42 ==== watch-notify(notify_complete (2) cookie 94483722461008 notify 133143986193 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f0dac04b1b0 con 0x7f0da401fc60 2024-03-28T16:50:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 43 ==== osd_op_reply(159 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:160 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9fe0 con 0x55eeb4b24170 2024-03-28T16:50:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(160 9te6NH5mcdcq0Tc5i8i1 [delete] v31'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db7201980 10 removing email index: foo@foo.com 2024-03-28T16:50:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db7201980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-03-28T16:50:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:161 7.e 7:7759931f:::notify.3:head [notify cookie 94483722461008 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3b390 con 0x7f0da401fc60 2024-03-28T16:50:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 44 ==== watch-notify(notify (1) cookie 94483722205728 notify 133143986194 ret 0) v3 ==== 204+0+0 (crc 0 0 0) 0x7f0dac04b1b0 con 0x7f0da401fc60 2024-03-28T16:50:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0c8b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986194 cookie 94483722205728 notifier 4774 bl.length()=162 2024-03-28T16:50:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 45 ==== osd_op_reply(161 notify.3 [notify cookie 94483722461008 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.860+0000 7f0c8b7fe640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:162 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f0be8002e60 con 0x7f0da401fc60 2024-03-28T16:50:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 46 ==== watch-notify(notify_complete (2) cookie 94483722461008 notify 133143986194 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f0dac00ad90 con 0x7f0da401fc60 2024-03-28T16:50:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 47 ==== osd_op_reply(162 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:163 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bd9fe0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 54 ==== osd_op_reply(163 foo@foo.com [delete] v31'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db7201980 10 removing user buckets index 2024-03-28T16:50:28.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db7201980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-03-28T16:50:28.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify cookie 94483722461008 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3b390 con 0x55eeb4b24170 2024-03-28T16:50:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== watch-notify(notify (1) cookie 94483722199552 notify 133143986226 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x55eeb4ae3d60 con 0x55eeb4b24170 2024-03-28T16:50:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0c8b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986226 cookie 94483722199552 notifier 4774 bl.length()=160 2024-03-28T16:50:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0c8b7fe640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:165 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f0be8002e60 con 0x55eeb4b24170 2024-03-28T16:50:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 41 ==== osd_op_reply(164 notify.5 [notify cookie 94483722461008 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 42 ==== osd_op_reply(165 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 43 ==== watch-notify(notify_complete (2) cookie 94483722461008 notify 133143986226 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55eeb4ae3930 con 0x55eeb4b24170 2024-03-28T16:50:28.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.864+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:166 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6df50 con 0x7f0da401fc60 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 48 ==== osd_op_reply(166 foo.buckets [delete] v31'51 uv51 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db7201980 10 removing user index: foo 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db7201980 10 removing default.rgw.meta+users.uid+foo from cache 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db7201980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:167 7.d 7:b4812045:::notify.4:head [notify cookie 94483722903520 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b3b390 con 0x55eeb4b1fc00 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 55 ==== watch-notify(notify (1) cookie 94483722209808 notify 133143986198 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x7f0dac00ad90 con 0x55eeb4b1fc00 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0c8bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986198 cookie 94483722209808 notifier 4774 bl.length()=152 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0c8bfff640 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:168 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f0c78003d60 con 0x55eeb4b1fc00 2024-03-28T16:50:28.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 56 ==== osd_op_reply(167 notify.4 [notify cookie 94483722903520 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 57 ==== osd_op_reply(168 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 58 ==== watch-notify(notify_complete (2) cookie 94483722903520 notify 133143986198 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f0dac055720 con 0x55eeb4b1fc00 2024-03-28T16:50:28.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.868+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:169 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4b6df50 con 0x55eeb4b1fc00 2024-03-28T16:50:28.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 59 ==== osd_op_reply(169 foo [call,call,delete] v31'19 uv19 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0c6afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:28.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:170 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94483722219712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 60 ==== osd_op_reply(170 data_loggenerations_metadata [watch unwatch cookie 94483722219712] v31'236 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:171 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94483722193760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x7f0da401fc60 2024-03-28T16:50:28.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 49 ==== osd_op_reply(171 notify.0 [watch unwatch cookie 94483722193760] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db7201980 20 remove_watcher() i=0 2024-03-28T16:50:28.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db7201980 2 removed watcher, disabling cache 2024-03-28T16:50:28.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.872+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:172 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94483722198496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.876+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 61 ==== osd_op_reply(172 notify.1 [watch unwatch cookie 94483722198496] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.876+0000 7f0db7201980 20 remove_watcher() i=1 2024-03-28T16:50:28.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.876+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:173 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94483722202176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x55eeb4b28d70 2024-03-28T16:50:28.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.876+0000 7f0db451b640 1 -- 172.21.15.170:0/3065344744 <== osd.3 v2:172.21.15.170:6816/2102412426 34 ==== osd_op_reply(173 notify.2 [watch unwatch cookie 94483722202176] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0da4033040 con 0x55eeb4b28d70 2024-03-28T16:50:28.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db7201980 20 remove_watcher() i=2 2024-03-28T16:50:28.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:174 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94483722205728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x7f0da401fc60 2024-03-28T16:50:28.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.2 v2:172.21.15.170:6800/447751618 50 ==== osd_op_reply(174 notify.3 [watch unwatch cookie 94483722205728] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x7f0da401fc60 2024-03-28T16:50:28.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db7201980 20 remove_watcher() i=3 2024-03-28T16:50:28.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:175 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94483722209808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 <== osd.1 v2:172.21.15.170:6805/11349688 62 ==== osd_op_reply(175 notify.4 [watch unwatch cookie 94483722209808] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0dac0083f0 con 0x55eeb4b1fc00 2024-03-28T16:50:28.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db7201980 20 remove_watcher() i=4 2024-03-28T16:50:28.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.880+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:176 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94483722199552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x55eeb4b24170 2024-03-28T16:50:28.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.884+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 44 ==== osd_op_reply(176 notify.5 [watch unwatch cookie 94483722199552] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.884+0000 7f0db7201980 20 remove_watcher() i=5 2024-03-28T16:50:28.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.884+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:177 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94483722212064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x55eeb4b24170 2024-03-28T16:50:28.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.892+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 45 ==== osd_op_reply(177 notify.6 [watch unwatch cookie 94483722212064] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.892+0000 7f0db7201980 20 remove_watcher() i=6 2024-03-28T16:50:28.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.892+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:178 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94483722218656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55eeb4bdace0 con 0x55eeb4b24170 2024-03-28T16:50:28.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db4d1c640 1 -- 172.21.15.170:0/3065344744 <== osd.0 v2:172.21.15.119:6800/2028815515 46 ==== osd_op_reply(178 notify.7 [watch unwatch cookie 94483722218656] v31'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f0d9c126040 con 0x55eeb4b24170 2024-03-28T16:50:28.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 20 remove_watcher() i=7 2024-03-28T16:50:28.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4b24170 msgr2=0x55eeb4b26560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4b24170 0x55eeb4b26560 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb4b1fc00 msgr2=0x55eeb4b220c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb4b1fc00 0x55eeb4b220c0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0da401fc60 msgr2=0x7f0da403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f0da401fc60 0x7f0da403a3c0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55eeb4b28d70 msgr2=0x55eeb4b2b230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55eeb4b28d70 0x55eeb4b2b230 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c800342b0 msgr2=0x7f0c80036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c800342b0 0x7f0c80036770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c010 tx=0x7f0da4010000 comp rx=0 tx=0).stop 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 msgr2=0x55eeb4b149a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/3065344744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b0daf0 0x55eeb4b149a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0d3a40 tx=0x7f0d9c009f50 comp rx=0 tx=0).stop 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 reap_dead start 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db551d640 1 -- 172.21.15.170:0/3065344744 reap_dead start 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 shutdown_connections 2024-03-28T16:50:28.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 >> 172.21.15.170:0/3065344744 conn(0x55eeb4b09e80 msgr2=0x55eeb4b074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 shutdown_connections 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/3065344744 wait complete. 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c8c0342b0 msgr2=0x7f0c8c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c8c0342b0 0x7f0c8c036770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55eeb4aef130 tx=0x7f0da400f000 comp rx=0 tx=0).stop 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b00ba0 msgr2=0x55eeb4b00fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b00ba0 0x55eeb4b00fc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0d18d0 tx=0x7f0d9c0a95f0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 shutdown_connections 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c8c0342b0 0x7f0c8c036770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/1297369645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb4b00ba0 0x55eeb4b00fc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 >> 172.21.15.170:0/1297369645 conn(0x55eeb4af59b0 msgr2=0x55eeb4af64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 shutdown_connections 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/1297369645 wait complete. 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4aec200 msgr2=0x55eeb4aee610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55eeb4aec200 0x55eeb4aee610 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb49ef730 msgr2=0x55eeb4ae9fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55eeb49ef730 0x55eeb4ae9fd0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55eeb4af01c0 msgr2=0x55eeb4af25b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55eeb4af01c0 0x55eeb4af25b0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c9c034300 msgr2=0x7f0c9c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c9c034300 0x7f0c9c0367c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000f70 tx=0x7f0da4001380 comp rx=0 tx=0).stop 2024-03-28T16:50:28.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 msgr2=0x55eeb4ae33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 --2- 172.21.15.170:0/757558440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eeb49eee10 0x55eeb4ae33f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c018cc0 tx=0x7f0d9c017e30 comp rx=0 tx=0).stop 2024-03-28T16:50:28.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db551d640 1 -- 172.21.15.170:0/757558440 reap_dead start 2024-03-28T16:50:28.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 shutdown_connections 2024-03-28T16:50:28.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 >> 172.21.15.170:0/757558440 conn(0x55eeb49ea6e0 msgr2=0x55eeb49f7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 shutdown_connections 2024-03-28T16:50:28.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.896+0000 7f0db7201980 1 -- 172.21.15.170:0/757558440 wait complete. 2024-03-28T16:50:28.910 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-03-28T16:50:28.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-03-28T16:50:28.910 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-03-28T16:50:28.986 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:28.987 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:28.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e6aec3980 1 Processor -- start 2024-03-28T16:50:28.987 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e6aec3980 1 -- start start 2024-03-28T16:50:28.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e6aec3980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c1173ea140 0x55c1173e86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:28.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e6aec3980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c1173c0520 con 0x55c1173e8c30 2024-03-28T16:50:28.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e6aec3980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c1173870f0 con 0x55c1173ea140 2024-03-28T16:50:28.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e689de640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c1173ea140 0x55c1173e86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:28.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e689de640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55c1173ea140 0x55c1173e86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47760/0 (socket says 172.21.15.170:47760) 2024-03-28T16:50:28.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e689de640 1 -- 172.21.15.170:0/2900871684 learned_addr learned my addr 172.21.15.170:0/2900871684 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:28.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e689de640 1 -- 172.21.15.170:0/2900871684 >> v1:172.21.15.119:6789/0 conn(0x55c1173e8c30 legacy=0x55c11747bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.988+0000 7f9e689de640 1 -- 172.21.15.170:0/2900871684 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1173e9040 con 0x55c1173ea140 2024-03-28T16:50:28.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e689de640 1 --2- 172.21.15.170:0/2900871684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173ea140 0x55c1173e86f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800a4e0 tx=0x7f9e5801b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8195ff32b6f4ec90 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:28.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e679dc640 1 -- 172.21.15.170:0/2900871684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580ab410 con 0x55c1173ea140 2024-03-28T16:50:28.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e679dc640 1 -- 172.21.15.170:0/2900871684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580ab570 con 0x55c1173ea140 2024-03-28T16:50:28.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e679dc640 1 -- 172.21.15.170:0/2900871684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580ab890 con 0x55c1173ea140 2024-03-28T16:50:28.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2900871684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173ea140 msgr2=0x55c1173e86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:28.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/2900871684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173ea140 0x55c1173e86f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800a4e0 tx=0x7f9e5801b030 comp rx=0 tx=0).stop 2024-03-28T16:50:28.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2900871684 shutdown_connections 2024-03-28T16:50:28.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/2900871684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173ea140 0x55c1173e86f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:28.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2900871684 >> 172.21.15.170:0/2900871684 conn(0x55c11746f810 msgr2=0x55c117471c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:28.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2900871684 shutdown_connections 2024-03-28T16:50:28.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:28.992+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2900871684 wait complete. 2024-03-28T16:50:29.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e6aec3980 1 Processor -- start 2024-03-28T16:50:29.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e6aec3980 1 -- start start 2024-03-28T16:50:29.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e6aec3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c1175605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e6aec3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c117384620 con 0x55c1173e8c30 2024-03-28T16:50:29.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c1175605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c1175605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47770/0 (socket says 172.21.15.170:47770) 2024-03-28T16:50:29.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e689de640 1 -- 172.21.15.170:0/3987434976 learned_addr learned my addr 172.21.15.170:0/3987434976 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e689de640 1 -- 172.21.15.170:0/3987434976 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5800a1c0 con 0x55c1173e8c30 2024-03-28T16:50:29.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9e689de640 1 --2- 172.21.15.170:0/3987434976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c1175605b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9e58018cc0 tx=0x7f9e580033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6012c5826080485e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9d56e0d640 1 -- 172.21.15.170:0/3987434976 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580ab810 con 0x55c1173e8c30 2024-03-28T16:50:29.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9d56e0d640 1 -- 172.21.15.170:0/3987434976 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580ab970 con 0x55c1173e8c30 2024-03-28T16:50:29.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.008+0000 7f9d56e0d640 1 -- 172.21.15.170:0/3987434976 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580abc10 con 0x55c1173e8c30 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3987434976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 msgr2=0x55c1175605b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/3987434976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c1175605b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9e58018cc0 tx=0x7f9e580033b0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3987434976 shutdown_connections 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/3987434976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c1175605b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3987434976 >> 172.21.15.170:0/3987434976 conn(0x55c11746f810 msgr2=0x55c11755c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3987434976 shutdown_connections 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3987434976 wait complete. 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 Processor -- start 2024-03-28T16:50:29.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- start start 2024-03-28T16:50:29.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c117567200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11737fe00 con 0x55c1173e8c30 2024-03-28T16:50:29.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c117567200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c117567200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47784/0 (socket says 172.21.15.170:47784) 2024-03-28T16:50:29.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e689de640 1 -- 172.21.15.170:0/1085356753 learned_addr learned my addr 172.21.15.170:0/1085356753 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e689de640 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580af040 con 0x55c1173e8c30 2024-03-28T16:50:29.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e689de640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c117567200 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9e58003430 tx=0x7f9e58017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9d5560a640 1 -- 172.21.15.170:0/1085356753 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580bd540 con 0x55c1173e8c30 2024-03-28T16:50:29.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c117567740 con 0x55c1173e8c30 2024-03-28T16:50:29.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c117567bd0 con 0x55c1173e8c30 2024-03-28T16:50:29.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9d5560a640 1 -- 172.21.15.170:0/1085356753 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580bd6a0 con 0x55c1173e8c30 2024-03-28T16:50:29.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9d5560a640 1 -- 172.21.15.170:0/1085356753 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580bd940 con 0x55c1173e8c30 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.012+0000 7f9d5560a640 1 -- 172.21.15.170:0/1085356753 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9e580bdaa0 con 0x55c1173e8c30 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9d5560a640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d50034300 0x7f9d500367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9d5560a640 1 -- 172.21.15.170:0/1085356753 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9e580e3400 con 0x55c1173e8c30 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d50034300 0x7f9d500367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1173ea140 0x55c11756de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c11756e3c0 con 0x55c1173ea140 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e691df640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1173ea140 0x55c11756de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d50034300 0x7f9d500367c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e50001e20 tx=0x7f9e500013a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e691df640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1173ea140 0x55c11756de80 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e691df640 1 -- 172.21.15.170:0/1085356753 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1173ea140 2024-03-28T16:50:29.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c11756e720 con 0x55c1173ea140 2024-03-28T16:50:29.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e691df640 1 -- 172.21.15.170:0/1085356753 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1173ea140 2024-03-28T16:50:29.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c117570440 0x55c117572830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c117572d70 con 0x55c117570440 2024-03-28T16:50:29.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e689de640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c117570440 0x55c117572830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e689de640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c117570440 0x55c117572830 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e689de640 1 -- 172.21.15.170:0/1085356753 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f9e580c3020 con 0x55c117570440 2024-03-28T16:50:29.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c11756f280 con 0x55c1173ea140 2024-03-28T16:50:29.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e691df640 1 -- 172.21.15.170:0/1085356753 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1173ea140 2024-03-28T16:50:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c117574100 0x55c1175764f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c117576b90 con 0x55c117574100 2024-03-28T16:50:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.016+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c117574100 0x55c1175764f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c117574100 0x55c1175764f0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e681dd640 1 -- 172.21.15.170:0/1085356753 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f9e50010630 con 0x55c117574100 2024-03-28T16:50:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e6aec3980 1 Processor -- start 2024-03-28T16:50:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e6aec3980 1 -- start start 2024-03-28T16:50:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e6aec3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c117584ac0 0x55c117584ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e6aec3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c117318e30 con 0x55c117584ac0 2024-03-28T16:50:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c117584ac0 0x55c117584ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c117584ac0 0x55c117584ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47796/0 (socket says 172.21.15.170:47796) 2024-03-28T16:50:29.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e689de640 1 -- 172.21.15.170:0/2629058815 learned_addr learned my addr 172.21.15.170:0/2629058815 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e689de640 1 -- 172.21.15.170:0/2629058815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c117585420 con 0x55c117584ac0 2024-03-28T16:50:29.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e689de640 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c117584ac0 0x55c117584ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e5801b050 tx=0x7f9e580b3910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9d4e7fc640 1 -- 172.21.15.170:0/2629058815 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580e0040 con 0x55c117584ac0 2024-03-28T16:50:29.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c117585670 con 0x55c117584ac0 2024-03-28T16:50:29.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9d4e7fc640 1 -- 172.21.15.170:0/2629058815 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580e1cb0 con 0x55c117584ac0 2024-03-28T16:50:29.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9d4e7fc640 1 -- 172.21.15.170:0/2629058815 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580c0120 con 0x55c117584ac0 2024-03-28T16:50:29.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1175861c0 con 0x55c117584ac0 2024-03-28T16:50:29.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.020+0000 7f9d4e7fc640 1 -- 172.21.15.170:0/2629058815 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9e580c0280 con 0x55c117584ac0 2024-03-28T16:50:29.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9d4e7fc640 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d400342b0 0x7f9d40036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9d4e7fc640 1 -- 172.21.15.170:0/2629058815 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9e580b8040 con 0x55c117584ac0 2024-03-28T16:50:29.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e681dd640 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d400342b0 0x7f9d40036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 Processor -- start 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- start start 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e681dd640 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d400342b0 0x7f9d40036770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55c11756b8d0 tx=0x7f9e50006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117591cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11730c790 con 0x55c1175918b0 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117591cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117591cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47806/0 (socket says 172.21.15.170:47806) 2024-03-28T16:50:29.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 -- 172.21.15.170:0/3577941226 learned_addr learned my addr 172.21.15.170:0/3577941226 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 -- 172.21.15.170:0/3577941226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1175922a0 con 0x55c1175918b0 2024-03-28T16:50:29.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 --2- 172.21.15.170:0/3577941226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117591cd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9e580ecc30 tx=0x7f9e580ec510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39b904845d03801 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9d3ffff640 1 -- 172.21.15.170:0/3577941226 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580f1070 con 0x55c1175918b0 2024-03-28T16:50:29.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9d3ffff640 1 -- 172.21.15.170:0/3577941226 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580f2040 con 0x55c1175918b0 2024-03-28T16:50:29.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9d3ffff640 1 -- 172.21.15.170:0/3577941226 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580c0530 con 0x55c1175918b0 2024-03-28T16:50:29.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3577941226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 msgr2=0x55c117591cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/3577941226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117591cd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9e580ecc30 tx=0x7f9e580ec510 comp rx=0 tx=0).stop 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3577941226 shutdown_connections 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/3577941226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117591cd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3577941226 >> 172.21.15.170:0/3577941226 conn(0x55c11758dbf0 msgr2=0x55c11758bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3577941226 shutdown_connections 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- 172.21.15.170:0/3577941226 wait complete. 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 Processor -- start 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- start start 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117598740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e6aec3980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c117598c80 con 0x55c1175918b0 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117598740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117598740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47808/0 (socket says 172.21.15.170:47808) 2024-03-28T16:50:29.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.024+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 learned_addr learned my addr 172.21.15.170:0/1143626025 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c117585420 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e689de640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117598740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9e580e0720 tx=0x7f9e580acde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9d3e7fc640 1 -- 172.21.15.170:0/1143626025 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580d2c60 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c117585420 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9d3e7fc640 1 -- 172.21.15.170:0/1143626025 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580d2dc0 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9d3e7fc640 1 -- 172.21.15.170:0/1143626025 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e580cf7f0 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c117585420 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9d3e7fc640 1 -- 172.21.15.170:0/1143626025 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9e580cf950 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9d3e7fc640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d340342b0 0x7f9d34036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9d3e7fc640 1 -- 172.21.15.170:0/1143626025 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9e5810cbf0 con 0x55c1175918b0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d340342b0 0x7f9d34036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1175a3970 0x55c1175a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a6560 con 0x55c1175a3970 2024-03-28T16:50:29.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e691df640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1175a3970 0x55c1175a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d340342b0 0x7f9d34036770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e50010fd0 tx=0x7f9e50011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e691df640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1175a3970 0x55c1175a5e10 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 20 realm 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a6930 con 0x55c1175a3970 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.028+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a74e0 con 0x55c1175a3970 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1175a7e90 0x55c1175aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175aa800 con 0x55c1175a7e90 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e689de640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1175a7e90 0x55c1175aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e689de640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1175a7e90 0x55c1175aa280 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 20 rados_obj.operate() r=0 bl.length=874 2024-03-28T16:50:29.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 20 searching for the correct realm 2024-03-28T16:50:29.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1175ac700 0x55c1175aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175af220 con 0x55c1175ac700 2024-03-28T16:50:29.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1175ac700 0x55c1175aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1175ac700 0x55c1175aeb50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e681dd640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e5000f3d0 0x7f9e5000f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175af220 con 0x7f9e5000f3d0 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e691df640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e5000f3d0 0x7f9e5000f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.032+0000 7f9e691df640 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e5000f3d0 0x7f9e5000f7f0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.037 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x7f9e5000f3d0 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.036+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.039 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x7f9e5000f3d0 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.040+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x7f9e5000f3d0 2024-03-28T16:50:29.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x7f9e5000f3d0 2024-03-28T16:50:29.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.044+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175ac700 2024-03-28T16:50:29.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a7e90 2024-03-28T16:50:29.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x55c1175a3970 2024-03-28T16:50:29.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175a7180 con 0x7f9e5000f3d0 2024-03-28T16:50:29.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:29.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175ab070 con 0x55c1175a3970 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175ab070 con 0x55c1175a3970 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175ab040 con 0x7f9e5000f3d0 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+398 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados_obj.operate() r=0 bl.length=398 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 zone default found 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 4 Realm: () 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 zonegroup default 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.051 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175ab8c0 con 0x55c1175a7e90 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175abcc0 con 0x55c1175ac700 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.048+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b10c0 con 0x55c1175ac700 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 20 started sync module instance, tier type = 2024-03-28T16:50:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b0920 con 0x7f9e5000f3d0 2024-03-28T16:50:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94287808889984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b0cd0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b1e20 con 0x55c1175a3970 2024-03-28T16:50:29.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94287808889984] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94287808894176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b1e40 con 0x55c1175a3970 2024-03-28T16:50:29.055 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.052+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b28e0 con 0x55c1175ac700 2024-03-28T16:50:29.056 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.056+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94287808894176] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.056+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.056+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94287808897984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b2c90 con 0x55c1175ac700 2024-03-28T16:50:29.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.056+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b3600 con 0x7f9e5000f3d0 2024-03-28T16:50:29.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.056+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.056+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94287808901776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b3e50 con 0x7f9e5000f3d0 2024-03-28T16:50:29.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.056+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b47c0 con 0x55c1175a3970 2024-03-28T16:50:29.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.060+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94287808897984] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.060+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94287808901776] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.060+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.060+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94287808906016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b3db0 con 0x55c1175a3970 2024-03-28T16:50:29.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.060+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b4740 con 0x55c1175a7e90 2024-03-28T16:50:29.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.060+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94287808906016] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.060+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.064+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94287808903712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b5040 con 0x55c1175a7e90 2024-03-28T16:50:29.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.064+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b5840 con 0x55c1175a7e90 2024-03-28T16:50:29.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.064+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.064+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94287808912288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b6460 con 0x55c1175a7e90 2024-03-28T16:50:29.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.064+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b6e50 con 0x55c1175a7e90 2024-03-28T16:50:29.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.064+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94287808903712] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.064+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94287808912288] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94287808913344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b4ba0 con 0x55c1175a7e90 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=0 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=1 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=2 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=3 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=4 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=5 2024-03-28T16:50:29.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=6 2024-03-28T16:50:29.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94287808913344] v31'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 20 add_watcher() i=7 2024-03-28T16:50:29.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:29.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175ba090 con 0x55c1175a3970 2024-03-28T16:50:29.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.070 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.068+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94287808915760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b75d0 con 0x55c1175a3970 2024-03-28T16:50:29.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94287808915760] v31'237 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9d1efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:29.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:29.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:29.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 5 note: GC not initialized 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9d1dfe3640 20 reqs_thread_entry: start 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 20 Filter name: none 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9c8f7fe640 20 reqs_thread_entry: start 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 20 rados->read ofs=0 len=0 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175dbba0 con 0x55c1175a3970 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:29.073 INFO:teuthology.orchestra.run.smithi170.stderr:could not fetch user info: no user info saved 2024-03-28T16:50:29.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9d1efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:29.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.072+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94287808915760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175dbb60 con 0x55c1175a3970 2024-03-28T16:50:29.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94287808915760] v31'238 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94287808889984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175dbb60 con 0x7f9e5000f3d0 2024-03-28T16:50:29.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94287808889984] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e6aec3980 20 remove_watcher() i=0 2024-03-28T16:50:29.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e6aec3980 2 removed watcher, disabling cache 2024-03-28T16:50:29.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94287808894176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175dbb60 con 0x55c1175a3970 2024-03-28T16:50:29.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94287808894176] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e6aec3980 20 remove_watcher() i=1 2024-03-28T16:50:29.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.076+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94287808897984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175dbb60 con 0x55c1175ac700 2024-03-28T16:50:29.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.080+0000 7f9e681dd640 1 -- 172.21.15.170:0/1143626025 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94287808897984] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e50032040 con 0x55c1175ac700 2024-03-28T16:50:29.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.080+0000 7f9e6aec3980 20 remove_watcher() i=2 2024-03-28T16:50:29.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.080+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94287808901776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b14e0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.081 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.080+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94287808901776] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x7f9e5000f3d0 2024-03-28T16:50:29.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.080+0000 7f9e6aec3980 20 remove_watcher() i=3 2024-03-28T16:50:29.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.080+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94287808906016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b14e0 con 0x55c1175a3970 2024-03-28T16:50:29.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.084+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94287808906016] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e600083f0 con 0x55c1175a3970 2024-03-28T16:50:29.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.084+0000 7f9e6aec3980 20 remove_watcher() i=4 2024-03-28T16:50:29.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.084+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94287808903712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b4320 con 0x55c1175a7e90 2024-03-28T16:50:29.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.084+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94287808903712] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.084+0000 7f9e6aec3980 20 remove_watcher() i=5 2024-03-28T16:50:29.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.084+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94287808912288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b14e0 con 0x55c1175a7e90 2024-03-28T16:50:29.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.088+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94287808912288] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.088+0000 7f9e6aec3980 20 remove_watcher() i=6 2024-03-28T16:50:29.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.088+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94287808913344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55c1175b3a20 con 0x55c1175a7e90 2024-03-28T16:50:29.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e689de640 1 -- 172.21.15.170:0/1143626025 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94287808913344] v31'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f9e580fc020 con 0x55c1175a7e90 2024-03-28T16:50:29.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 20 remove_watcher() i=7 2024-03-28T16:50:29.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1175a7e90 msgr2=0x55c1175aa280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c1175a7e90 0x55c1175aa280 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1175a3970 msgr2=0x55c1175a5e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1175a3970 0x55c1175a5e10 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e5000f3d0 msgr2=0x7f9e5000f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f9e5000f3d0 0x7f9e5000f7f0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1175ac700 msgr2=0x55c1175aeb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55c1175ac700 0x55c1175aeb50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d340342b0 msgr2=0x7f9d34036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d340342b0 0x7f9d34036770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e50010fd0 tx=0x7f9e50011000 comp rx=0 tx=0).stop 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 msgr2=0x55c117598740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e691df640 1 -- 172.21.15.170:0/1143626025 reap_dead start 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117598740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9e580e0720 tx=0x7f9e580acde0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 shutdown_connections 2024-03-28T16:50:29.094 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1143626025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1175918b0 0x55c117598740 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 >> 172.21.15.170:0/1143626025 conn(0x55c11758dbf0 msgr2=0x55c11758b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 shutdown_connections 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1143626025 wait complete. 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d400342b0 msgr2=0x7f9d40036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d400342b0 0x7f9d40036770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55c11756b8d0 tx=0x7f9e50006000 comp rx=0 tx=0).stop 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c117584ac0 msgr2=0x55c117584ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c117584ac0 0x55c117584ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9e5801b050 tx=0x7f9e580b3910 comp rx=0 tx=0).stop 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 shutdown_connections 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d400342b0 0x7f9d40036770 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/2629058815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c117584ac0 0x55c117584ee0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 >> 172.21.15.170:0/2629058815 conn(0x55c1175798d0 msgr2=0x55c11757a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 shutdown_connections 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/2629058815 wait complete. 2024-03-28T16:50:29.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c117570440 msgr2=0x55c117572830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55c117570440 0x55c117572830 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1173ea140 msgr2=0x55c11756de80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55c1173ea140 0x55c11756de80 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c117574100 msgr2=0x55c1175764f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x55c117574100 0x55c1175764f0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d50034300 msgr2=0x7f9d500367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9d50034300 0x7f9d500367c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e50001e20 tx=0x7f9e500013a0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 msgr2=0x55c117567200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 --2- 172.21.15.170:0/1085356753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c1173e8c30 0x55c117567200 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9e58003430 tx=0x7f9e58017e30 comp rx=0 tx=0).stop 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e691df640 1 -- 172.21.15.170:0/1085356753 reap_dead start 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 shutdown_connections 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 >> 172.21.15.170:0/1085356753 conn(0x55c11746f810 msgr2=0x55c11755ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 shutdown_connections 2024-03-28T16:50:29.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.092+0000 7f9e6aec3980 1 -- 172.21.15.170:0/1085356753 wait complete. 2024-03-28T16:50:29.104 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-03-28T16:50:29.104 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-03-28T16:50:29.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-03-28T16:50:29.105 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-03-28T16:50:29.181 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:29.181 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:29.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8e741980 1 Processor -- start 2024-03-28T16:50:29.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8e741980 1 -- start start 2024-03-28T16:50:29.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8e741980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55cd938bd6c0 0x55cd938bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8e741980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cd93803520 con 0x55cd938b91b0 2024-03-28T16:50:29.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8e741980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cd937ca0f0 con 0x55cd938bd6c0 2024-03-28T16:50:29.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8c25c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55cd938b91b0 0x55cd938b9590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:49668/0 (socket says 172.21.15.170:49668) 2024-03-28T16:50:29.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8c25c640 1 -- 172.21.15.170:0/2445534335 learned_addr learned my addr 172.21.15.170:0/2445534335 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:29.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.180+0000 7f0d8ba5b640 1 --2- 172.21.15.170:0/2445534335 >> v2:172.21.15.119:3300/0 conn(0x55cd938bd6c0 0x55cd938bfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8ba5b640 1 -- 172.21.15.170:0/2445534335 >> v1:172.21.15.119:6789/0 conn(0x55cd938b91b0 legacy=0x55cd938b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8ba5b640 1 -- 172.21.15.170:0/2445534335 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd938c1000 con 0x55cd938bd6c0 2024-03-28T16:50:29.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8ba5b640 1 --2- 172.21.15.170:0/2445534335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938bd6c0 0x55cd938bfab0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000a480 tx=0x7f0d8001b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5310030535f3b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8b25a640 1 -- 172.21.15.170:0/2445534335 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d800ab540 con 0x55cd938bd6c0 2024-03-28T16:50:29.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8b25a640 1 -- 172.21.15.170:0/2445534335 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d800ab6a0 con 0x55cd938bd6c0 2024-03-28T16:50:29.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8b25a640 1 -- 172.21.15.170:0/2445534335 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d800ab960 con 0x55cd938bd6c0 2024-03-28T16:50:29.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8e741980 1 -- 172.21.15.170:0/2445534335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938bd6c0 msgr2=0x55cd938bfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2445534335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938bd6c0 0x55cd938bfab0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000a480 tx=0x7f0d8001b280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8e741980 1 -- 172.21.15.170:0/2445534335 shutdown_connections 2024-03-28T16:50:29.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2445534335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938bd6c0 0x55cd938bfab0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8e741980 1 -- 172.21.15.170:0/2445534335 >> 172.21.15.170:0/2445534335 conn(0x55cd9382c900 msgr2=0x55cd9382cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8e741980 1 -- 172.21.15.170:0/2445534335 shutdown_connections 2024-03-28T16:50:29.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.184+0000 7f0d8e741980 1 -- 172.21.15.170:0/2445534335 wait complete. 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8e741980 1 Processor -- start 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8e741980 1 -- start start 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8e741980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8e741980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd937c7620 con 0x55cd938b91b0 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8c25c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8c25c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47826/0 (socket says 172.21.15.170:47826) 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8c25c640 1 -- 172.21.15.170:0/1246910842 learned_addr learned my addr 172.21.15.170:0/1246910842 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.200+0000 7f0d8c25c640 1 -- 172.21.15.170:0/1246910842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80005d30 con 0x55cd938b91b0 2024-03-28T16:50:29.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8c25c640 1 --2- 172.21.15.170:0/1246910842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a3100 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c001c00 tx=0x7f0d7c00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880a0ddcf8026ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0c7a60c640 1 -- 172.21.15.170:0/1246910842 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d7c00e690 con 0x55cd938b91b0 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0c7a60c640 1 -- 172.21.15.170:0/1246910842 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00e810 con 0x55cd938b91b0 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0c7a60c640 1 -- 172.21.15.170:0/1246910842 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d7c00eb30 con 0x55cd938b91b0 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- 172.21.15.170:0/1246910842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 msgr2=0x55cd939a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 --2- 172.21.15.170:0/1246910842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a3100 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c001c00 tx=0x7f0d7c00b6d0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- 172.21.15.170:0/1246910842 shutdown_connections 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 --2- 172.21.15.170:0/1246910842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a3100 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- 172.21.15.170:0/1246910842 >> 172.21.15.170:0/1246910842 conn(0x55cd9382c900 msgr2=0x55cd9399ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- 172.21.15.170:0/1246910842 shutdown_connections 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- 172.21.15.170:0/1246910842 wait complete. 2024-03-28T16:50:29.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 Processor -- start 2024-03-28T16:50:29.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- start start 2024-03-28T16:50:29.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd937c2e00 con 0x55cd938b91b0 2024-03-28T16:50:29.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8c25c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8c25c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47828/0 (socket says 172.21.15.170:47828) 2024-03-28T16:50:29.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8c25c640 1 -- 172.21.15.170:0/2432842350 learned_addr learned my addr 172.21.15.170:0/2432842350 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8c25c640 1 -- 172.21.15.170:0/2432842350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd93803520 con 0x55cd938b91b0 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8c25c640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a9f50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00b710 tx=0x7f0d7c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0c78e09640 1 -- 172.21.15.170:0/2432842350 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d7c01f070 con 0x55cd938b91b0 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd939aa490 con 0x55cd938b91b0 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0c78e09640 1 -- 172.21.15.170:0/2432842350 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c01a910 con 0x55cd938b91b0 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd939aa8c0 con 0x55cd938b91b0 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.204+0000 7f0c78e09640 1 -- 172.21.15.170:0/2432842350 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d7c01abf0 con 0x55cd938b91b0 2024-03-28T16:50:29.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 Processor -- start 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- start start 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9375be30 con 0x55cd939b6d90 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0c78e09640 1 -- 172.21.15.170:0/2432842350 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0d7c00a040 con 0x55cd938b91b0 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47830/0 (socket says 172.21.15.170:47830) 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 -- 172.21.15.170:0/451532396 learned_addr learned my addr 172.21.15.170:0/451532396 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0c78e09640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c740342b0 0x7f0c74036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0c78e09640 1 -- 172.21.15.170:0/2432842350 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0d7c049ee0 con 0x55cd938b91b0 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8c25c640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c740342b0 0x7f0c74036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 -- 172.21.15.170:0/451532396 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd939b76f0 con 0x55cd939b6d90 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 --2- 172.21.15.170:0/451532396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939b71b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400a430 tx=0x7f0d8400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f376b5141efc9ac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8c25c640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c740342b0 0x7f0c74036770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c018fd0 tx=0x7f0d7c018f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0c727fc640 1 -- 172.21.15.170:0/451532396 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d8400f6e0 con 0x55cd939b6d90 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0c727fc640 1 -- 172.21.15.170:0/451532396 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8400f860 con 0x55cd939b6d90 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0c727fc640 1 -- 172.21.15.170:0/451532396 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d8400fb40 con 0x55cd939b6d90 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- 172.21.15.170:0/451532396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 msgr2=0x55cd939b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 --2- 172.21.15.170:0/451532396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939b71b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400a430 tx=0x7f0d8400a8f0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- 172.21.15.170:0/451532396 shutdown_connections 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 --2- 172.21.15.170:0/451532396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939b71b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- 172.21.15.170:0/451532396 >> 172.21.15.170:0/451532396 conn(0x55cd938bd6c0 msgr2=0x55cd939b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- 172.21.15.170:0/451532396 shutdown_connections 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- 172.21.15.170:0/451532396 wait complete. 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 Processor -- start 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- start start 2024-03-28T16:50:29.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8e741980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9374f790 con 0x55cd939b6d90 2024-03-28T16:50:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939bdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47832/0 (socket says 172.21.15.170:47832) 2024-03-28T16:50:29.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 -- 172.21.15.170:0/2110119210 learned_addr learned my addr 172.21.15.170:0/2110119210 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.208+0000 7f0d8ca5d640 1 -- 172.21.15.170:0/2110119210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8400a0e0 con 0x55cd939b6d90 2024-03-28T16:50:29.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8ca5d640 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939bdd00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x55cd939b7b70 tx=0x7f0d84003380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c70ff9640 1 -- 172.21.15.170:0/2110119210 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d84010a20 con 0x55cd939b6d90 2024-03-28T16:50:29.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd939be240 con 0x55cd939b6d90 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd939be6d0 con 0x55cd939b6d90 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c70ff9640 1 -- 172.21.15.170:0/2110119210 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84010b80 con 0x55cd939b6d90 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c70ff9640 1 -- 172.21.15.170:0/2110119210 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d84010ea0 con 0x55cd939b6d90 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c70ff9640 1 -- 172.21.15.170:0/2110119210 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0d8401a480 con 0x55cd939b6d90 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 Processor -- start 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c70ff9640 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c640342b0 0x7f0c64036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 -- start start 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c70ff9640 1 -- 172.21.15.170:0/2110119210 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0d84049b70 con 0x55cd939b6d90 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8ca5d640 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c640342b0 0x7f0c64036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd9382dd20 0x55cd9382e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9382e680 con 0x55cd9382dd20 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8ca5d640 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c640342b0 0x7f0c64036770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d84023fd0 tx=0x7f0d84042040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8c25c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd9382dd20 0x55cd9382e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8c25c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd9382dd20 0x55cd9382e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47838/0 (socket says 172.21.15.170:47838) 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8c25c640 1 -- 172.21.15.170:0/2069110092 learned_addr learned my addr 172.21.15.170:0/2069110092 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8c25c640 1 -- 172.21.15.170:0/2069110092 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9382e7c0 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8c25c640 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd9382dd20 0x55cd9382e140 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0482c0 tx=0x7f0d7c00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c50fe1640 1 -- 172.21.15.170:0/2069110092 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d7c024e40 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd9382f0d0 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c50fe1640 1 -- 172.21.15.170:0/2069110092 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c006b30 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0c50fe1640 1 -- 172.21.15.170:0/2069110092 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d7c023e20 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.212+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd9382f560 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0c50fe1640 1 -- 172.21.15.170:0/2069110092 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f0d7c045070 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0c50fe1640 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0bcc0342b0 0x7f0bcc036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0c50fe1640 1 -- 172.21.15.170:0/2069110092 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0d7c042050 con 0x55cd9382dd20 2024-03-28T16:50:29.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8ba5b640 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0bcc0342b0 0x7f0bcc036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0c227fc640 20 reqs_thread_entry: start 2024-03-28T16:50:29.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55cd93830820 0x55cd93830ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cd938311e0 con 0x55cd93830820 2024-03-28T16:50:29.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8ba5b640 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0bcc0342b0 0x7f0bcc036770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0d800ad000 tx=0x7f0d800a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8ca5d640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55cd93830820 0x55cd93830ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8ca5d640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55cd93830820 0x55cd93830ca0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8ca5d640 1 -- 172.21.15.170:0/2432842350 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f0d84035040 con 0x55cd93830820 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55cd939dc6b0 0x55cd939deb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:2 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55cd939df2b0 con 0x55cd939dc6b0 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8c25c640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55cd939dc6b0 0x55cd939deb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.216+0000 7f0d8c25c640 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55cd939dc6b0 0x55cd939deb20 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8c25c640 1 -- 172.21.15.170:0/2432842350 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(2 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f0d7c04e070 con 0x55cd939dc6b0 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c640342b0 msgr2=0x7f0c64036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c640342b0 0x7f0c64036770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0d84023fd0 tx=0x7f0d84042040 comp rx=0 tx=0).stop 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 msgr2=0x55cd939bdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939bdd00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x55cd939b7b70 tx=0x7f0d84003380 comp rx=0 tx=0).stop 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 shutdown_connections 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c640342b0 0x7f0c64036770 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2110119210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd939b6d90 0x55cd939bdd00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 >> 172.21.15.170:0/2110119210 conn(0x55cd938bd6c0 msgr2=0x55cd939b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 shutdown_connections 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2110119210 wait complete. 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0bcc0342b0 msgr2=0x7f0bcc036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0bcc0342b0 0x7f0bcc036770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0d800ad000 tx=0x7f0d800a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd9382dd20 msgr2=0x55cd9382e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd9382dd20 0x55cd9382e140 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0482c0 tx=0x7f0d7c00af70 comp rx=0 tx=0).stop 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 shutdown_connections 2024-03-28T16:50:29.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0bcc0342b0 0x7f0bcc036770 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2069110092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd9382dd20 0x55cd9382e140 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 >> 172.21.15.170:0/2069110092 conn(0x55cd939cc7a0 msgr2=0x55cd939cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 shutdown_connections 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2069110092 wait complete. 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55cd939dc6b0 msgr2=0x55cd939deb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55cd939dc6b0 0x55cd939deb20 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55cd93830820 msgr2=0x55cd93830ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55cd93830820 0x55cd93830ca0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c740342b0 msgr2=0x7f0c74036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c740342b0 0x7f0c74036770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c018fd0 tx=0x7f0d7c018f60 comp rx=0 tx=0).stop 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 msgr2=0x55cd939a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a9f50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00b710 tx=0x7f0d7c000bd0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 shutdown_connections 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55cd939dc6b0 0x55cd939deb20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55cd93830820 0x55cd93830ca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f0c740342b0 0x7f0c74036770 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 --2- 172.21.15.170:0/2432842350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd938b91b0 0x55cd939a9f50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 >> 172.21.15.170:0/2432842350 conn(0x55cd9382c900 msgr2=0x55cd9399d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 shutdown_connections 2024-03-28T16:50:29.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.220+0000 7f0d8e741980 1 -- 172.21.15.170:0/2432842350 wait complete. 2024-03-28T16:50:29.231 INFO:teuthology.orchestra.run.smithi170.stdout:{"id":"b88c2392-6ef8-4660-b080-19480b748fc0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-28T16:50:29.231 DEBUG:tasks.util.rgw: json result: {'id': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-28T16:50:29.231 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-03-28T16:50:29.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-03-28T16:50:29.231 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-03-28T16:50:29.307 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:29.307 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:29.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248f820980 1 Processor -- start 2024-03-28T16:50:29.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248f820980 1 -- start start 2024-03-28T16:50:29.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248f820980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55daccc8c120 0x55daccc8c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248f820980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55daccbd6510 con 0x55daccc8ca40 2024-03-28T16:50:29.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248f820980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55daccb9d0f0 con 0x55daccc8c120 2024-03-28T16:50:29.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248d33b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55daccc8c120 0x55daccc8c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248cb3a640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55daccc8ca40 0x55daccc8d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:49680/0 (socket says 172.21.15.170:49680) 2024-03-28T16:50:29.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248d33b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55daccc8c120 0x55daccc8c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:47848/0 (socket says 172.21.15.170:47848) 2024-03-28T16:50:29.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248cb3a640 1 -- 172.21.15.170:0/1400901009 learned_addr learned my addr 172.21.15.170:0/1400901009 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:29.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248d33b640 1 -- 172.21.15.170:0/1400901009 >> v1:172.21.15.119:6789/0 conn(0x55daccc8ca40 legacy=0x55daccc8d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.308+0000 7f248d33b640 1 -- 172.21.15.170:0/1400901009 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daccc8dc20 con 0x55daccc8c120 2024-03-28T16:50:29.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248d33b640 1 --2- 172.21.15.170:0/1400901009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccc8c500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f248000a480 tx=0x7f248001b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b6a43f8ad8ae1f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f247ffff640 1 -- 172.21.15.170:0/1400901009 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800ab540 con 0x55daccc8c120 2024-03-28T16:50:29.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f247ffff640 1 -- 172.21.15.170:0/1400901009 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24800ab6a0 con 0x55daccc8c120 2024-03-28T16:50:29.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f247ffff640 1 -- 172.21.15.170:0/1400901009 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800ab960 con 0x55daccc8c120 2024-03-28T16:50:29.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248f820980 1 -- 172.21.15.170:0/1400901009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 msgr2=0x55daccc8c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248f820980 1 --2- 172.21.15.170:0/1400901009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccc8c500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f248000a480 tx=0x7f248001b280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248f820980 1 -- 172.21.15.170:0/1400901009 shutdown_connections 2024-03-28T16:50:29.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248f820980 1 --2- 172.21.15.170:0/1400901009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccc8c500 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248f820980 1 -- 172.21.15.170:0/1400901009 >> 172.21.15.170:0/1400901009 conn(0x55daccbff840 msgr2=0x55daccbffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248f820980 1 -- 172.21.15.170:0/1400901009 shutdown_connections 2024-03-28T16:50:29.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.312+0000 7f248f820980 1 -- 172.21.15.170:0/1400901009 wait complete. 2024-03-28T16:50:29.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 Processor -- start 2024-03-28T16:50:29.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 -- start start 2024-03-28T16:50:29.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daccb9a620 con 0x55daccc8c120 2024-03-28T16:50:29.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248d33b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248d33b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37674/0 (socket says 172.21.15.170:37674) 2024-03-28T16:50:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248d33b640 1 -- 172.21.15.170:0/1530595190 learned_addr learned my addr 172.21.15.170:0/1530595190 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248d33b640 1 -- 172.21.15.170:0/1530595190 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480005d30 con 0x55daccc8c120 2024-03-28T16:50:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248d33b640 1 --2- 172.21.15.170:0/1530595190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd765b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f24800ae040 tx=0x7f2480003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac19d648f84b94e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f237b606640 1 -- 172.21.15.170:0/1530595190 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800ab810 con 0x55daccc8c120 2024-03-28T16:50:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f237b606640 1 -- 172.21.15.170:0/1530595190 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24800ab970 con 0x55daccc8c120 2024-03-28T16:50:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f237b606640 1 -- 172.21.15.170:0/1530595190 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800abc10 con 0x55daccc8c120 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 -- 172.21.15.170:0/1530595190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 msgr2=0x55daccd765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 --2- 172.21.15.170:0/1530595190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd765b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f24800ae040 tx=0x7f2480003030 comp rx=0 tx=0).stop 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 -- 172.21.15.170:0/1530595190 shutdown_connections 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 --2- 172.21.15.170:0/1530595190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd765b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 -- 172.21.15.170:0/1530595190 >> 172.21.15.170:0/1530595190 conn(0x55daccbff840 msgr2=0x55daccd72140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 -- 172.21.15.170:0/1530595190 shutdown_connections 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.328+0000 7f248f820980 1 -- 172.21.15.170:0/1530595190 wait complete. 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 Processor -- start 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 -- start start 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daccb95e00 con 0x55daccc8c120 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248d33b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248d33b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd7d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37676/0 (socket says 172.21.15.170:37676) 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 learned_addr learned my addr 172.21.15.170:0/1862402868 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daccc8dc20 con 0x55daccc8c120 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248d33b640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd7d2e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f248001bb20 tx=0x7f2480018020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f2379e03640 1 -- 172.21.15.170:0/1862402868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800c2070 con 0x55daccc8c120 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55daccd7d820 con 0x55daccc8c120 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f2379e03640 1 -- 172.21.15.170:0/1862402868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24800b59d0 con 0x55daccc8c120 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f2379e03640 1 -- 172.21.15.170:0/1862402868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800b5c70 con 0x55daccc8c120 2024-03-28T16:50:29.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daccd7dcb0 con 0x55daccc8c120 2024-03-28T16:50:29.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 Processor -- start 2024-03-28T16:50:29.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 -- start start 2024-03-28T16:50:29.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd8a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248f820980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daccb2ee30 con 0x55daccd8a0c0 2024-03-28T16:50:29.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f2379e03640 1 -- 172.21.15.170:0/1862402868 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f24800b2070 con 0x55daccc8c120 2024-03-28T16:50:29.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248db3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd8a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248db3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd8a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37682/0 (socket says 172.21.15.170:37682) 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248db3c640 1 -- 172.21.15.170:0/3336372315 learned_addr learned my addr 172.21.15.170:0/3336372315 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f2379e03640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23740342b0 0x7f2374036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f2379e03640 1 -- 172.21.15.170:0/1862402868 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f24800e3480 con 0x55daccc8c120 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248d33b640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23740342b0 0x7f2374036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248db3c640 1 -- 172.21.15.170:0/3336372315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daccd8aa20 con 0x55daccd8a0c0 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248d33b640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23740342b0 0x7f2374036770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2480005690 tx=0x7f24800e3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.332+0000 7f248db3c640 1 --2- 172.21.15.170:0/3336372315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd8a4e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f248800a430 tx=0x7f248800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4215c31c3ecaa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f23737fe640 1 -- 172.21.15.170:0/3336372315 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f248800f6e0 con 0x55daccd8a0c0 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f23737fe640 1 -- 172.21.15.170:0/3336372315 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248800f860 con 0x55daccd8a0c0 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f23737fe640 1 -- 172.21.15.170:0/3336372315 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f248800fb40 con 0x55daccd8a0c0 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- 172.21.15.170:0/3336372315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 msgr2=0x55daccd8a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 --2- 172.21.15.170:0/3336372315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd8a4e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f248800a430 tx=0x7f248800a8f0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- 172.21.15.170:0/3336372315 shutdown_connections 2024-03-28T16:50:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 --2- 172.21.15.170:0/3336372315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd8a4e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- 172.21.15.170:0/3336372315 >> 172.21.15.170:0/3336372315 conn(0x55daccc8ca40 msgr2=0x55daccd85530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- 172.21.15.170:0/3336372315 shutdown_connections 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- 172.21.15.170:0/3336372315 wait complete. 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 Processor -- start 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- start start 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd91010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daccb22790 con 0x55daccd8a0c0 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248db3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd91010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248db3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd91010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37692/0 (socket says 172.21.15.170:37692) 2024-03-28T16:50:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248db3c640 1 -- 172.21.15.170:0/3279107140 learned_addr learned my addr 172.21.15.170:0/3279107140 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248db3c640 1 -- 172.21.15.170:0/3279107140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248800a0e0 con 0x55daccd8a0c0 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248db3c640 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd91010 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55daccd8aea0 tx=0x7f2488003380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f2371ffb640 1 -- 172.21.15.170:0/3279107140 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2488010a20 con 0x55daccd8a0c0 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55daccd91550 con 0x55daccd8a0c0 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daccd919e0 con 0x55daccd8a0c0 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f2371ffb640 1 -- 172.21.15.170:0/3279107140 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2488010b80 con 0x55daccd8a0c0 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f2371ffb640 1 -- 172.21.15.170:0/3279107140 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2488010ea0 con 0x55daccd8a0c0 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:29.341 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f248f820980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.336+0000 7f2371ffb640 1 -- 172.21.15.170:0/3279107140 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f248801a480 con 0x55daccd8a0c0 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 Processor -- start 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 -- start start 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2371ffb640 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2364034300 0x7f23640367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2371ffb640 1 -- 172.21.15.170:0/3279107140 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2488049b60 con 0x55daccd8a0c0 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248db3c640 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2364034300 0x7f23640367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccdaade0 0x55daccdab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daccdab740 con 0x55daccdaade0 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248db3c640 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2364034300 0x7f23640367c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f24880198d0 tx=0x7f2488019760 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248d33b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccdaade0 0x55daccdab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248d33b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccdaade0 0x55daccdab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37704/0 (socket says 172.21.15.170:37704) 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248d33b640 1 -- 172.21.15.170:0/2023034349 learned_addr learned my addr 172.21.15.170:0/2023034349 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248d33b640 1 -- 172.21.15.170:0/2023034349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daccdab880 con 0x55daccdaade0 2024-03-28T16:50:29.342 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248d33b640 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccdaade0 0x55daccdab200 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f24800e18d0 tx=0x7f24800bdf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2351fe3640 1 -- 172.21.15.170:0/2023034349 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800e32c0 con 0x55daccdaade0 2024-03-28T16:50:29.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55daccdac1e0 con 0x55daccdaade0 2024-03-28T16:50:29.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2351fe3640 1 -- 172.21.15.170:0/2023034349 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24800aa830 con 0x55daccdaade0 2024-03-28T16:50:29.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2351fe3640 1 -- 172.21.15.170:0/2023034349 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24800e1e40 con 0x55daccdaade0 2024-03-28T16:50:29.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daccdac670 con 0x55daccdaade0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2351fe3640 1 -- 172.21.15.170:0/2023034349 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f24800cfc80 con 0x55daccdaade0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2351fe3640 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f22d00342b0 0x7f22d0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f2351fe3640 1 -- 172.21.15.170:0/2023034349 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f24800eec30 con 0x55daccdaade0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248cb3a640 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f22d00342b0 0x7f22d0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f22cb7fe640 20 reqs_thread_entry: start 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55daccdafdc0 0x55daccdb21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb26f0 con 0x55daccdafdc0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.340+0000 7f248cb3a640 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f22d00342b0 0x7f22d0036770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55daccd86170 tx=0x7f2484001280 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248db3c640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55daccdafdc0 0x55daccdb21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248db3c640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55daccdafdc0 0x55daccdb21b0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2488016020 con 0x55daccdafdc0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb2a50 con 0x55daccdafdc0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f2488016020 con 0x55daccdafdc0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55daccdb44d0 0x55daccdb6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb70f0 con 0x55daccdb44d0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248d33b640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55daccdb44d0 0x55daccdb6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248d33b640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55daccdb44d0 0x55daccdb6960 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f24800b7e20 con 0x55daccdb44d0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdbd060 con 0x55daccdafdc0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f2488016020 con 0x55daccdafdc0 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:NOTICE: set zone's realm_id= 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55daccdc1090 0x55daccdbc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248cb3a640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55daccdc1090 0x55daccdbc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.344+0000 7f248cb3a640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55daccdc1090 0x55daccdbc8f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(5 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248cb3a640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f248400d8f0 0x7f248400c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x7f248400d8f0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f248400d8f0 0x7f248400c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f248400d8f0 0x7f248400c370 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=112b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f2488016020 con 0x7f248400d8f0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800b7e20 con 0x55daccdb44d0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2488016020 con 0x55daccdafdc0 2024-03-28T16:50:29.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2488016020 con 0x55daccdafdc0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x7f248400d8f0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2488016020 con 0x7f248400d8f0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.348+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2488016020 con 0x55daccdafdc0 2024-03-28T16:50:29.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f2488016020 con 0x55daccdafdc0 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x7f248400d8f0 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(17 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f2488016020 con 0x7f248400d8f0 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248805b040 con 0x55daccdafdc0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=74b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f248805b040 con 0x55daccdafdc0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.352+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(25 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x7f248400d8f0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248805b040 con 0x7f248400d8f0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248805b040 con 0x55daccdafdc0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x7f248400d8f0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248805b040 con 0x7f248400d8f0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdc1090 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248400e620 con 0x55daccdc1090 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.356+0000 7f248cb3a640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdb44d0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=107b] v29'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x55daccdafdc0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(35 [pgnls start_epoch 31 out=115b] v22'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f248805b040 con 0x55daccdafdc0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdb74e0 con 0x7f248400d8f0 2024-03-28T16:50:29.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(36 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f248805b040 con 0x7f248400d8f0 2024-03-28T16:50:29.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdbe660 con 0x55daccdafdc0 2024-03-28T16:50:29.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f248805b040 con 0x55daccdafdc0 2024-03-28T16:50:29.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:38 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdba160 con 0x55daccdb44d0 2024-03-28T16:50:29.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(38 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.360+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:39 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [create,call version.set in=58b,writefull 0~1036 in=1036b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdba570 con 0x55daccdb44d0 2024-03-28T16:50:29.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248d33b640 1 -- 172.21.15.170:0/1862402868 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(39 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [create,call,writefull 0~1036] v31'3 uv3 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x7f24800f7040 con 0x55daccdb44d0 2024-03-28T16:50:29.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdbc420 con 0x55daccdafdc0 2024-03-28T16:50:29.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v31'3 uv3 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x7f248805b040 con 0x55daccdafdc0 2024-03-28T16:50:29.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55daccdbeee0 con 0x55daccdafdc0 2024-03-28T16:50:29.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v31'2 uv1 ondisk = -17 ((17) File exists)) v8 ==== 199+0+0 (crc 0 0 0) 0x7f248805b040 con 0x55daccdafdc0 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2364034300 msgr2=0x7f23640367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2364034300 0x7f23640367c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f24880198d0 tx=0x7f2488019760 comp rx=0 tx=0).stop 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 msgr2=0x55daccd91010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd91010 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55daccd8aea0 tx=0x7f2488003380 comp rx=0 tx=0).stop 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 shutdown_connections 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2364034300 0x7f23640367c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/3279107140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccd8a0c0 0x55daccd91010 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 >> 172.21.15.170:0/3279107140 conn(0x55daccc8ca40 msgr2=0x55daccd84330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 shutdown_connections 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/3279107140 wait complete. 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f22d00342b0 msgr2=0x7f22d0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f22d00342b0 0x7f22d0036770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55daccd86170 tx=0x7f2484001280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccdaade0 msgr2=0x55daccdab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccdaade0 0x55daccdab200 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f24800e18d0 tx=0x7f24800bdf30 comp rx=0 tx=0).stop 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 shutdown_connections 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f22d00342b0 0x7f22d0036770 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 --2- 172.21.15.170:0/2023034349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccdaade0 0x55daccdab200 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.364+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 >> 172.21.15.170:0/2023034349 conn(0x55daccd9fab0 msgr2=0x55daccda04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 shutdown_connections 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/2023034349 wait complete. 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55daccdb44d0 msgr2=0x55daccdb6960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55daccdb44d0 0x55daccdb6960 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55daccdafdc0 msgr2=0x55daccdb21b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55daccdafdc0 0x55daccdb21b0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f248400d8f0 msgr2=0x7f248400c370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f248400d8f0 0x7f248400c370 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55daccdc1090 msgr2=0x55daccdbc8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55daccdc1090 0x55daccdbc8f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23740342b0 msgr2=0x7f2374036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f23740342b0 0x7f2374036770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2480005690 tx=0x7f24800e3040 comp rx=0 tx=0).stop 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 msgr2=0x55daccd7d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248db3c640 1 -- 172.21.15.170:0/1862402868 reap_dead start 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd7d2e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f248001bb20 tx=0x7f2480018020 comp rx=0 tx=0).stop 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 shutdown_connections 2024-03-28T16:50:29.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 --2- 172.21.15.170:0/1862402868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55daccc8c120 0x55daccd7d2e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 >> 172.21.15.170:0/1862402868 conn(0x55daccbff840 msgr2=0x55daccd70c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 shutdown_connections 2024-03-28T16:50:29.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.368+0000 7f248f820980 1 -- 172.21.15.170:0/1862402868 wait complete. 2024-03-28T16:50:29.376 INFO:teuthology.orchestra.run.smithi170.stdout:{"id":"b88c2392-6ef8-4660-b080-19480b748fc0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-28T16:50:29.377 DEBUG:tasks.util.rgw: json result: {'id': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-28T16:50:29.377 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-03-28T16:50:29.377 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-03-28T16:50:29.377 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-03-28T16:50:29.455 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:29.455 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:29.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f3623601980 1 Processor -- start 2024-03-28T16:50:29.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f3623601980 1 -- start start 2024-03-28T16:50:29.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f3623601980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x564775dd9700 0x564775ddbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f3623601980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564775d1f510 con 0x564775d47060 2024-03-28T16:50:29.456 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f3623601980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x564775ce60f0 con 0x564775dd9700 2024-03-28T16:50:29.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f362091b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x564775dd9700 0x564775ddbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f362111c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x564775d47060 0x564775dd7fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:33962/0 (socket says 172.21.15.170:33962) 2024-03-28T16:50:29.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f362111c640 1 -- 172.21.15.170:0/2702782725 learned_addr learned my addr 172.21.15.170:0/2702782725 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:29.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f362091b640 1 -- 172.21.15.170:0/2702782725 >> v1:172.21.15.119:6789/0 conn(0x564775d47060 legacy=0x564775dd7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.457 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.456+0000 7f362091b640 1 -- 172.21.15.170:0/2702782725 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564775ddd040 con 0x564775dd9700 2024-03-28T16:50:29.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f362091b640 1 --2- 172.21.15.170:0/2702782725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775dd9700 0x564775ddbaf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f361800a480 tx=0x7f361801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa071503e60ec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3613fff640 1 -- 172.21.15.170:0/2702782725 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36180ab540 con 0x564775dd9700 2024-03-28T16:50:29.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3613fff640 1 -- 172.21.15.170:0/2702782725 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36180ab6a0 con 0x564775dd9700 2024-03-28T16:50:29.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3613fff640 1 -- 172.21.15.170:0/2702782725 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36180ab960 con 0x564775dd9700 2024-03-28T16:50:29.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3623601980 1 -- 172.21.15.170:0/2702782725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775dd9700 msgr2=0x564775ddbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3623601980 1 --2- 172.21.15.170:0/2702782725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775dd9700 0x564775ddbaf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f361800a480 tx=0x7f361801b280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3623601980 1 -- 172.21.15.170:0/2702782725 shutdown_connections 2024-03-28T16:50:29.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3623601980 1 --2- 172.21.15.170:0/2702782725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775dd9700 0x564775ddbaf0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3623601980 1 -- 172.21.15.170:0/2702782725 >> 172.21.15.170:0/2702782725 conn(0x564775dcd6c0 msgr2=0x564775dcfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3623601980 1 -- 172.21.15.170:0/2702782725 shutdown_connections 2024-03-28T16:50:29.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.460+0000 7f3623601980 1 -- 172.21.15.170:0/2702782725 wait complete. 2024-03-28T16:50:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.472+0000 7f3623601980 1 Processor -- start 2024-03-28T16:50:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.472+0000 7f3623601980 1 -- start start 2024-03-28T16:50:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.472+0000 7f3623601980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ebf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.472+0000 7f3623601980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564775ce3620 con 0x564775d47060 2024-03-28T16:50:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.472+0000 7f362111c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ebf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.472+0000 7f362111c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ebf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37726/0 (socket says 172.21.15.170:37726) 2024-03-28T16:50:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.472+0000 7f362111c640 1 -- 172.21.15.170:0/1030746884 learned_addr learned my addr 172.21.15.170:0/1030746884 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f362111c640 1 -- 172.21.15.170:0/1030746884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618005d30 con 0x564775d47060 2024-03-28T16:50:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f362111c640 1 --2- 172.21.15.170:0/1030746884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ebf160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3614001c00 tx=0x7f361400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3245ecfbd07c2bb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f350f606640 1 -- 172.21.15.170:0/1030746884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361400e690 con 0x564775d47060 2024-03-28T16:50:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f350f606640 1 -- 172.21.15.170:0/1030746884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361400e810 con 0x564775d47060 2024-03-28T16:50:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f350f606640 1 -- 172.21.15.170:0/1030746884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361400eb30 con 0x564775d47060 2024-03-28T16:50:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- 172.21.15.170:0/1030746884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 msgr2=0x564775ebf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 --2- 172.21.15.170:0/1030746884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ebf160 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3614001c00 tx=0x7f361400b6d0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- 172.21.15.170:0/1030746884 shutdown_connections 2024-03-28T16:50:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 --2- 172.21.15.170:0/1030746884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ebf160 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- 172.21.15.170:0/1030746884 >> 172.21.15.170:0/1030746884 conn(0x564775dcd6c0 msgr2=0x564775ebac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- 172.21.15.170:0/1030746884 shutdown_connections 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- 172.21.15.170:0/1030746884 wait complete. 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 Processor -- start 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- start start 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ec6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564775cdee00 con 0x564775d47060 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f362111c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ec6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f362111c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ec6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37730/0 (socket says 172.21.15.170:37730) 2024-03-28T16:50:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f362111c640 1 -- 172.21.15.170:0/702472860 learned_addr learned my addr 172.21.15.170:0/702472860 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f362111c640 1 -- 172.21.15.170:0/702472860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564775d1f510 con 0x564775d47060 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f362111c640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ec6200 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f361400b710 tx=0x7f3614000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f350de03640 1 -- 172.21.15.170:0/702472860 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361401f070 con 0x564775d47060 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564775ec6740 con 0x564775d47060 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f350de03640 1 -- 172.21.15.170:0/702472860 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361401a910 con 0x564775d47060 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.476+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564775ec6b70 con 0x564775d47060 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f350de03640 1 -- 172.21.15.170:0/702472860 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361401abf0 con 0x564775d47060 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 Processor -- start 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- start start 2024-03-28T16:50:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564775c77e30 con 0x564775ed2f20 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37744/0 (socket says 172.21.15.170:37744) 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 -- 172.21.15.170:0/4060241507 learned_addr learned my addr 172.21.15.170:0/4060241507 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f350de03640 1 -- 172.21.15.170:0/702472860 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f361400a040 con 0x564775d47060 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f350de03640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f350de03640 1 -- 172.21.15.170:0/702472860 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f3614049e20 con 0x564775d47060 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 -- 172.21.15.170:0/4060241507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564775ed3880 con 0x564775ed2f20 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362111c640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 --2- 172.21.15.170:0/4060241507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed3340 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f361c00a430 tx=0x7f361c00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75aacf8f6b982eec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362111c640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3614048380 tx=0x7f3614024c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f35077fe640 1 -- 172.21.15.170:0/4060241507 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361c00f6e0 con 0x564775ed2f20 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f35077fe640 1 -- 172.21.15.170:0/4060241507 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c00f860 con 0x564775ed2f20 2024-03-28T16:50:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f35077fe640 1 -- 172.21.15.170:0/4060241507 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361c00fb40 con 0x564775ed2f20 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- 172.21.15.170:0/4060241507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 msgr2=0x564775ed3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 --2- 172.21.15.170:0/4060241507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed3340 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f361c00a430 tx=0x7f361c00a8f0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- 172.21.15.170:0/4060241507 shutdown_connections 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 --2- 172.21.15.170:0/4060241507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed3340 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- 172.21.15.170:0/4060241507 >> 172.21.15.170:0/4060241507 conn(0x564775dd9700 msgr2=0x564775ece390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- 172.21.15.170:0/4060241507 shutdown_connections 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- 172.21.15.170:0/4060241507 wait complete. 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 Processor -- start 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- start start 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f3623601980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564775c6b790 con 0x564775ed2f20 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37746/0 (socket says 172.21.15.170:37746) 2024-03-28T16:50:29.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.480+0000 7f362191d640 1 -- 172.21.15.170:0/4287874389 learned_addr learned my addr 172.21.15.170:0/4287874389 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362191d640 1 -- 172.21.15.170:0/4287874389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c00a0e0 con 0x564775ed2f20 2024-03-28T16:50:29.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362191d640 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed9eb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x564775ed3d00 tx=0x7f361c003380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3505ffb640 1 -- 172.21.15.170:0/4287874389 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361c010a20 con 0x564775ed2f20 2024-03-28T16:50:29.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564775eda3f0 con 0x564775ed2f20 2024-03-28T16:50:29.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564775eda880 con 0x564775ed2f20 2024-03-28T16:50:29.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3505ffb640 1 -- 172.21.15.170:0/4287874389 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c010b80 con 0x564775ed2f20 2024-03-28T16:50:29.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3505ffb640 1 -- 172.21.15.170:0/4287874389 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f361c010ea0 con 0x564775ed2f20 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3505ffb640 1 -- 172.21.15.170:0/4287874389 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f361c01a480 con 0x564775ed2f20 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3505ffb640 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f8034300 0x7f34f80367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3505ffb640 1 -- 172.21.15.170:0/4287874389 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f361c049b60 con 0x564775ed2f20 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 1 Processor -- start 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362191d640 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f8034300 0x7f34f80367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 1 -- start start 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ef3c30 0x564775ef4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564775ef4590 con 0x564775ef3c30 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362191d640 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f8034300 0x7f34f80367c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f361c025000 tx=0x7f361c00ce90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362111c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ef3c30 0x564775ef4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362111c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ef3c30 0x564775ef4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37748/0 (socket says 172.21.15.170:37748) 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362111c640 1 -- 172.21.15.170:0/354146623 learned_addr learned my addr 172.21.15.170:0/354146623 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362111c640 1 -- 172.21.15.170:0/354146623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564775ef46d0 con 0x564775ef3c30 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f362111c640 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ef3c30 0x564775ef4050 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3614020fd0 tx=0x7f361400af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f34e5fe3640 1 -- 172.21.15.170:0/354146623 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3614024e40 con 0x564775ef3c30 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.484+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564775ef4fe0 con 0x564775ef3c30 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f34e5fe3640 1 -- 172.21.15.170:0/354146623 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361404c020 con 0x564775ef3c30 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f34e5fe3640 1 -- 172.21.15.170:0/354146623 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3614023e20 con 0x564775ef3c30 2024-03-28T16:50:29.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564775ef5410 con 0x564775ef3c30 2024-03-28T16:50:29.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f34e5fe3640 1 -- 172.21.15.170:0/354146623 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3614045070 con 0x564775ef3c30 2024-03-28T16:50:29.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f34e5fe3640 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34640342b0 0x7f3464036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f34e5fe3640 1 -- 172.21.15.170:0/354146623 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f361404d030 con 0x564775ef3c30 2024-03-28T16:50:29.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362091b640 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34640342b0 0x7f3464036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f345f7fe640 20 reqs_thread_entry: start 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x564775eca060 0x564775efabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564775efb1a0 con 0x564775eca060 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362091b640 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34640342b0 0x7f3464036770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36180ad000 tx=0x7f36180a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362191d640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x564775eca060 0x564775efabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362191d640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x564775eca060 0x564775efabd0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362191d640 1 -- 172.21.15.170:0/702472860 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f361c016020 con 0x564775eca060 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564775efb600 con 0x564775eca060 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362191d640 1 -- 172.21.15.170:0/702472860 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f361c016020 con 0x564775eca060 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x564775efd1c0 0x564775eff680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x564775effe10 con 0x564775efd1c0 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362111c640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x564775efd1c0 0x564775eff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.488+0000 7f362111c640 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x564775efd1c0 0x564775eff680 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f362111c640 1 -- 172.21.15.170:0/702472860 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x7f361400afe0 con 0x564775efd1c0 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f8034300 msgr2=0x7f34f80367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f8034300 0x7f34f80367c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f361c025000 tx=0x7f361c00ce90 comp rx=0 tx=0).stop 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 msgr2=0x564775ed9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed9eb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x564775ed3d00 tx=0x7f361c003380 comp rx=0 tx=0).stop 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 shutdown_connections 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34f8034300 0x7f34f80367c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/4287874389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ed2f20 0x564775ed9eb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 >> 172.21.15.170:0/4287874389 conn(0x564775dd9700 msgr2=0x564775ecd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 shutdown_connections 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/4287874389 wait complete. 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34640342b0 msgr2=0x7f3464036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34640342b0 0x7f3464036770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f36180ad000 tx=0x7f36180a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:29.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ef3c30 msgr2=0x564775ef4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ef3c30 0x564775ef4050 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3614020fd0 tx=0x7f361400af70 comp rx=0 tx=0).stop 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 shutdown_connections 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f34640342b0 0x7f3464036770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/354146623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775ef3c30 0x564775ef4050 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 >> 172.21.15.170:0/354146623 conn(0x564775ee8950 msgr2=0x564775ee9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 shutdown_connections 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/354146623 wait complete. 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x564775efd1c0 msgr2=0x564775eff680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x564775efd1c0 0x564775eff680 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x564775eca060 msgr2=0x564775efabd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x564775eca060 0x564775efabd0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 msgr2=0x7f3508036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3614048380 tx=0x7f3614024c60 comp rx=0 tx=0).stop 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 msgr2=0x564775ec6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ec6200 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f361400b710 tx=0x7f3614000bd0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 shutdown_connections 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x564775efd1c0 0x564775eff680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x564775eca060 0x564775efabd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35080342b0 0x7f3508036770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 --2- 172.21.15.170:0/702472860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564775d47060 0x564775ec6200 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 >> 172.21.15.170:0/702472860 conn(0x564775dcd6c0 msgr2=0x564775eb9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 shutdown_connections 2024-03-28T16:50:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.492+0000 7f3623601980 1 -- 172.21.15.170:0/702472860 wait complete. 2024-03-28T16:50:29.504 INFO:teuthology.orchestra.run.smithi170.stdout:{"id":"b88c2392-6ef8-4660-b080-19480b748fc0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-28T16:50:29.505 DEBUG:tasks.util.rgw: json result: {'id': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-28T16:50:29.505 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-03-28T16:50:29.505 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-03-28T16:50:29.505 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-03-28T16:50:29.584 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:29.584 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:29.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4bc871980 1 Processor -- start 2024-03-28T16:50:29.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4bc871980 1 -- start start 2024-03-28T16:50:29.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4bc871980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55873d62e450 0x55873d630840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4bc871980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55873d5764c0 con 0x55873d629f90 2024-03-28T16:50:29.584 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4bc871980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55873d53d0f0 con 0x55873d62e450 2024-03-28T16:50:29.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4b9b8b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55873d62e450 0x55873d630840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4ba38c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55873d629f90 0x55873d62a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:33964/0 (socket says 172.21.15.170:33964) 2024-03-28T16:50:29.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4ba38c640 1 -- 172.21.15.170:0/3840771996 learned_addr learned my addr 172.21.15.170:0/3840771996 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:29.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4b9b8b640 1 -- 172.21.15.170:0/3840771996 >> v1:172.21.15.119:6789/0 conn(0x55873d629f90 legacy=0x55873d62a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.585 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.584+0000 7fe4b9b8b640 1 -- 172.21.15.170:0/3840771996 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55873d631d90 con 0x55873d62e450 2024-03-28T16:50:29.586 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4b9b8b640 1 --2- 172.21.15.170:0/3840771996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d62e450 0x55873d630840 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a480 tx=0x7fe4b001b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dc72f838922f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4b938a640 1 -- 172.21.15.170:0/3840771996 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b00ab540 con 0x55873d62e450 2024-03-28T16:50:29.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4b938a640 1 -- 172.21.15.170:0/3840771996 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b00ab6a0 con 0x55873d62e450 2024-03-28T16:50:29.587 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4b938a640 1 -- 172.21.15.170:0/3840771996 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b00ab960 con 0x55873d62e450 2024-03-28T16:50:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4bc871980 1 -- 172.21.15.170:0/3840771996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d62e450 msgr2=0x55873d630840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3840771996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d62e450 0x55873d630840 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a480 tx=0x7fe4b001b280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4bc871980 1 -- 172.21.15.170:0/3840771996 shutdown_connections 2024-03-28T16:50:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3840771996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d62e450 0x55873d630840 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4bc871980 1 -- 172.21.15.170:0/3840771996 >> 172.21.15.170:0/3840771996 conn(0x55873d625860 msgr2=0x55873d627c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4bc871980 1 -- 172.21.15.170:0/3840771996 shutdown_connections 2024-03-28T16:50:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.588+0000 7fe4bc871980 1 -- 172.21.15.170:0/3840771996 wait complete. 2024-03-28T16:50:29.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.600+0000 7fe4bc871980 1 Processor -- start 2024-03-28T16:50:29.602 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.600+0000 7fe4bc871980 1 -- start start 2024-03-28T16:50:29.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.600+0000 7fe4bc871980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d713ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.600+0000 7fe4bc871980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873d53a620 con 0x55873d629f90 2024-03-28T16:50:29.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d713ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d713ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37776/0 (socket says 172.21.15.170:37776) 2024-03-28T16:50:29.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 -- 172.21.15.170:0/485339079 learned_addr learned my addr 172.21.15.170:0/485339079 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 -- 172.21.15.170:0/485339079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b0005d30 con 0x55873d629f90 2024-03-28T16:50:29.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 --2- 172.21.15.170:0/485339079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d713ed0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fb0 tx=0x7fe4b4001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f07dec99f483649 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe3a8608640 1 -- 172.21.15.170:0/485339079 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4004530 con 0x55873d629f90 2024-03-28T16:50:29.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe3a8608640 1 -- 172.21.15.170:0/485339079 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4004690 con 0x55873d629f90 2024-03-28T16:50:29.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe3a8608640 1 -- 172.21.15.170:0/485339079 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b40049b0 con 0x55873d629f90 2024-03-28T16:50:29.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 -- 172.21.15.170:0/485339079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 msgr2=0x55873d713ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 --2- 172.21.15.170:0/485339079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d713ed0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fb0 tx=0x7fe4b4001940 comp rx=0 tx=0).stop 2024-03-28T16:50:29.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 -- 172.21.15.170:0/485339079 shutdown_connections 2024-03-28T16:50:29.605 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 --2- 172.21.15.170:0/485339079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d713ed0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 -- 172.21.15.170:0/485339079 >> 172.21.15.170:0/485339079 conn(0x55873d625860 msgr2=0x55873d70fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 -- 172.21.15.170:0/485339079 shutdown_connections 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 -- 172.21.15.170:0/485339079 wait complete. 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 Processor -- start 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 -- start start 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d71ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4bc871980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873d535e00 con 0x55873d629f90 2024-03-28T16:50:29.606 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d71ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d71ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37784/0 (socket says 172.21.15.170:37784) 2024-03-28T16:50:29.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.604+0000 7fe4ba38c640 1 -- 172.21.15.170:0/3399812747 learned_addr learned my addr 172.21.15.170:0/3399812747 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4ba38c640 1 -- 172.21.15.170:0/3399812747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55873d5764c0 con 0x55873d629f90 2024-03-28T16:50:29.607 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4ba38c640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d71ae40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x55873d714d60 tx=0x7fe4b4000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe3a6e05640 1 -- 172.21.15.170:0/3399812747 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4013760 con 0x55873d629f90 2024-03-28T16:50:29.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe3a6e05640 1 -- 172.21.15.170:0/3399812747 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b40138c0 con 0x55873d629f90 2024-03-28T16:50:29.608 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe3a6e05640 1 -- 172.21.15.170:0/3399812747 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4013b60 con 0x55873d629f90 2024-03-28T16:50:29.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55873d71b380 con 0x55873d629f90 2024-03-28T16:50:29.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55873d71b7b0 con 0x55873d629f90 2024-03-28T16:50:29.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bc871980 1 Processor -- start 2024-03-28T16:50:29.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bc871980 1 -- start start 2024-03-28T16:50:29.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bc871980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d728000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.609 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bc871980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873d4cee30 con 0x55873d727be0 2024-03-28T16:50:29.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bab8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d728000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bab8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d728000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37794/0 (socket says 172.21.15.170:37794) 2024-03-28T16:50:29.610 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.608+0000 7fe4bab8d640 1 -- 172.21.15.170:0/1644191132 learned_addr learned my addr 172.21.15.170:0/1644191132 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe3a6e05640 1 -- 172.21.15.170:0/3399812747 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe4b4013cc0 con 0x55873d629f90 2024-03-28T16:50:29.611 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe3a6e05640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3a0034350 0x7fe3a0036810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bab8d640 1 -- 172.21.15.170:0/1644191132 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55873d728540 con 0x55873d727be0 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe3a6e05640 1 -- 172.21.15.170:0/3399812747 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe4b40494b0 con 0x55873d629f90 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4ba38c640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3a0034350 0x7fe3a0036810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bab8d640 1 --2- 172.21.15.170:0/1644191132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d728000 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00a440 tx=0x7fe4ac00a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ef42e1a4645d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4ba38c640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3a0034350 0x7fe3a0036810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4013fd0 tx=0x7fe4b4028500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe3a4e01640 1 -- 172.21.15.170:0/1644191132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac008850 con 0x55873d727be0 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe3a4e01640 1 -- 172.21.15.170:0/1644191132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac0089d0 con 0x55873d727be0 2024-03-28T16:50:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe3a4e01640 1 -- 172.21.15.170:0/1644191132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac008cb0 con 0x55873d727be0 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- 172.21.15.170:0/1644191132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 msgr2=0x55873d728000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 --2- 172.21.15.170:0/1644191132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d728000 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00a440 tx=0x7fe4ac00a900 comp rx=0 tx=0).stop 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- 172.21.15.170:0/1644191132 shutdown_connections 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 --2- 172.21.15.170:0/1644191132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d728000 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- 172.21.15.170:0/1644191132 >> 172.21.15.170:0/1644191132 conn(0x55873d62e450 msgr2=0x55873d723050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- 172.21.15.170:0/1644191132 shutdown_connections 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- 172.21.15.170:0/1644191132 wait complete. 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 Processor -- start 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- start start 2024-03-28T16:50:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d72eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873d4c2790 con 0x55873d727be0 2024-03-28T16:50:29.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bab8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d72eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bab8d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d72eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37806/0 (socket says 172.21.15.170:37806) 2024-03-28T16:50:29.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bab8d640 1 -- 172.21.15.170:0/3149503592 learned_addr learned my addr 172.21.15.170:0/3149503592 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bab8d640 1 -- 172.21.15.170:0/3149503592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac005cd0 con 0x55873d727be0 2024-03-28T16:50:29.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bab8d640 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d72eb40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55873d7289c0 tx=0x7fe4ac003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe39effd640 1 -- 172.21.15.170:0/3149503592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac013aa0 con 0x55873d727be0 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55873d72f080 con 0x55873d727be0 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe39effd640 1 -- 172.21.15.170:0/3149503592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac013c00 con 0x55873d727be0 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55873d72f4b0 con 0x55873d727be0 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.612+0000 7fe39effd640 1 -- 172.21.15.170:0/3149503592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac013ea0 con 0x55873d727be0 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe39effd640 1 -- 172.21.15.170:0/3149503592 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe4ac028460 con 0x55873d727be0 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe39effd640 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3940342b0 0x7fe394036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe39effd640 1 -- 172.21.15.170:0/3149503592 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe4ac0498d0 con 0x55873d727be0 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 1 Processor -- start 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bab8d640 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3940342b0 0x7fe394036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 1 -- start start 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d7487e0 0x55873d748c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873d749140 con 0x55873d7487e0 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bab8d640 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3940342b0 0x7fe394036770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0118c0 tx=0x7fe4ac011750 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4ba38c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d7487e0 0x55873d748c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4ba38c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d7487e0 0x55873d748c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37818/0 (socket says 172.21.15.170:37818) 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4ba38c640 1 -- 172.21.15.170:0/784160496 learned_addr learned my addr 172.21.15.170:0/784160496 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4ba38c640 1 -- 172.21.15.170:0/784160496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55873d749280 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4ba38c640 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d7487e0 0x55873d748c00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4001980 tx=0x7fe4b4047820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe37f7e6640 1 -- 172.21.15.170:0/784160496 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b40278e0 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55873d749bd0 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe37f7e6640 1 -- 172.21.15.170:0/784160496 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4027700 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe37f7e6640 1 -- 172.21.15.170:0/784160496 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b401d380 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.616+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55873d749bd0 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe37f7e6640 1 -- 172.21.15.170:0/784160496 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe4b4046020 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe37f7e6640 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2f00342b0 0x7fe2f0036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe37f7e6640 1 -- 172.21.15.170:0/784160496 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe4b404d910 con 0x55873d7487e0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4b9b8b640 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2f00342b0 0x7fe2f0036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe37d7e2640 20 reqs_thread_entry: start 2024-03-28T16:50:29.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55873d74d7b0 0x55873d74fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55873d7500e0 con 0x55873d74d7b0 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4b9b8b640 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2f00342b0 0x7fe2f0036770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00ad000 tx=0x7fe4b00a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bab8d640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55873d74d7b0 0x55873d74fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bab8d640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55873d74d7b0 0x55873d74fba0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bab8d640 1 -- 172.21.15.170:0/3399812747 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fe4ac024020 con 0x55873d74d7b0 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55873d751960 0x55873d753d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:2 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55873d754490 con 0x55873d751960 2024-03-28T16:50:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4ba38c640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55873d751960 0x55873d753d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4ba38c640 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55873d751960 0x55873d753d50 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4ba38c640 1 -- 172.21.15.170:0/3399812747 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(2 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x7fe4b4024020 con 0x55873d751960 2024-03-28T16:50:29.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.620+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55873d759030 con 0x55873d751960 2024-03-28T16:50:29.624 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4ba38c640 1 -- 172.21.15.170:0/3399812747 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [stat,call,call,writefull 0~874] v31'4 uv4 ondisk = 0) v8 ==== 316+0+0 (crc 0 0 0) 0x7fe4b4024020 con 0x55873d751960 2024-03-28T16:50:29.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3940342b0 msgr2=0x7fe394036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3940342b0 0x7fe394036770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0118c0 tx=0x7fe4ac011750 comp rx=0 tx=0).stop 2024-03-28T16:50:29.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 msgr2=0x55873d72eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d72eb40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55873d7289c0 tx=0x7fe4ac003000 comp rx=0 tx=0).stop 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 shutdown_connections 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3940342b0 0x7fe394036770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3149503592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d727be0 0x55873d72eb40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 >> 172.21.15.170:0/3149503592 conn(0x55873d62e450 msgr2=0x55873d721e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 shutdown_connections 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3149503592 wait complete. 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2f00342b0 msgr2=0x7fe2f0036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2f00342b0 0x7fe2f0036770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00ad000 tx=0x7fe4b00a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d7487e0 msgr2=0x55873d748c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d7487e0 0x55873d748c00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4001980 tx=0x7fe4b4047820 comp rx=0 tx=0).stop 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 shutdown_connections 2024-03-28T16:50:29.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe2f00342b0 0x7fe2f0036770 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/784160496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d7487e0 0x55873d748c00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 >> 172.21.15.170:0/784160496 conn(0x55873d73d560 msgr2=0x55873d73df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 shutdown_connections 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/784160496 wait complete. 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55873d751960 msgr2=0x55873d753d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55873d751960 0x55873d753d50 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55873d74d7b0 msgr2=0x55873d74fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.624+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55873d74d7b0 0x55873d74fba0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3a0034350 msgr2=0x7fe3a0036810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3a0034350 0x7fe3a0036810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4013fd0 tx=0x7fe4b4028500 comp rx=0 tx=0).stop 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 msgr2=0x55873d71ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d71ae40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x55873d714d60 tx=0x7fe4b4000f50 comp rx=0 tx=0).stop 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 shutdown_connections 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55873d751960 0x55873d753d50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55873d74d7b0 0x55873d74fba0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe3a0034350 0x7fe3a0036810 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 --2- 172.21.15.170:0/3399812747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55873d629f90 0x55873d71ae40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 >> 172.21.15.170:0/3399812747 conn(0x55873d625860 msgr2=0x55873d70e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 shutdown_connections 2024-03-28T16:50:29.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.628+0000 7fe4bc871980 1 -- 172.21.15.170:0/3399812747 wait complete. 2024-03-28T16:50:29.637 INFO:teuthology.orchestra.run.smithi170.stdout:{"id":"b88c2392-6ef8-4660-b080-19480b748fc0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-28T16:50:29.637 DEBUG:tasks.util.rgw: json result: {'id': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-28T16:50:29.637 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-03-28T16:50:29.637 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-03-28T16:50:29.638 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-03-28T16:50:29.716 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:29.716 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:29.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f2602980 1 Processor -- start 2024-03-28T16:50:29.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f2602980 1 -- start start 2024-03-28T16:50:29.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f2602980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x559b8061f0a0 0x559b806adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f2602980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559b805f7510 con 0x559b806ae0e0 2024-03-28T16:50:29.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f2602980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x559b805be0f0 con 0x559b8061f0a0 2024-03-28T16:50:29.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f011d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x559b8061f0a0 0x559b806adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22ef91c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x559b806ae0e0 0x559b806b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:33966/0 (socket says 172.21.15.170:33966) 2024-03-28T16:50:29.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f011d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x559b8061f0a0 0x559b806adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37826/0 (socket says 172.21.15.170:37826) 2024-03-28T16:50:29.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22ef91c640 1 -- 172.21.15.170:0/2950221545 learned_addr learned my addr 172.21.15.170:0/2950221545 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:29.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f011d640 1 -- 172.21.15.170:0/2950221545 >> v1:172.21.15.119:6789/0 conn(0x559b806ae0e0 legacy=0x559b806b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.716+0000 7f22f011d640 1 -- 172.21.15.170:0/2950221545 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b806b2cc0 con 0x559b8061f0a0 2024-03-28T16:50:29.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f011d640 1 --2- 172.21.15.170:0/2950221545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b8061f0a0 0x559b806adba0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f22e000a480 tx=0x7f22e001b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c8c7c93646952a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22ef11b640 1 -- 172.21.15.170:0/2950221545 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00ab540 con 0x559b8061f0a0 2024-03-28T16:50:29.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22ef11b640 1 -- 172.21.15.170:0/2950221545 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e00ab6a0 con 0x559b8061f0a0 2024-03-28T16:50:29.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22ef11b640 1 -- 172.21.15.170:0/2950221545 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00ab960 con 0x559b8061f0a0 2024-03-28T16:50:29.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f2602980 1 -- 172.21.15.170:0/2950221545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b8061f0a0 msgr2=0x559b806adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f2602980 1 --2- 172.21.15.170:0/2950221545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b8061f0a0 0x559b806adba0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f22e000a480 tx=0x7f22e001b280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f2602980 1 -- 172.21.15.170:0/2950221545 shutdown_connections 2024-03-28T16:50:29.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f2602980 1 --2- 172.21.15.170:0/2950221545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b8061f0a0 0x559b806adba0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f2602980 1 -- 172.21.15.170:0/2950221545 >> 172.21.15.170:0/2950221545 conn(0x559b806a56c0 msgr2=0x559b806a7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f2602980 1 -- 172.21.15.170:0/2950221545 shutdown_connections 2024-03-28T16:50:29.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.720+0000 7f22f2602980 1 -- 172.21.15.170:0/2950221545 wait complete. 2024-03-28T16:50:29.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.736+0000 7f22f2602980 1 Processor -- start 2024-03-28T16:50:29.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.736+0000 7f22f2602980 1 -- start start 2024-03-28T16:50:29.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.736+0000 7f22f2602980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b80794fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.736+0000 7f22f2602980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b805bb620 con 0x559b806ae0e0 2024-03-28T16:50:29.738 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.736+0000 7f22f011d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b80794fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.736+0000 7f22f011d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b80794fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37840/0 (socket says 172.21.15.170:37840) 2024-03-28T16:50:29.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.736+0000 7f22f011d640 1 -- 172.21.15.170:0/3795068468 learned_addr learned my addr 172.21.15.170:0/3795068468 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f011d640 1 -- 172.21.15.170:0/3795068468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e0005d30 con 0x559b806ae0e0 2024-03-28T16:50:29.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f011d640 1 --2- 172.21.15.170:0/3795068468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b80794fb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f22e001b2c0 tx=0x7f22e0003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f62ecdf51cf8b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f21de60c640 1 -- 172.21.15.170:0/3795068468 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00ab940 con 0x559b806ae0e0 2024-03-28T16:50:29.739 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f21de60c640 1 -- 172.21.15.170:0/3795068468 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e00abaa0 con 0x559b806ae0e0 2024-03-28T16:50:29.740 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f21de60c640 1 -- 172.21.15.170:0/3795068468 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00abd80 con 0x559b806ae0e0 2024-03-28T16:50:29.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- 172.21.15.170:0/3795068468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 msgr2=0x559b80794fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 --2- 172.21.15.170:0/3795068468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b80794fb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f22e001b2c0 tx=0x7f22e0003030 comp rx=0 tx=0).stop 2024-03-28T16:50:29.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- 172.21.15.170:0/3795068468 shutdown_connections 2024-03-28T16:50:29.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 --2- 172.21.15.170:0/3795068468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b80794fb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- 172.21.15.170:0/3795068468 >> 172.21.15.170:0/3795068468 conn(0x559b806a56c0 msgr2=0x559b80790aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- 172.21.15.170:0/3795068468 shutdown_connections 2024-03-28T16:50:29.741 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- 172.21.15.170:0/3795068468 wait complete. 2024-03-28T16:50:29.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 Processor -- start 2024-03-28T16:50:29.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- start start 2024-03-28T16:50:29.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b8079bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b805b6e00 con 0x559b806ae0e0 2024-03-28T16:50:29.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f011d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b8079bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f011d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b8079bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37856/0 (socket says 172.21.15.170:37856) 2024-03-28T16:50:29.742 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f011d640 1 -- 172.21.15.170:0/3109743765 learned_addr learned my addr 172.21.15.170:0/3109743765 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f011d640 1 -- 172.21.15.170:0/3109743765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b806b2cc0 con 0x559b806ae0e0 2024-03-28T16:50:29.743 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f011d640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b8079bcc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f22e001b2c0 tx=0x7f22e0018020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f21dce09640 1 -- 172.21.15.170:0/3109743765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00c3070 con 0x559b806ae0e0 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.740+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559b8079c200 con 0x559b806ae0e0 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21dce09640 1 -- 172.21.15.170:0/3109743765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e00beba0 con 0x559b806ae0e0 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21dce09640 1 -- 172.21.15.170:0/3109743765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00bee60 con 0x559b806ae0e0 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b8079c690 con 0x559b806ae0e0 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 Processor -- start 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- start start 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b806ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.744 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b8054fe30 con 0x559b806aadd0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f091e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b806ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f091e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b806ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37872/0 (socket says 172.21.15.170:37872) 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f091e640 1 -- 172.21.15.170:0/4033679438 learned_addr learned my addr 172.21.15.170:0/4033679438 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21dce09640 1 -- 172.21.15.170:0/3109743765 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f22e00b2070 con 0x559b806ae0e0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21dce09640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21d80342b0 0x7f21d8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21dce09640 1 -- 172.21.15.170:0/3109743765 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f22e00e3e30 con 0x559b806ae0e0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f091e640 1 -- 172.21.15.170:0/4033679438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b806ab730 con 0x559b806aadd0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f011d640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21d80342b0 0x7f21d8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f091e640 1 --2- 172.21.15.170:0/4033679438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b806ab1f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f22e800a430 tx=0x7f22e800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a770f46b33c2cbc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f011d640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21d80342b0 0x7f21d8036770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f22e00d1030 tx=0x7f22e00b5eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21d67fc640 1 -- 172.21.15.170:0/4033679438 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e800f6e0 con 0x559b806aadd0 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21d67fc640 1 -- 172.21.15.170:0/4033679438 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e800f860 con 0x559b806aadd0 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f21d67fc640 1 -- 172.21.15.170:0/4033679438 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e800fb40 con 0x559b806aadd0 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- 172.21.15.170:0/4033679438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 msgr2=0x559b806ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 --2- 172.21.15.170:0/4033679438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b806ab1f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f22e800a430 tx=0x7f22e800a8f0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- 172.21.15.170:0/4033679438 shutdown_connections 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 --2- 172.21.15.170:0/4033679438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b806ab1f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- 172.21.15.170:0/4033679438 >> 172.21.15.170:0/4033679438 conn(0x559b807a3f30 msgr2=0x559b807a4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- 172.21.15.170:0/4033679438 shutdown_connections 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- 172.21.15.170:0/4033679438 wait complete. 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 Processor -- start 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- start start 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b807b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f2602980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b80543790 con 0x559b806aadd0 2024-03-28T16:50:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.744+0000 7f22f091e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b807b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f091e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b807b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37886/0 (socket says 172.21.15.170:37886) 2024-03-28T16:50:29.749 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f091e640 1 -- 172.21.15.170:0/129194035 learned_addr learned my addr 172.21.15.170:0/129194035 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f091e640 1 -- 172.21.15.170:0/129194035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e800a0e0 con 0x559b806aadd0 2024-03-28T16:50:29.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f091e640 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b807b2080 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x559b807a5a60 tx=0x7f22e8003380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f21d4ff9640 1 -- 172.21.15.170:0/129194035 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e8010a20 con 0x559b806aadd0 2024-03-28T16:50:29.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559b807b25c0 con 0x559b806aadd0 2024-03-28T16:50:29.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f21d4ff9640 1 -- 172.21.15.170:0/129194035 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e8010b80 con 0x559b806aadd0 2024-03-28T16:50:29.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f21d4ff9640 1 -- 172.21.15.170:0/129194035 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e801a480 con 0x559b806aadd0 2024-03-28T16:50:29.751 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b807b2a50 con 0x559b806aadd0 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f21d4ff9640 1 -- 172.21.15.170:0/129194035 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f22e801a6f0 con 0x559b806aadd0 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f21d4ff9640 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21c80342b0 0x7f21c8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 1 Processor -- start 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f21d4ff9640 1 -- 172.21.15.170:0/129194035 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f22e8049e40 con 0x559b806aadd0 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f091e640 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21c80342b0 0x7f21c8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 1 -- start start 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b807cbe10 0x559b807cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f2602980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b807cc770 con 0x559b807cbe10 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f091e640 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21c80342b0 0x7f21c8036770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f22e802efd0 tx=0x7f22e8024f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f011d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b807cbe10 0x559b807cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f011d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b807cbe10 0x559b807cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37902/0 (socket says 172.21.15.170:37902) 2024-03-28T16:50:29.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.748+0000 7f22f011d640 1 -- 172.21.15.170:0/3561257564 learned_addr learned my addr 172.21.15.170:0/3561257564 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f011d640 1 -- 172.21.15.170:0/3561257564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b807cc8b0 con 0x559b807cbe10 2024-03-28T16:50:29.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f011d640 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b807cbe10 0x559b807cc230 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f22e00be6a0 tx=0x7f22e00b1650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f21b4fe1640 1 -- 172.21.15.170:0/3561257564 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00aa3b0 con 0x559b807cbe10 2024-03-28T16:50:29.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559b807cd200 con 0x559b807cbe10 2024-03-28T16:50:29.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f21b4fe1640 1 -- 172.21.15.170:0/3561257564 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e00c4450 con 0x559b807cbe10 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f21b4fe1640 1 -- 172.21.15.170:0/3561257564 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e00c4270 con 0x559b807cbe10 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b807cd690 con 0x559b807cbe10 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f21b4fe1640 1 -- 172.21.15.170:0/3561257564 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f22e00b5830 con 0x559b807cbe10 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f21b4fe1640 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f212c0342b0 0x7f212c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f21b4fe1640 1 -- 172.21.15.170:0/3561257564 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f22e00ebba0 con 0x559b807cbe10 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22ef91c640 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f212c0342b0 0x7f212c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f21367fc640 20 reqs_thread_entry: start 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x559b8079fb70 0x559b807d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559b807d3370 con 0x559b8079fb70 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22ef91c640 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f212c0342b0 0x7f212c036770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x559b807a4b90 tx=0x7f22e4001280 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f091e640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x559b8079fb70 0x559b807d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f091e640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x559b8079fb70 0x559b807d2df0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f091e640 1 -- 172.21.15.170:0/3109743765 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f22e8016020 con 0x559b8079fb70 2024-03-28T16:50:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.752+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559b807d3b70 con 0x559b8079fb70 2024-03-28T16:50:29.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f091e640 1 -- 172.21.15.170:0/3109743765 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f22e8016020 con 0x559b8079fb70 2024-03-28T16:50:29.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x559b807d5480 0x559b807d78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:3 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x559b807d8080 con 0x559b807d5480 2024-03-28T16:50:29.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f011d640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x559b807d5480 0x559b807d78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f011d640 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x559b807d5480 0x559b807d78f0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f011d640 1 -- 172.21.15.170:0/3109743765 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(3 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f22e00d7070 con 0x559b807d5480 2024-03-28T16:50:29.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21c80342b0 msgr2=0x7f21c8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21c80342b0 0x7f21c8036770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f22e802efd0 tx=0x7f22e8024f00 comp rx=0 tx=0).stop 2024-03-28T16:50:29.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 msgr2=0x559b807b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b807b2080 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x559b807a5a60 tx=0x7f22e8003380 comp rx=0 tx=0).stop 2024-03-28T16:50:29.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 shutdown_connections 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21c80342b0 0x7f21c8036770 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/129194035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806aadd0 0x559b807b2080 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 >> 172.21.15.170:0/129194035 conn(0x559b807a3f30 msgr2=0x559b807a4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 shutdown_connections 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/129194035 wait complete. 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f212c0342b0 msgr2=0x7f212c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f212c0342b0 0x7f212c036770 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x559b807a4b90 tx=0x7f22e4001280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b807cbe10 msgr2=0x559b807cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b807cbe10 0x559b807cc230 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f22e00be6a0 tx=0x7f22e00b1650 comp rx=0 tx=0).stop 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 shutdown_connections 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f212c0342b0 0x7f212c036770 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3561257564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b807cbe10 0x559b807cc230 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 >> 172.21.15.170:0/3561257564 conn(0x559b807c0b30 msgr2=0x559b807c1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 shutdown_connections 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3561257564 wait complete. 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x559b8079fb70 msgr2=0x559b807d2df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x559b8079fb70 0x559b807d2df0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x559b807d5480 msgr2=0x559b807d78f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x559b807d5480 0x559b807d78f0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21d80342b0 msgr2=0x7f21d8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21d80342b0 0x7f21d8036770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f22e00d1030 tx=0x7f22e00b5eb0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 msgr2=0x559b8079bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b8079bcc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f22e001b2c0 tx=0x7f22e0018020 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 shutdown_connections 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x559b807d5480 0x559b807d78f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x559b8079fb70 0x559b807d2df0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f21d80342b0 0x7f21d8036770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 --2- 172.21.15.170:0/3109743765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b806ae0e0 0x559b8079bcc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 >> 172.21.15.170:0/3109743765 conn(0x559b806a56c0 msgr2=0x559b8078f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 shutdown_connections 2024-03-28T16:50:29.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.756+0000 7f22f2602980 1 -- 172.21.15.170:0/3109743765 wait complete. 2024-03-28T16:50:29.770 INFO:teuthology.orchestra.run.smithi170.stdout:{"id":"c7513231-bef6-43ae-ade6-ce57a95c9845","name":"default","api_name":"default","is_master":true,"endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"b88c2392-6ef8-4660-b080-19480b748fc0","zones":[{"id":"b88c2392-6ef8-4660-b080-19480b748fc0","name":"default","endpoints":[],"log_meta":false,"log_data":false,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"","sync_policy":{"groups":[]},"enabled_features":["resharding"]} 2024-03-28T16:50:29.770 DEBUG:tasks.util.rgw: json result: {'id': 'c7513231-bef6-43ae-ade6-ce57a95c9845', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'zones': [{'id': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['resharding']} 2024-03-28T16:50:29.770 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zonegroup', 'default', '--placement-id', 'default-placement', '--storage-class', 'GLACIER'] 2024-03-28T16:50:29.770 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zonegroup', 'default', '--placement-id', 'default-placement', '--storage-class', 'GLACIER'] 2024-03-28T16:50:29.770 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zonegroup default --placement-id default-placement --storage-class GLACIER 2024-03-28T16:50:29.849 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:29.849 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:29.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.848+0000 7f74330e1980 1 Processor -- start 2024-03-28T16:50:29.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.848+0000 7f74330e1980 1 -- start start 2024-03-28T16:50:29.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.848+0000 7f74330e1980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56538324f6f0 0x565383251ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.848+0000 7f74330e1980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5653831954c0 con 0x56538324b1e0 2024-03-28T16:50:29.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.848+0000 7f74330e1980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56538315c0f0 con 0x56538324f6f0 2024-03-28T16:50:29.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.848+0000 7f74303fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56538324f6f0 0x565383251ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f7430bfc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x56538324b1e0 0x56538324b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:33978/0 (socket says 172.21.15.170:33978) 2024-03-28T16:50:29.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74303fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56538324f6f0 0x565383251ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37914/0 (socket says 172.21.15.170:37914) 2024-03-28T16:50:29.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f7430bfc640 1 -- 172.21.15.170:0/3460763921 learned_addr learned my addr 172.21.15.170:0/3460763921 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:29.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74303fb640 1 -- 172.21.15.170:0/3460763921 >> v1:172.21.15.119:6789/0 conn(0x56538324b1e0 legacy=0x56538324b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74303fb640 1 -- 172.21.15.170:0/3460763921 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565383253030 con 0x56538324f6f0 2024-03-28T16:50:29.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74303fb640 1 --2- 172.21.15.170:0/3460763921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324f6f0 0x565383251ae0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f741800a480 tx=0x7f741801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866fef7e23eb09dd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f742fbfa640 1 -- 172.21.15.170:0/3460763921 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74180ab540 con 0x56538324f6f0 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f742fbfa640 1 -- 172.21.15.170:0/3460763921 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74180ab6a0 con 0x56538324f6f0 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f742fbfa640 1 -- 172.21.15.170:0/3460763921 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74180ab960 con 0x56538324f6f0 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74330e1980 1 -- 172.21.15.170:0/3460763921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324f6f0 msgr2=0x565383251ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74330e1980 1 --2- 172.21.15.170:0/3460763921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324f6f0 0x565383251ae0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f741800a480 tx=0x7f741801b280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74330e1980 1 -- 172.21.15.170:0/3460763921 shutdown_connections 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74330e1980 1 --2- 172.21.15.170:0/3460763921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324f6f0 0x565383251ae0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74330e1980 1 -- 172.21.15.170:0/3460763921 >> 172.21.15.170:0/3460763921 conn(0x5653831be930 msgr2=0x5653831bed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74330e1980 1 -- 172.21.15.170:0/3460763921 shutdown_connections 2024-03-28T16:50:29.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.852+0000 7f74330e1980 1 -- 172.21.15.170:0/3460763921 wait complete. 2024-03-28T16:50:29.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f74330e1980 1 Processor -- start 2024-03-28T16:50:29.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f74330e1980 1 -- start start 2024-03-28T16:50:29.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f74330e1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x5653833351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f74330e1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565383159620 con 0x56538324b1e0 2024-03-28T16:50:29.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f7430bfc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x5653833351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f7430bfc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x5653833351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37918/0 (socket says 172.21.15.170:37918) 2024-03-28T16:50:29.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f7430bfc640 1 -- 172.21.15.170:0/2650148766 learned_addr learned my addr 172.21.15.170:0/2650148766 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.868+0000 7f7430bfc640 1 -- 172.21.15.170:0/2650148766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418005d30 con 0x56538324b1e0 2024-03-28T16:50:29.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f7430bfc640 1 --2- 172.21.15.170:0/2650148766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x5653833351e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7420006fb0 tx=0x7f7420001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2471367b779b9ae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f731ee0d640 1 -- 172.21.15.170:0/2650148766 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7420004530 con 0x56538324b1e0 2024-03-28T16:50:29.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f731ee0d640 1 -- 172.21.15.170:0/2650148766 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7420004690 con 0x56538324b1e0 2024-03-28T16:50:29.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f731ee0d640 1 -- 172.21.15.170:0/2650148766 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74200049b0 con 0x56538324b1e0 2024-03-28T16:50:29.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- 172.21.15.170:0/2650148766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 msgr2=0x5653833351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 --2- 172.21.15.170:0/2650148766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x5653833351e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7420006fb0 tx=0x7f7420001940 comp rx=0 tx=0).stop 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- 172.21.15.170:0/2650148766 shutdown_connections 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 --2- 172.21.15.170:0/2650148766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x5653833351e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- 172.21.15.170:0/2650148766 >> 172.21.15.170:0/2650148766 conn(0x5653831be930 msgr2=0x565383330d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- 172.21.15.170:0/2650148766 shutdown_connections 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- 172.21.15.170:0/2650148766 wait complete. 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 Processor -- start 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- start start 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x56538333c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x565383154e00 con 0x56538324b1e0 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f7430bfc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x56538333c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f7430bfc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x56538333c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37930/0 (socket says 172.21.15.170:37930) 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f7430bfc640 1 -- 172.21.15.170:0/4147615039 learned_addr learned my addr 172.21.15.170:0/4147615039 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f7430bfc640 1 -- 172.21.15.170:0/4147615039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653831954c0 con 0x56538324b1e0 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f7430bfc640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x56538333c030 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7420001980 tx=0x7f7420000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f731d60a640 1 -- 172.21.15.170:0/4147615039 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7420013630 con 0x56538324b1e0 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56538333c570 con 0x56538324b1e0 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f731d60a640 1 -- 172.21.15.170:0/4147615039 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7420013790 con 0x56538324b1e0 2024-03-28T16:50:29.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56538333c9a0 con 0x56538324b1e0 2024-03-28T16:50:29.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.872+0000 7f731d60a640 1 -- 172.21.15.170:0/4147615039 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7420013a30 con 0x56538324b1e0 2024-03-28T16:50:29.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 Processor -- start 2024-03-28T16:50:29.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- start start 2024-03-28T16:50:29.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x565383349260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653830ede30 con 0x565383348e40 2024-03-28T16:50:29.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f731d60a640 1 -- 172.21.15.170:0/4147615039 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f7420013b90 con 0x56538324b1e0 2024-03-28T16:50:29.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f731d60a640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73180342b0 0x7f7318036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x565383349260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x565383349260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37932/0 (socket says 172.21.15.170:37932) 2024-03-28T16:50:29.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 -- 172.21.15.170:0/4214376634 learned_addr learned my addr 172.21.15.170:0/4214376634 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f731d60a640 1 -- 172.21.15.170:0/4147615039 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f74200493c0 con 0x56538324b1e0 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f7430bfc640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73180342b0 0x7f7318036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 -- 172.21.15.170:0/4214376634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653833497a0 con 0x565383348e40 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f7430bfc640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73180342b0 0x7f7318036770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f742001e590 tx=0x7f742001e4a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 --2- 172.21.15.170:0/4214376634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x565383349260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f742800a440 tx=0x7f742800a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9586694780bc6a16 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f7316ffd640 1 -- 172.21.15.170:0/4214376634 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7428008850 con 0x565383348e40 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f7316ffd640 1 -- 172.21.15.170:0/4214376634 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74280089d0 con 0x565383348e40 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f7316ffd640 1 -- 172.21.15.170:0/4214376634 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7428008cb0 con 0x565383348e40 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- 172.21.15.170:0/4214376634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 msgr2=0x565383349260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 --2- 172.21.15.170:0/4214376634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x565383349260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f742800a440 tx=0x7f742800a900 comp rx=0 tx=0).stop 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- 172.21.15.170:0/4214376634 shutdown_connections 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 --2- 172.21.15.170:0/4214376634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x565383349260 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- 172.21.15.170:0/4214376634 >> 172.21.15.170:0/4214376634 conn(0x56538324f6f0 msgr2=0x5653833442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- 172.21.15.170:0/4214376634 shutdown_connections 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- 172.21.15.170:0/4214376634 wait complete. 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 Processor -- start 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- start start 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x56538334fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74330e1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653830e1790 con 0x565383348e40 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x56538334fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x56538334fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37934/0 (socket says 172.21.15.170:37934) 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.876+0000 7f74313fd640 1 -- 172.21.15.170:0/4254607345 learned_addr learned my addr 172.21.15.170:0/4254607345 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74313fd640 1 -- 172.21.15.170:0/4254607345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428005cd0 con 0x565383348e40 2024-03-28T16:50:29.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74313fd640 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x56538334fd90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x565383349c20 tx=0x7f7428003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f73157fa640 1 -- 172.21.15.170:0/4254607345 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7428013aa0 con 0x565383348e40 2024-03-28T16:50:29.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653833502d0 con 0x565383348e40 2024-03-28T16:50:29.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565383350700 con 0x565383348e40 2024-03-28T16:50:29.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f73157fa640 1 -- 172.21.15.170:0/4254607345 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428013c00 con 0x565383348e40 2024-03-28T16:50:29.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f73157fa640 1 -- 172.21.15.170:0/4254607345 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7428013ea0 con 0x565383348e40 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f73157fa640 1 -- 172.21.15.170:0/4254607345 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f742801e420 con 0x565383348e40 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f73157fa640 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73080342b0 0x7f7308036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f73157fa640 1 -- 172.21.15.170:0/4254607345 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7428049940 con 0x565383348e40 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74303fb640 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73080342b0 0x7f7308036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 1 Processor -- start 2024-03-28T16:50:29.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 1 -- start start 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653833699f0 0x565383369e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74303fb640 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73080342b0 0x7f7308036770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f74180ad000 tx=0x7f74180a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f74330e1980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56538336a350 con 0x5653833699f0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f7430bfc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653833699f0 0x565383369e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f7430bfc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653833699f0 0x565383369e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37936/0 (socket says 172.21.15.170:37936) 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f7430bfc640 1 -- 172.21.15.170:0/1427681555 learned_addr learned my addr 172.21.15.170:0/1427681555 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.880+0000 7f7430bfc640 1 -- 172.21.15.170:0/1427681555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56538336a490 con 0x5653833699f0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f7430bfc640 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653833699f0 0x565383369e10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7420006960 tx=0x7f74200477f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f72f5fe3640 1 -- 172.21.15.170:0/1427681555 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7420028e70 con 0x5653833699f0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56538336ae10 con 0x5653833699f0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f72f5fe3640 1 -- 172.21.15.170:0/1427681555 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7420012ca0 con 0x5653833699f0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f72f5fe3640 1 -- 172.21.15.170:0/1427681555 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74200473a0 con 0x5653833699f0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56538336ae10 con 0x5653833699f0 2024-03-28T16:50:29.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f72f5fe3640 1 -- 172.21.15.170:0/1427681555 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f74200496d0 con 0x5653833699f0 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f72f5fe3640 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f72680342b0 0x7f7268036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f72f5fe3640 1 -- 172.21.15.170:0/1427681555 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f742004d910 con 0x5653833699f0 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74303fb640 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f72680342b0 0x7f7268036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f72eeffd640 20 reqs_thread_entry: start 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56538336e9f0 0x565383370de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565383371320 con 0x56538336e9f0 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74303fb640 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f72680342b0 0x7f7268036770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f741801b2c0 tx=0x7f7418005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74313fd640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56538336e9f0 0x565383370de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74313fd640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56538336e9f0 0x565383370de0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74313fd640 1 -- 172.21.15.170:0/4147615039 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f7428024020 con 0x56538336e9f0 2024-03-28T16:50:29.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x565383372ba0 0x565383374fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:2 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x565383375710 con 0x565383372ba0 2024-03-28T16:50:29.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f7430bfc640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x565383372ba0 0x565383374fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f7430bfc640 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x565383372ba0 0x565383374fb0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f7430bfc640 1 -- 172.21.15.170:0/4147615039 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(2 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~398 out=398b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+446 (crc 0 0 0) 0x7f7420024020 con 0x565383372ba0 2024-03-28T16:50:29.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.884+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:3 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~421 in=421b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56538337a1b0 con 0x565383372ba0 2024-03-28T16:50:29.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f7430bfc640 1 -- 172.21.15.170:0/4147615039 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(3 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [stat,call,call,writefull 0~421] v31'2 uv2 ondisk = 0) v8 ==== 321+0+0 (crc 0 0 0) 0x7f7420024020 con 0x565383372ba0 2024-03-28T16:50:29.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73080342b0 msgr2=0x7f7308036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73080342b0 0x7f7308036770 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f74180ad000 tx=0x7f74180a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:29.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 msgr2=0x56538334fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x56538334fd90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x565383349c20 tx=0x7f7428003000 comp rx=0 tx=0).stop 2024-03-28T16:50:29.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 shutdown_connections 2024-03-28T16:50:29.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73080342b0 0x7f7308036770 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4254607345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565383348e40 0x56538334fd90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 >> 172.21.15.170:0/4254607345 conn(0x56538324f6f0 msgr2=0x5653833430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 shutdown_connections 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4254607345 wait complete. 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f72680342b0 msgr2=0x7f7268036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f72680342b0 0x7f7268036770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f741801b2c0 tx=0x7f7418005a10 comp rx=0 tx=0).stop 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653833699f0 msgr2=0x565383369e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653833699f0 0x565383369e10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7420006960 tx=0x7f74200477f0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 shutdown_connections 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f72680342b0 0x7f7268036770 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/1427681555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5653833699f0 0x565383369e10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 >> 172.21.15.170:0/1427681555 conn(0x56538335e780 msgr2=0x56538335f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 shutdown_connections 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/1427681555 wait complete. 2024-03-28T16:50:29.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56538336e9f0 msgr2=0x565383370de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56538336e9f0 0x565383370de0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x565383372ba0 msgr2=0x565383374fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x565383372ba0 0x565383374fb0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73180342b0 msgr2=0x7f7318036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73180342b0 0x7f7318036770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f742001e590 tx=0x7f742001e4a0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 msgr2=0x56538333c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.888+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x56538333c030 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7420001980 tx=0x7f7420000f50 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 shutdown_connections 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x565383372ba0 0x565383374fb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56538336e9f0 0x565383370de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f73180342b0 0x7f7318036770 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 --2- 172.21.15.170:0/4147615039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56538324b1e0 0x56538333c030 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 >> 172.21.15.170:0/4147615039 conn(0x5653831be930 msgr2=0x56538332f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 shutdown_connections 2024-03-28T16:50:29.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.892+0000 7f74330e1980 1 -- 172.21.15.170:0/4147615039 wait complete. 2024-03-28T16:50:29.900 INFO:teuthology.orchestra.run.smithi170.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["GLACIER","STANDARD"]}}] 2024-03-28T16:50:29.901 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['GLACIER', 'STANDARD']}}] 2024-03-28T16:50:29.901 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'GLACIER', '--data-pool', '.rgw.buckets.2'] 2024-03-28T16:50:29.901 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'GLACIER', '--data-pool', '.rgw.buckets.2'] 2024-03-28T16:50:29.901 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class GLACIER --data-pool .rgw.buckets.2 2024-03-28T16:50:29.984 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:29.984 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:29.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f37a0025980 1 Processor -- start 2024-03-28T16:50:29.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f37a0025980 1 -- start start 2024-03-28T16:50:29.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f37a0025980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56077f14b370 0x56077f14d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:29.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f37a0025980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56077f0934c0 con 0x56077f0bb240 2024-03-28T16:50:29.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f37a0025980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x56077f05a0f0 con 0x56077f14b370 2024-03-28T16:50:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f379db40640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x56077f0bb240 0x56077f149c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:33986/0 (socket says 172.21.15.170:33986) 2024-03-28T16:50:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f379d33f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x56077f14b370 0x56077f14d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f379db40640 1 -- 172.21.15.170:0/877270212 learned_addr learned my addr 172.21.15.170:0/877270212 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f379d33f640 1 -- 172.21.15.170:0/877270212 >> v1:172.21.15.119:6789/0 conn(0x56077f0bb240 legacy=0x56077f149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.986 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.984+0000 7f379d33f640 1 -- 172.21.15.170:0/877270212 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56077f14ed70 con 0x56077f14b370 2024-03-28T16:50:29.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f379d33f640 1 --2- 172.21.15.170:0/877270212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f14b370 0x56077f14d760 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f378800a480 tx=0x7f378801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d86a405cedbd1db server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:29.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f379cb3e640 1 -- 172.21.15.170:0/877270212 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37880ab540 con 0x56077f14b370 2024-03-28T16:50:29.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f379cb3e640 1 -- 172.21.15.170:0/877270212 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37880ab6a0 con 0x56077f14b370 2024-03-28T16:50:29.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f379cb3e640 1 -- 172.21.15.170:0/877270212 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37880ab960 con 0x56077f14b370 2024-03-28T16:50:29.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f37a0025980 1 -- 172.21.15.170:0/877270212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f14b370 msgr2=0x56077f14d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:29.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f37a0025980 1 --2- 172.21.15.170:0/877270212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f14b370 0x56077f14d760 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f378800a480 tx=0x7f378801b280 comp rx=0 tx=0).stop 2024-03-28T16:50:29.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f37a0025980 1 -- 172.21.15.170:0/877270212 shutdown_connections 2024-03-28T16:50:29.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f37a0025980 1 --2- 172.21.15.170:0/877270212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f14b370 0x56077f14d760 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:29.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f37a0025980 1 -- 172.21.15.170:0/877270212 >> 172.21.15.170:0/877270212 conn(0x56077f141770 msgr2=0x56077f143b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:29.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f37a0025980 1 -- 172.21.15.170:0/877270212 shutdown_connections 2024-03-28T16:50:29.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:29.988+0000 7f37a0025980 1 -- 172.21.15.170:0/877270212 wait complete. 2024-03-28T16:50:30.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f37a0025980 1 Processor -- start 2024-03-28T16:50:30.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f37a0025980 1 -- start start 2024-03-28T16:50:30.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f37a0025980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f231080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f37a0025980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077f057620 con 0x56077f0bb240 2024-03-28T16:50:30.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f379db40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f231080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f379db40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f231080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37960/0 (socket says 172.21.15.170:37960) 2024-03-28T16:50:30.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f379db40640 1 -- 172.21.15.170:0/2112169676 learned_addr learned my addr 172.21.15.170:0/2112169676 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f379db40640 1 -- 172.21.15.170:0/2112169676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3788005d30 con 0x56077f0bb240 2024-03-28T16:50:30.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f379db40640 1 --2- 172.21.15.170:0/2112169676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f231080 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3790001c90 tx=0x7f379000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee1b1e46b45bae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f368be07640 1 -- 172.21.15.170:0/2112169676 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f379000e690 con 0x56077f0bb240 2024-03-28T16:50:30.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f368be07640 1 -- 172.21.15.170:0/2112169676 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379000e7f0 con 0x56077f0bb240 2024-03-28T16:50:30.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f368be07640 1 -- 172.21.15.170:0/2112169676 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f379000eaf0 con 0x56077f0bb240 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.004+0000 7f37a0025980 1 -- 172.21.15.170:0/2112169676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 msgr2=0x56077f231080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 --2- 172.21.15.170:0/2112169676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f231080 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3790001c90 tx=0x7f379000b6d0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- 172.21.15.170:0/2112169676 shutdown_connections 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 --2- 172.21.15.170:0/2112169676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f231080 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- 172.21.15.170:0/2112169676 >> 172.21.15.170:0/2112169676 conn(0x56077f141770 msgr2=0x56077f22cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- 172.21.15.170:0/2112169676 shutdown_connections 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- 172.21.15.170:0/2112169676 wait complete. 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 Processor -- start 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- start start 2024-03-28T16:50:30.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f237ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077f052e00 con 0x56077f0bb240 2024-03-28T16:50:30.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379db40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f237ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379db40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f237ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37976/0 (socket says 172.21.15.170:37976) 2024-03-28T16:50:30.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379db40640 1 -- 172.21.15.170:0/639398173 learned_addr learned my addr 172.21.15.170:0/639398173 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379db40640 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56077f0934c0 con 0x56077f0bb240 2024-03-28T16:50:30.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379db40640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f237ed0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3790001ee0 tx=0x7f3790000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f368a604640 1 -- 172.21.15.170:0/639398173 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f379001f070 con 0x56077f0bb240 2024-03-28T16:50:30.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56077f238410 con 0x56077f0bb240 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f368a604640 1 -- 172.21.15.170:0/639398173 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379001a4d0 con 0x56077f0bb240 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56077f238840 con 0x56077f0bb240 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f368a604640 1 -- 172.21.15.170:0/639398173 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f379001a770 con 0x56077f0bb240 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 Processor -- start 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- start start 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f245110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f37a0025980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077efebe30 con 0x56077f244cf0 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379e341640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f245110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379e341640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f245110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37992/0 (socket says 172.21.15.170:37992) 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379e341640 1 -- 172.21.15.170:0/2119456115 learned_addr learned my addr 172.21.15.170:0/2119456115 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f368a604640 1 -- 172.21.15.170:0/639398173 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f379000a040 con 0x56077f0bb240 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f368a604640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f36840342b0 0x7f3684036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f368a604640 1 -- 172.21.15.170:0/639398173 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f3790049770 con 0x56077f0bb240 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379db40640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f36840342b0 0x7f3684036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.008+0000 7f379e341640 1 -- 172.21.15.170:0/2119456115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56077f245650 con 0x56077f244cf0 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f379e341640 1 --2- 172.21.15.170:0/2119456115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f245110 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f379800a430 tx=0x7f379800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3c019192269b48 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f379db40640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f36840342b0 0x7f3684036770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3790018fd0 tx=0x7f379001a9d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f3683fff640 1 -- 172.21.15.170:0/2119456115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f379800f6e0 con 0x56077f244cf0 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f3683fff640 1 -- 172.21.15.170:0/2119456115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800f860 con 0x56077f244cf0 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f3683fff640 1 -- 172.21.15.170:0/2119456115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f379800fb40 con 0x56077f244cf0 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- 172.21.15.170:0/2119456115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 msgr2=0x56077f245110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 --2- 172.21.15.170:0/2119456115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f245110 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f379800a430 tx=0x7f379800a8f0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- 172.21.15.170:0/2119456115 shutdown_connections 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 --2- 172.21.15.170:0/2119456115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f245110 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- 172.21.15.170:0/2119456115 >> 172.21.15.170:0/2119456115 conn(0x56077f14b370 msgr2=0x56077f240160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- 172.21.15.170:0/2119456115 shutdown_connections 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- 172.21.15.170:0/2119456115 wait complete. 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 Processor -- start 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- start start 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077efdf790 con 0x56077f244cf0 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f379e341640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f379e341640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:37994/0 (socket says 172.21.15.170:37994) 2024-03-28T16:50:30.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f379e341640 1 -- 172.21.15.170:0/2976396080 learned_addr learned my addr 172.21.15.170:0/2976396080 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f379e341640 1 -- 172.21.15.170:0/2976396080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379800a0e0 con 0x56077f244cf0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f379e341640 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f147510 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f379800a3d0 tx=0x7f3798003380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f36827fc640 1 -- 172.21.15.170:0/2976396080 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37980109a0 con 0x56077f244cf0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56077f147a50 con 0x56077f244cf0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56077f145ed0 con 0x56077f244cf0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f36827fc640 1 -- 172.21.15.170:0/2976396080 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3798010b00 con 0x56077f244cf0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.012+0000 7f36827fc640 1 -- 172.21.15.170:0/2976396080 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3798010e20 con 0x56077f244cf0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f36827fc640 1 -- 172.21.15.170:0/2976396080 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f379801a480 con 0x56077f244cf0 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:30.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f36827fc640 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3678034300 0x7f36780367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 1 Processor -- start 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f36827fc640 1 -- 172.21.15.170:0/2976396080 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f3798049d10 con 0x56077f244cf0 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f379e341640 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3678034300 0x7f36780367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 1 -- start start 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f267be0 0x56077f268000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56077f268540 con 0x56077f267be0 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f379e341640 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3678034300 0x7f36780367c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3798024b00 tx=0x7f3798024760 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f379db40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f267be0 0x56077f268000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f379db40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f267be0 0x56077f268000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38008/0 (socket says 172.21.15.170:38008) 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f379db40640 1 -- 172.21.15.170:0/1592612382 learned_addr learned my addr 172.21.15.170:0/1592612382 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f379db40640 1 -- 172.21.15.170:0/1592612382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56077f268680 con 0x56077f267be0 2024-03-28T16:50:30.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f379db40640 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f267be0 0x56077f268000 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3790023580 tx=0x7f3790017ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f3662fe5640 1 -- 172.21.15.170:0/1592612382 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3790047070 con 0x56077f267be0 2024-03-28T16:50:30.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56077f268fd0 con 0x56077f267be0 2024-03-28T16:50:30.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56077f268fd0 con 0x56077f267be0 2024-03-28T16:50:30.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f3662fe5640 1 -- 172.21.15.170:0/1592612382 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379003c070 con 0x56077f267be0 2024-03-28T16:50:30.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f3662fe5640 1 -- 172.21.15.170:0/1592612382 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3790049a80 con 0x56077f267be0 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f3662fe5640 1 -- 172.21.15.170:0/1592612382 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f3790049be0 con 0x56077f267be0 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f3662fe5640 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35d40342b0 0x7f35d4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.016+0000 7f3662fe5640 1 -- 172.21.15.170:0/1592612382 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f3790044bf0 con 0x56077f267be0 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379d33f640 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35d40342b0 0x7f35d4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f3660fe1640 20 reqs_thread_entry: start 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56077f26cbb0 0x56077f26efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379d33f640 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35d40342b0 0x7f35d4036770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f37880ad000 tx=0x7f37880a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56077f26f4e0 con 0x56077f26cbb0 2024-03-28T16:50:30.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379e341640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56077f26cbb0 0x56077f26efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379e341640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56077f26cbb0 0x56077f26efa0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379e341640 1 -- 172.21.15.170:0/639398173 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f3798016020 con 0x56077f26cbb0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56077f270d60 0x56077f273150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:2 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56077f273890 con 0x56077f270d60 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379db40640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56077f270d60 0x56077f273150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379db40640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56077f270d60 0x56077f273150 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379db40640 1 -- 172.21.15.170:0/639398173 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(2 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~874 out=874b] v0'0 uv4 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x7f379005a3b0 con 0x56077f270d60 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56077f26fce0 con 0x56077f26cbb0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379e341640 1 -- 172.21.15.170:0/639398173 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3798016020 con 0x56077f26cbb0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56077f270090 con 0x56077f26cbb0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379e341640 1 -- 172.21.15.170:0/639398173 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f3798016020 con 0x56077f26cbb0 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56077f2751a0 0x56077f277590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x56077f277ad0 con 0x56077f2751a0 2024-03-28T16:50:30.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.020+0000 7f379d33f640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56077f2751a0 0x56077f277590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.024+0000 7f379d33f640 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56077f2751a0 0x56077f277590 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.024+0000 7f379d33f640 1 -- 172.21.15.170:0/639398173 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~421 out=421b] v0'0 uv2 ondisk = 0) v8 ==== 237+0+469 (crc 0 0 0) 0x7f37880abb60 con 0x56077f2751a0 2024-03-28T16:50:30.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.024+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x56077efebc20 con 0x56077f244cf0 2024-03-28T16:50:30.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.024+0000 7f36827fc640 1 -- 172.21.15.170:0/2976396080 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3798019de0 con 0x56077f244cf0 2024-03-28T16:50:30.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.024+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:6 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~921 in=921b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x56077f27c600 con 0x56077f270d60 2024-03-28T16:50:30.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.024+0000 7f379db40640 1 -- 172.21.15.170:0/639398173 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(6 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [stat,call,call,writefull 0~921] v31'5 uv5 ondisk = 0) v8 ==== 316+0+0 (crc 0 0 0) 0x7f379005a3b0 con 0x56077f270d60 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3678034300 msgr2=0x7f36780367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3678034300 0x7f36780367c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3798024b00 tx=0x7f3798024760 comp rx=0 tx=0).stop 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 msgr2=0x56077f147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f147510 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f379800a3d0 tx=0x7f3798003380 comp rx=0 tx=0).stop 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 shutdown_connections 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f3678034300 0x7f36780367c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/2976396080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f244cf0 0x56077f147510 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 >> 172.21.15.170:0/2976396080 conn(0x56077f14b370 msgr2=0x56077f23ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 shutdown_connections 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/2976396080 wait complete. 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35d40342b0 msgr2=0x7f35d4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35d40342b0 0x7f35d4036770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f37880ad000 tx=0x7f37880a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:30.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f267be0 msgr2=0x56077f268000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f267be0 0x56077f268000 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3790023580 tx=0x7f3790017ce0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 shutdown_connections 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f35d40342b0 0x7f35d4036770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/1592612382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f267be0 0x56077f268000 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 >> 172.21.15.170:0/1592612382 conn(0x56077f25c900 msgr2=0x56077f25d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 shutdown_connections 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/1592612382 wait complete. 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56077f270d60 msgr2=0x56077f273150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x56077f270d60 0x56077f273150 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56077f26cbb0 msgr2=0x56077f26efa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x56077f26cbb0 0x56077f26efa0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56077f2751a0 msgr2=0x56077f277590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x56077f2751a0 0x56077f277590 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f36840342b0 msgr2=0x7f3684036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f36840342b0 0x7f3684036770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3790018fd0 tx=0x7f379001a9d0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 msgr2=0x56077f237ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 --2- 172.21.15.170:0/639398173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56077f0bb240 0x56077f237ed0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3790001ee0 tx=0x7f3790000ee0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f379e341640 1 -- 172.21.15.170:0/639398173 reap_dead start 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 shutdown_connections 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 >> 172.21.15.170:0/639398173 conn(0x56077f141770 msgr2=0x56077f22b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 shutdown_connections 2024-03-28T16:50:30.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.028+0000 7f37a0025980 1 -- 172.21.15.170:0/639398173 wait complete. 2024-03-28T16:50:30.038 INFO:teuthology.orchestra.run.smithi170.stdout:{"id":"b88c2392-6ef8-4660-b080-19480b748fc0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"GLACIER":{"data_pool":".rgw.buckets.2"},"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":"","notif_pool":"default.rgw.log:notif"} 2024-03-28T16:50:30.038 DEBUG:tasks.util.rgw: json result: {'id': 'b88c2392-6ef8-4660-b080-19480b748fc0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'GLACIER': {'data_pool': '.rgw.buckets.2'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-03-28T16:50:30.038 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-03-28T16:50:30.038 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-03-28T16:50:30.039 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-03-28T16:50:30.122 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:30.122 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46ffca0980 1 Processor -- start 2024-03-28T16:50:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46ffca0980 1 -- start start 2024-03-28T16:50:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46ffca0980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55aab5af06d0 0x55aab5b83fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46ffca0980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55aab5ac74c0 con 0x55aab5af19c0 2024-03-28T16:50:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46ffca0980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x55aab5a8e0f0 con 0x55aab5af06d0 2024-03-28T16:50:30.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46fcfba640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x55aab5af06d0 0x55aab5b83fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46fd7bb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x55aab5af19c0 0x55aab5aeffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:34000/0 (socket says 172.21.15.170:34000) 2024-03-28T16:50:30.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46fd7bb640 1 -- 172.21.15.170:0/3267932600 learned_addr learned my addr 172.21.15.170:0/3267932600 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:30.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46fcfba640 1 -- 172.21.15.170:0/3267932600 >> v1:172.21.15.119:6789/0 conn(0x55aab5af19c0 legacy=0x55aab5aeffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46fcfba640 1 -- 172.21.15.170:0/3267932600 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab5af0ae0 con 0x55aab5af06d0 2024-03-28T16:50:30.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.124+0000 7f46fcfba640 1 --2- 172.21.15.170:0/3267932600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5b83fd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f46e800a480 tx=0x7f46e801b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a712977dcee7bb57 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46f7fff640 1 -- 172.21.15.170:0/3267932600 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e80aa530 con 0x55aab5af06d0 2024-03-28T16:50:30.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46f7fff640 1 -- 172.21.15.170:0/3267932600 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e80a6050 con 0x55aab5af06d0 2024-03-28T16:50:30.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46f7fff640 1 -- 172.21.15.170:0/3267932600 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46e80aa7e0 con 0x55aab5af06d0 2024-03-28T16:50:30.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46ffca0980 1 -- 172.21.15.170:0/3267932600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 msgr2=0x55aab5b83fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46ffca0980 1 --2- 172.21.15.170:0/3267932600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5b83fd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f46e800a480 tx=0x7f46e801b280 comp rx=0 tx=0).stop 2024-03-28T16:50:30.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46ffca0980 1 -- 172.21.15.170:0/3267932600 shutdown_connections 2024-03-28T16:50:30.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46ffca0980 1 --2- 172.21.15.170:0/3267932600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5b83fd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46ffca0980 1 -- 172.21.15.170:0/3267932600 >> 172.21.15.170:0/3267932600 conn(0x55aab5b76890 msgr2=0x55aab5b78cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46ffca0980 1 -- 172.21.15.170:0/3267932600 shutdown_connections 2024-03-28T16:50:30.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.128+0000 7f46ffca0980 1 -- 172.21.15.170:0/3267932600 wait complete. 2024-03-28T16:50:30.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.140+0000 7f46ffca0980 1 Processor -- start 2024-03-28T16:50:30.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.140+0000 7f46ffca0980 1 -- start start 2024-03-28T16:50:30.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.140+0000 7f46ffca0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.140+0000 7f46ffca0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5a8b620 con 0x55aab5af06d0 2024-03-28T16:50:30.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c673f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38018/0 (socket says 172.21.15.170:38018) 2024-03-28T16:50:30.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 -- 172.21.15.170:0/1616766837 learned_addr learned my addr 172.21.15.170:0/1616766837 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 -- 172.21.15.170:0/1616766837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8005d30 con 0x55aab5af06d0 2024-03-28T16:50:30.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/1616766837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c673f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f46f0006fb0 tx=0x7f46f0001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1a5aa079feb573 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f45eb606640 1 -- 172.21.15.170:0/1616766837 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0004530 con 0x55aab5af06d0 2024-03-28T16:50:30.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f45eb606640 1 -- 172.21.15.170:0/1616766837 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f0004690 con 0x55aab5af06d0 2024-03-28T16:50:30.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f45eb606640 1 -- 172.21.15.170:0/1616766837 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f00049b0 con 0x55aab5af06d0 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 -- 172.21.15.170:0/1616766837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 msgr2=0x55aab5c673f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 --2- 172.21.15.170:0/1616766837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c673f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f46f0006fb0 tx=0x7f46f0001940 comp rx=0 tx=0).stop 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 -- 172.21.15.170:0/1616766837 shutdown_connections 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 --2- 172.21.15.170:0/1616766837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c673f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 -- 172.21.15.170:0/1616766837 >> 172.21.15.170:0/1616766837 conn(0x55aab5b76890 msgr2=0x55aab5c62f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 -- 172.21.15.170:0/1616766837 shutdown_connections 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 -- 172.21.15.170:0/1616766837 wait complete. 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 Processor -- start 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 -- start start 2024-03-28T16:50:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c6e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46ffca0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5a86e00 con 0x55aab5af06d0 2024-03-28T16:50:30.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c6e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c6e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38024/0 (socket says 172.21.15.170:38024) 2024-03-28T16:50:30.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 -- 172.21.15.170:0/2592346188 learned_addr learned my addr 172.21.15.170:0/2592346188 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.144+0000 7f46fd7bb640 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab5ac74c0 con 0x55aab5af06d0 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c6e240 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x55aab5c68280 tx=0x7f46f0000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f45e9e03640 1 -- 172.21.15.170:0/2592346188 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0013760 con 0x55aab5af06d0 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aab5c6e780 con 0x55aab5af06d0 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f45e9e03640 1 -- 172.21.15.170:0/2592346188 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f00138c0 con 0x55aab5af06d0 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aab5c6ebb0 con 0x55aab5af06d0 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f45e9e03640 1 -- 172.21.15.170:0/2592346188 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0013b60 con 0x55aab5af06d0 2024-03-28T16:50:30.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f45e9e03640 1 -- 172.21.15.170:0/2592346188 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f46f0013cc0 con 0x55aab5af06d0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f45e9e03640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45e4034300 0x7f45e40367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f45e9e03640 1 -- 172.21.15.170:0/2592346188 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f46f0049560 con 0x55aab5af06d0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fcfba640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45e4034300 0x7f45e40367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5af19c0 0x55aab5c74ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5c75400 con 0x55aab5af19c0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fdfbc640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5af19c0 0x55aab5c74ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fcfba640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45e4034300 0x7f45e40367c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46e80ac000 tx=0x7f46e80a5000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fdfbc640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5af19c0 0x55aab5c74ec0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fdfbc640 1 -- 172.21.15.170:0/2592346188 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5af19c0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5c75760 con 0x55aab5af19c0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fdfbc640 1 -- 172.21.15.170:0/2592346188 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5af19c0 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5c77480 0x55aab5c79870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5c79db0 con 0x55aab5c77480 2024-03-28T16:50:30.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.148+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5c77480 0x55aab5c79870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5c77480 0x55aab5c79870 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fd7bb640 1 -- 172.21.15.170:0/2592346188 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~921 out=921b] v0'0 uv5 ondisk = 0) v8 ==== 232+0+969 (crc 0 0 0) 0x7f46f0024020 con 0x55aab5c77480 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5c763b0 con 0x55aab5af19c0 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fdfbc640 1 -- 172.21.15.170:0/2592346188 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5af19c0 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46f0048000 0x55aab5c7d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5c7d920 con 0x7f46f0048000 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fcfba640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46f0048000 0x55aab5c7d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fcfba640 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46f0048000 0x55aab5c7d1c0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fcfba640 1 -- 172.21.15.170:0/2592346188 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~421 out=421b] v0'0 uv2 ondisk = 0) v8 ==== 237+0+469 (crc 0 0 0) 0x7f46e80aab40 con 0x7f46f0048000 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46ffca0980 1 Processor -- start 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46ffca0980 1 -- start start 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46ffca0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c8b8b0 0x55aab5c8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46ffca0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5a1fe30 con 0x55aab5c8b8b0 2024-03-28T16:50:30.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c8b8b0 0x55aab5c8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c8b8b0 0x55aab5c8bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38028/0 (socket says 172.21.15.170:38028) 2024-03-28T16:50:30.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fd7bb640 1 -- 172.21.15.170:0/99496506 learned_addr learned my addr 172.21.15.170:0/99496506 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.152+0000 7f46fd7bb640 1 -- 172.21.15.170:0/99496506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab5c8c210 con 0x55aab5c8b8b0 2024-03-28T16:50:30.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c8b8b0 0x55aab5c8bcd0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f46f0029fd0 tx=0x7f46f001df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e2ffd640 1 -- 172.21.15.170:0/99496506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0039070 con 0x55aab5c8b8b0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aab5c8c460 con 0x55aab5c8b8b0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e2ffd640 1 -- 172.21.15.170:0/99496506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f001e560 con 0x55aab5c8b8b0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aab5c8cfb0 con 0x55aab5c8b8b0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e2ffd640 1 -- 172.21.15.170:0/99496506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0047dc0 con 0x55aab5c8b8b0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e2ffd640 1 -- 172.21.15.170:0/99496506 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f46f0012a80 con 0x55aab5c8b8b0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e2ffd640 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45d40342b0 0x7f45d4036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e2ffd640 1 -- 172.21.15.170:0/99496506 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f46f00150c0 con 0x55aab5c8b8b0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fcfba640 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45d40342b0 0x7f45d4036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46ffca0980 1 Processor -- start 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46ffca0980 1 -- start start 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fcfba640 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45d40342b0 0x7f45d4036770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55aab5c76710 tx=0x7f46e8005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46ffca0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c98a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46ffca0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5a13790 con 0x55aab5c98660 2024-03-28T16:50:30.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c98a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c98a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38034/0 (socket says 172.21.15.170:38034) 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fd7bb640 1 -- 172.21.15.170:0/3086320329 learned_addr learned my addr 172.21.15.170:0/3086320329 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fd7bb640 1 -- 172.21.15.170:0/3086320329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab5c99050 con 0x55aab5c98660 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/3086320329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c98a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f46f0006340 tx=0x7f46f0045f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39219a5f35b63c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e0ff9640 1 -- 172.21.15.170:0/3086320329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0028460 con 0x55aab5c98660 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e0ff9640 1 -- 172.21.15.170:0/3086320329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f0044db0 con 0x55aab5c98660 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.156+0000 7f45e0ff9640 1 -- 172.21.15.170:0/3086320329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0012e60 con 0x55aab5c98660 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- 172.21.15.170:0/3086320329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 msgr2=0x55aab5c98a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 --2- 172.21.15.170:0/3086320329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c98a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f46f0006340 tx=0x7f46f0045f70 comp rx=0 tx=0).stop 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- 172.21.15.170:0/3086320329 shutdown_connections 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 --2- 172.21.15.170:0/3086320329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c98a80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- 172.21.15.170:0/3086320329 >> 172.21.15.170:0/3086320329 conn(0x55aab5c949f0 msgr2=0x55aab5c92a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- 172.21.15.170:0/3086320329 shutdown_connections 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- 172.21.15.170:0/3086320329 wait complete. 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 Processor -- start 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- start start 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c9f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5c9fab0 con 0x55aab5c98660 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c9f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46fd7bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c9f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38042/0 (socket says 172.21.15.170:38042) 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 learned_addr learned my addr 172.21.15.170:0/648301044 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab5c8c210 con 0x55aab5c98660 2024-03-28T16:50:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c9f570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f46f0006340 tx=0x7f46f0027ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f45d2ffd640 1 -- 172.21.15.170:0/648301044 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0006fe0 con 0x55aab5c98660 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aab5c8c210 con 0x55aab5c98660 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aab5c8c210 con 0x55aab5c98660 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f45d2ffd640 1 -- 172.21.15.170:0/648301044 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f0007140 con 0x55aab5c98660 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f45d2ffd640 1 -- 172.21.15.170:0/648301044 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46f0027830 con 0x55aab5c98660 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f45d2ffd640 1 -- 172.21.15.170:0/648301044 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f46f005a070 con 0x55aab5c98660 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f45d2ffd640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45c80342b0 0x7f45c8036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f45d2ffd640 1 -- 172.21.15.170:0/648301044 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f46f000c150 con 0x55aab5c98660 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.160+0000 7f46fcfba640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45c80342b0 0x7f45c8036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5caa730 0x55aab5cacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cad340 con 0x55aab5caa730 2024-03-28T16:50:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fdfbc640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5caa730 0x55aab5cacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fcfba640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45c80342b0 0x7f45c8036770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f46e801b2c0 tx=0x7f46e80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fdfbc640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5caa730 0x55aab5cacbf0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 realm 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cad710 con 0x55aab5caa730 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cae2f0 con 0x55aab5caa730 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5caeca0 0x55aab5cb1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb16a0 con 0x55aab5caeca0 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5caeca0 0x55aab5cb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fd7bb640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5caeca0 0x55aab5cb1090 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~921 out=921b] v0'0 uv5 ondisk = 0) v8 ==== 190+0+921 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 rados_obj.operate() r=0 bl.length=921 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 20 searching for the correct realm 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55aab5cb3990 0x55aab5cb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.164+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb6520 con 0x55aab5cb3990 2024-03-28T16:50:30.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fcfba640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55aab5cb3990 0x55aab5cb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fcfba640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55aab5cb3990 0x55aab5cb5e50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fcfba640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46e80da040 0x7f46e80cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb6520 con 0x7f46e80da040 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46e80da040 0x7f46e80cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46e80da040 0x7f46e80cb940 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.168+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x7f46e80da040 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x7f46e80da040 2024-03-28T16:50:30.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.172+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v31'2 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x7f46e80da040 2024-03-28T16:50:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.176+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x7f46e80da040 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5cb3990 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caeca0 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v31'5 uv5 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x55aab5caa730 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v31'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cadf90 con 0x7f46e80da040 2024-03-28T16:50:30.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb1d90 con 0x55aab5caa730 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb1d90 con 0x55aab5caa730 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f46f80083f0 con 0x55aab5caa730 2024-03-28T16:50:30.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:30.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb1d40 con 0x7f46e80da040 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~421 out=421b] v0'0 uv2 ondisk = 0) v8 ==== 195+0+421 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 rados_obj.operate() r=0 bl.length=421 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 zone default found 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 4 Realm: () 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 zonegroup default 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb25e0 con 0x55aab5caeca0 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.180+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb8340 con 0x55aab5cb3990 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb86a0 con 0x55aab5cb3990 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 20 started sync module instance, tier type = 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb7c20 con 0x7f46e80da040 2024-03-28T16:50:30.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f80083f0 con 0x7f46e80da040 2024-03-28T16:50:30.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94191682811776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb7fd0 con 0x7f46e80da040 2024-03-28T16:50:30.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb7c20 con 0x55aab5caa730 2024-03-28T16:50:30.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.184+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94191682811776] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x7f46e80da040 2024-03-28T16:50:30.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.188+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.188+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94191682816512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb9390 con 0x55aab5caa730 2024-03-28T16:50:30.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.188+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb9d00 con 0x55aab5cb3990 2024-03-28T16:50:30.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.188+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94191682816512] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.188+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.188+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94191682820192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cba060 con 0x55aab5cb3990 2024-03-28T16:50:30.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.188+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cba9d0 con 0x7f46e80da040 2024-03-28T16:50:30.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x7f46e80da040 2024-03-28T16:50:30.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94191682823744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbb0c0 con 0x7f46e80da040 2024-03-28T16:50:30.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbbb90 con 0x55aab5caa730 2024-03-28T16:50:30.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94191682820192] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94191682823744] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x7f46e80da040 2024-03-28T16:50:30.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94191682827888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbbb50 con 0x55aab5caa730 2024-03-28T16:50:30.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.192+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbac90 con 0x55aab5caeca0 2024-03-28T16:50:30.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.196+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94191682827888] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.196+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.196+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94191682817568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbc3e0 con 0x55aab5caeca0 2024-03-28T16:50:30.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.196+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbcbe0 con 0x55aab5caeca0 2024-03-28T16:50:30.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.196+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94191682817568] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94191682830144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbd3e0 con 0x55aab5caeca0 2024-03-28T16:50:30.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbdd50 con 0x55aab5caeca0 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94191682836656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbe1d0 con 0x55aab5caeca0 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 20 add_watcher() i=0 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 20 add_watcher() i=1 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 20 add_watcher() i=2 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 20 add_watcher() i=3 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 20 add_watcher() i=4 2024-03-28T16:50:30.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 20 add_watcher() i=5 2024-03-28T16:50:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94191682830144] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.200+0000 7f46ffca0980 20 add_watcher() i=6 2024-03-28T16:50:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94191682836656] v31'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 20 add_watcher() i=7 2024-03-28T16:50:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbd050 con 0x55aab5caa730 2024-03-28T16:50:30.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94191682837712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbeb10 con 0x55aab5caa730 2024-03-28T16:50:30.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94191682837712] v31'239 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f45b37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:30.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:30.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:30.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 5 note: GC not initialized 2024-03-28T16:50:30.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f45b27e4640 20 reqs_thread_entry: start 2024-03-28T16:50:30.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:30.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 20 Filter name: none 2024-03-28T16:50:30.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f4523fff640 20 reqs_thread_entry: start 2024-03-28T16:50:30.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-03-28T16:50:30.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5ce32d0 con 0x55aab5caeca0 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.204+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5ce32d0 con 0x55aab5caeca0 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-03-28T16:50:30.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:50:30.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:50:30.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-03-28T16:50:30.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5ce1f90 con 0x55aab5caeca0 2024-03-28T16:50:30.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v31'8 uv8 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-03-28T16:50:30.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:50:30.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-03-28T16:50:30.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94191682986832 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5ce1fd0 con 0x55aab5caeca0 2024-03-28T16:50:30.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== watch-notify(notify (1) cookie 94191682817568 notify 133143986227 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55aab5c99050 con 0x55aab5caeca0 2024-03-28T16:50:30.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f45e0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986227 cookie 94191682817568 notifier 4837 bl.length()=522 2024-03-28T16:50:30.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f45e0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-03-28T16:50:30.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f45e0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-03-28T16:50:30.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(65 notify.5 [notify cookie 94191682986832 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f46f006f070 con 0x55aab5caeca0 2024-03-28T16:50:30.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.208+0000 7f45e0ff9640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f45c0003ea0 con 0x55aab5caeca0 2024-03-28T16:50:30.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== watch-notify(notify_complete (2) cookie 94191682986832 notify 133143986227 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f46f008b020 con 0x55aab5caeca0 2024-03-28T16:50:30.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f008d040 con 0x55aab5caeca0 2024-03-28T16:50:30.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5ce1f40 con 0x55aab5caeca0 2024-03-28T16:50:30.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v31'11 uv11 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f46f008d040 con 0x55aab5caeca0 2024-03-28T16:50:30.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46ffca0980 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46ffca0980 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46ffca0980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94191682984608 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5ce1ec0 con 0x55aab5cb3990 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== watch-notify(notify (1) cookie 94191682820192 notify 133143986192 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x7f46e80d3070 con 0x55aab5cb3990 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f45d3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 94191682820192 notifier 4837 bl.length()=177 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f45d3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f45d3fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(68 notify.2 [notify cookie 94191682984608 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f45d3fff640 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f4530003400 con 0x55aab5cb3990 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.212+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== watch-notify(notify_complete (2) cookie 94191682984608 notify 133143986192 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55aab5af0ae0 con 0x55aab5cb3990 2024-03-28T16:50:30.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.216 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:50:30.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f45b37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:30.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94191682837712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5ce49f0 con 0x55aab5caa730 2024-03-28T16:50:30.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94191682837712] v31'240 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94191682811776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbdcd0 con 0x7f46e80da040 2024-03-28T16:50:30.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94191682811776] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x7f46e80da040 2024-03-28T16:50:30.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46ffca0980 20 remove_watcher() i=0 2024-03-28T16:50:30.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46ffca0980 2 removed watcher, disabling cache 2024-03-28T16:50:30.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.216+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94191682816512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbdcd0 con 0x55aab5caa730 2024-03-28T16:50:30.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.220+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94191682816512] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.220+0000 7f46ffca0980 20 remove_watcher() i=1 2024-03-28T16:50:30.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.220+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94191682820192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb8a00 con 0x55aab5cb3990 2024-03-28T16:50:30.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.220+0000 7f46fcfba640 1 -- 172.21.15.170:0/648301044 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94191682820192] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46e80aab40 con 0x55aab5cb3990 2024-03-28T16:50:30.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.220+0000 7f46ffca0980 20 remove_watcher() i=2 2024-03-28T16:50:30.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.220+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94191682823744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbdcd0 con 0x7f46e80da040 2024-03-28T16:50:30.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.224+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94191682823744] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x7f46e80da040 2024-03-28T16:50:30.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.224+0000 7f46ffca0980 20 remove_watcher() i=3 2024-03-28T16:50:30.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.224+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94191682827888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cb8a00 con 0x55aab5caa730 2024-03-28T16:50:30.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.224+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94191682827888] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f803e040 con 0x55aab5caa730 2024-03-28T16:50:30.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.224+0000 7f46ffca0980 20 remove_watcher() i=4 2024-03-28T16:50:30.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.224+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94191682817568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cba640 con 0x55aab5caeca0 2024-03-28T16:50:30.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.228+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94191682817568] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f008d040 con 0x55aab5caeca0 2024-03-28T16:50:30.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.228+0000 7f46ffca0980 20 remove_watcher() i=5 2024-03-28T16:50:30.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.228+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94191682830144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cbdcd0 con 0x55aab5caeca0 2024-03-28T16:50:30.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.228+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94191682830144] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f008d040 con 0x55aab5caeca0 2024-03-28T16:50:30.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.228+0000 7f46ffca0980 20 remove_watcher() i=6 2024-03-28T16:50:30.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.228+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94191682836656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x55aab5cba640 con 0x55aab5caeca0 2024-03-28T16:50:30.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46fd7bb640 1 -- 172.21.15.170:0/648301044 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94191682836656] v31'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f46f008d040 con 0x55aab5caeca0 2024-03-28T16:50:30.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 20 remove_watcher() i=7 2024-03-28T16:50:30.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5caeca0 msgr2=0x55aab5cb1090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5caeca0 0x55aab5cb1090 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5caa730 msgr2=0x55aab5cacbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5caa730 0x55aab5cacbf0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46e80da040 msgr2=0x7f46e80cb940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46e80da040 0x7f46e80cb940 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55aab5cb3990 msgr2=0x55aab5cb5e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x55aab5cb3990 0x55aab5cb5e50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45c80342b0 msgr2=0x7f45c8036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45c80342b0 0x7f45c8036770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f46e801b2c0 tx=0x7f46e80a7000 comp rx=0 tx=0).stop 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 msgr2=0x55aab5c9f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46fdfbc640 1 -- 172.21.15.170:0/648301044 reap_dead start 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c9f570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f46f0006340 tx=0x7f46f0027ec0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 shutdown_connections 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/648301044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c98660 0x55aab5c9f570 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 >> 172.21.15.170:0/648301044 conn(0x55aab5c949f0 msgr2=0x55aab5c92050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 shutdown_connections 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/648301044 wait complete. 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45d40342b0 msgr2=0x7f45d4036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45d40342b0 0x7f45d4036770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55aab5c76710 tx=0x7f46e8005a10 comp rx=0 tx=0).stop 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c8b8b0 msgr2=0x55aab5c8bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c8b8b0 0x55aab5c8bcd0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f46f0029fd0 tx=0x7f46f001df70 comp rx=0 tx=0).stop 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 shutdown_connections 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45d40342b0 0x7f45d4036770 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/99496506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5c8b8b0 0x55aab5c8bcd0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 >> 172.21.15.170:0/99496506 conn(0x55aab5c806c0 msgr2=0x55aab5c811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 shutdown_connections 2024-03-28T16:50:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/99496506 wait complete. 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5c77480 msgr2=0x55aab5c79870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x55aab5c77480 0x55aab5c79870 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5af19c0 msgr2=0x55aab5c74ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x55aab5af19c0 0x55aab5c74ec0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46f0048000 msgr2=0x55aab5c7d1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f46f0048000 0x55aab5c7d1c0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45e4034300 msgr2=0x7f45e40367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f45e4034300 0x7f45e40367c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f46e80ac000 tx=0x7f46e80a5000 comp rx=0 tx=0).stop 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 msgr2=0x55aab5c6e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 --2- 172.21.15.170:0/2592346188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aab5af06d0 0x55aab5c6e240 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x55aab5c68280 tx=0x7f46f0000f50 comp rx=0 tx=0).stop 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46fdfbc640 1 -- 172.21.15.170:0/2592346188 reap_dead start 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 shutdown_connections 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 >> 172.21.15.170:0/2592346188 conn(0x55aab5b76890 msgr2=0x55aab5c61a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 shutdown_connections 2024-03-28T16:50:30.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.232+0000 7f46ffca0980 1 -- 172.21.15.170:0/2592346188 wait complete. 2024-03-28T16:50:30.244 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:50:30.244 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-03-28T16:50:30.244 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-03-28T16:50:30.244 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-03-28T16:50:30.331 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:30.331 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:30.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d577de980 1 Processor -- start 2024-03-28T16:50:30.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d577de980 1 -- start start 2024-03-28T16:50:30.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d577de980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5624d3f8a480 0x5624d3f8c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d577de980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5624d3ed24c0 con 0x5624d3f85fc0 2024-03-28T16:50:30.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d577de980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x5624d3e990f0 con 0x5624d3f8a480 2024-03-28T16:50:30.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d54af8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x5624d3f8a480 0x5624d3f8c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d552f9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x5624d3f85fc0 0x5624d3f863a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.170:34006/0 (socket says 172.21.15.170:34006) 2024-03-28T16:50:30.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d552f9640 1 -- 172.21.15.170:0/688250171 learned_addr learned my addr 172.21.15.170:0/688250171 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-03-28T16:50:30.334 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d54af8640 1 -- 172.21.15.170:0/688250171 >> v1:172.21.15.119:6789/0 conn(0x5624d3f85fc0 legacy=0x5624d3f863a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d54af8640 1 -- 172.21.15.170:0/688250171 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624d3f8ddc0 con 0x5624d3f8a480 2024-03-28T16:50:30.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.332+0000 7f8d47fff640 1 -- 172.21.15.170:0/688250171 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869675236 0 0) 0x5624d3ed24c0 con 0x5624d3f85fc0 2024-03-28T16:50:30.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d54af8640 1 --2- 172.21.15.170:0/688250171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f8a480 0x5624d3f8c870 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00a480 tx=0x7f8d4c01b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98dddbf5b7e7717 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d47fff640 1 -- 172.21.15.170:0/688250171 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d4c0ab540 con 0x5624d3f8a480 2024-03-28T16:50:30.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d47fff640 1 -- 172.21.15.170:0/688250171 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4c0ab6a0 con 0x5624d3f8a480 2024-03-28T16:50:30.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d47fff640 1 -- 172.21.15.170:0/688250171 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d4c0ab960 con 0x5624d3f8a480 2024-03-28T16:50:30.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d577de980 1 -- 172.21.15.170:0/688250171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f8a480 msgr2=0x5624d3f8c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d577de980 1 --2- 172.21.15.170:0/688250171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f8a480 0x5624d3f8c870 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00a480 tx=0x7f8d4c01b280 comp rx=0 tx=0).stop 2024-03-28T16:50:30.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d577de980 1 -- 172.21.15.170:0/688250171 shutdown_connections 2024-03-28T16:50:30.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d577de980 1 --2- 172.21.15.170:0/688250171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f8a480 0x5624d3f8c870 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d577de980 1 -- 172.21.15.170:0/688250171 >> 172.21.15.170:0/688250171 conn(0x5624d3f81890 msgr2=0x5624d3f83cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.337 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d577de980 1 -- 172.21.15.170:0/688250171 shutdown_connections 2024-03-28T16:50:30.338 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.336+0000 7f8d577de980 1 -- 172.21.15.170:0/688250171 wait complete. 2024-03-28T16:50:30.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d577de980 1 Processor -- start 2024-03-28T16:50:30.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d577de980 1 -- start start 2024-03-28T16:50:30.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d577de980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.352 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d577de980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624d3e96620 con 0x5624d3f85fc0 2024-03-28T16:50:30.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d406fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38074/0 (socket says 172.21.15.170:38074) 2024-03-28T16:50:30.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d552f9640 1 -- 172.21.15.170:0/926027387 learned_addr learned my addr 172.21.15.170:0/926027387 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d552f9640 1 -- 172.21.15.170:0/926027387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c005d30 con 0x5624d3f85fc0 2024-03-28T16:50:30.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8d552f9640 1 --2- 172.21.15.170:0/926027387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d406fcd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8d480016c0 tx=0x7f8d48001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc30573cc2a08eec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8c43606640 1 -- 172.21.15.170:0/926027387 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48003460 con 0x5624d3f85fc0 2024-03-28T16:50:30.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8c43606640 1 -- 172.21.15.170:0/926027387 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480035c0 con 0x5624d3f85fc0 2024-03-28T16:50:30.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.352+0000 7f8c43606640 1 -- 172.21.15.170:0/926027387 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d480038a0 con 0x5624d3f85fc0 2024-03-28T16:50:30.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- 172.21.15.170:0/926027387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 msgr2=0x5624d406fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 --2- 172.21.15.170:0/926027387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d406fcd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8d480016c0 tx=0x7f8d48001b80 comp rx=0 tx=0).stop 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- 172.21.15.170:0/926027387 shutdown_connections 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 --2- 172.21.15.170:0/926027387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d406fcd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- 172.21.15.170:0/926027387 >> 172.21.15.170:0/926027387 conn(0x5624d3f81890 msgr2=0x5624d406b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- 172.21.15.170:0/926027387 shutdown_connections 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- 172.21.15.170:0/926027387 wait complete. 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 Processor -- start 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- start start 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d4076bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624d3e91e00 con 0x5624d3f85fc0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d4076bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d4076bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38084/0 (socket says 172.21.15.170:38084) 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d552f9640 1 -- 172.21.15.170:0/724846843 learned_addr learned my addr 172.21.15.170:0/724846843 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d552f9640 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48006cf0 con 0x5624d3f85fc0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d552f9640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d4076bb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8d480012f0 tx=0x7f8d48001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8c41e03640 1 -- 172.21.15.170:0/724846843 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48013630 con 0x5624d3f85fc0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8c41e03640 1 -- 172.21.15.170:0/724846843 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48013790 con 0x5624d3f85fc0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8c41e03640 1 -- 172.21.15.170:0/724846843 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48013a30 con 0x5624d3f85fc0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5624d40770f0 con 0x5624d3f85fc0 2024-03-28T16:50:30.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.356+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624d4077520 con 0x5624d3f85fc0 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8c41e03640 1 -- 172.21.15.170:0/724846843 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8d48013b90 con 0x5624d3f85fc0 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8c41e03640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c3c0342b0 0x7f8c3c036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8c41e03640 1 -- 172.21.15.170:0/724846843 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8d480495e0 con 0x5624d3f85fc0 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d54af8640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c3c0342b0 0x7f8c3c036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d3f8a480 0x5624d407d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d407dd20 con 0x5624d3f8a480 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d55afa640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d3f8a480 0x5624d407d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d54af8640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c3c0342b0 0x7f8c3c036770 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0ad000 tx=0x7f8d4c0a5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d55afa640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d3f8a480 0x5624d407d7e0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d55afa640 1 -- 172.21.15.170:0/724846843 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d3f8a480 2024-03-28T16:50:30.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d407e080 con 0x5624d3f8a480 2024-03-28T16:50:30.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d55afa640 1 -- 172.21.15.170:0/724846843 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d3f8a480 2024-03-28T16:50:30.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d407fdf0 0x5624d40821e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d4082720 con 0x5624d407fdf0 2024-03-28T16:50:30.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d552f9640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d407fdf0 0x5624d40821e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d552f9640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d407fdf0 0x5624d40821e0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d552f9640 1 -- 172.21.15.170:0/724846843 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~921 out=921b] v0'0 uv5 ondisk = 0) v8 ==== 232+0+969 (crc 0 0 0) 0x7f8d48024020 con 0x5624d407fdf0 2024-03-28T16:50:30.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.360+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d407ed20 con 0x5624d3f8a480 2024-03-28T16:50:30.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d55afa640 1 -- 172.21.15.170:0/724846843 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d3f8a480 2024-03-28T16:50:30.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d48044000 0x5624d4085b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d4086270 con 0x7f8d48044000 2024-03-28T16:50:30.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d54af8640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d48044000 0x5624d4085b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d54af8640 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d48044000 0x5624d4085b30 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d54af8640 1 -- 172.21.15.170:0/724846843 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~421 out=421b] v0'0 uv2 ondisk = 0) v8 ==== 237+0+469 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x7f8d48044000 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 Processor -- start 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 -- start start 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40941e0 0x5624d4094600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624d3e2ae30 con 0x5624d40941e0 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40941e0 0x5624d4094600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40941e0 0x5624d4094600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38088/0 (socket says 172.21.15.170:38088) 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d552f9640 1 -- 172.21.15.170:0/2319822859 learned_addr learned my addr 172.21.15.170:0/2319822859 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d552f9640 1 -- 172.21.15.170:0/2319822859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624d4094b40 con 0x5624d40941e0 2024-03-28T16:50:30.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d552f9640 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40941e0 0x5624d4094600 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8d48047ae0 tx=0x7f8d48011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8c3affd640 1 -- 172.21.15.170:0/2319822859 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d4803a070 con 0x5624d40941e0 2024-03-28T16:50:30.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5624d4094d90 con 0x5624d40941e0 2024-03-28T16:50:30.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8c3affd640 1 -- 172.21.15.170:0/2319822859 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48041ab0 con 0x5624d40941e0 2024-03-28T16:50:30.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8c3affd640 1 -- 172.21.15.170:0/2319822859 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48041d90 con 0x5624d40941e0 2024-03-28T16:50:30.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.364+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624d40958e0 con 0x5624d40941e0 2024-03-28T16:50:30.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8c3affd640 1 -- 172.21.15.170:0/2319822859 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8d4801d9c0 con 0x5624d40941e0 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8c3affd640 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c300342b0 0x7f8c30036770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8c3affd640 1 -- 172.21.15.170:0/2319822859 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8d48040bc0 con 0x5624d40941e0 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d54af8640 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c300342b0 0x7f8c30036770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 Processor -- start 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- start start 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d40a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624d3e1e790 con 0x5624d40a0fe0 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d54af8640 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c300342b0 0x7f8c30036770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5624d407f080 tx=0x7f8d4c005a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d40a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d40a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38090/0 (socket says 172.21.15.170:38090) 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d552f9640 1 -- 172.21.15.170:0/2789339660 learned_addr learned my addr 172.21.15.170:0/2789339660 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d552f9640 1 -- 172.21.15.170:0/2789339660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624d40a19b0 con 0x5624d40a0fe0 2024-03-28T16:50:30.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d552f9640 1 --2- 172.21.15.170:0/2789339660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d40a13e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8d4801d850 tx=0x7f8d48010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31496ecf52eeec56 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8c38ff9640 1 -- 172.21.15.170:0/2789339660 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48027560 con 0x5624d40a0fe0 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8c38ff9640 1 -- 172.21.15.170:0/2789339660 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480460e0 con 0x5624d40a0fe0 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8c38ff9640 1 -- 172.21.15.170:0/2789339660 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48049ad0 con 0x5624d40a0fe0 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- 172.21.15.170:0/2789339660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 msgr2=0x5624d40a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 --2- 172.21.15.170:0/2789339660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d40a13e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8d4801d850 tx=0x7f8d48010ba0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- 172.21.15.170:0/2789339660 shutdown_connections 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 --2- 172.21.15.170:0/2789339660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d40a13e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- 172.21.15.170:0/2789339660 >> 172.21.15.170:0/2789339660 conn(0x5624d409d320 msgr2=0x5624d409b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- 172.21.15.170:0/2789339660 shutdown_connections 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- 172.21.15.170:0/2789339660 wait complete. 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 Processor -- start 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.368+0000 7f8d577de980 1 -- start start 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d577de980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d3efc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d577de980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624d3efcea0 con 0x5624d40a0fe0 2024-03-28T16:50:30.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d3efc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d552f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d3efc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38106/0 (socket says 172.21.15.170:38106) 2024-03-28T16:50:30.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 learned_addr learned my addr 172.21.15.170:0/416667430 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:30.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624d4094b40 con 0x5624d40a0fe0 2024-03-28T16:50:30.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d552f9640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d3efc960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800a810 tx=0x7f8d4801bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8c26ffd640 1 -- 172.21.15.170:0/416667430 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48065070 con 0x5624d40a0fe0 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5624d4094b40 con 0x5624d40a0fe0 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8c26ffd640 1 -- 172.21.15.170:0/416667430 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4800a880 con 0x5624d40a0fe0 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8c26ffd640 1 -- 172.21.15.170:0/416667430 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48027560 con 0x5624d40a0fe0 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624d4094b40 con 0x5624d40a0fe0 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8c26ffd640 1 -- 172.21.15.170:0/416667430 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f8d4805b020 con 0x5624d40a0fe0 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8c26ffd640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c20034260 0x7f8c20036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8c26ffd640 1 -- 172.21.15.170:0/416667430 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8d4806abc0 con 0x5624d40a0fe0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d54af8640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c20034260 0x7f8c20036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d40b5280 0x5624d40b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b7e90 con 0x5624d40b5280 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d55afa640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d40b5280 0x5624d40b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.372+0000 7f8d54af8640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c20034260 0x7f8c20036720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0ae040 tx=0x7f8d4c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d55afa640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d40b5280 0x5624d40b7740 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 realm 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8260 con 0x5624d40b5280 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8ae0 con 0x5624d40b5280 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d40b9b50 0x5624d40bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40bc550 con 0x5624d40b9b50 2024-03-28T16:50:30.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d552f9640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d40b9b50 0x5624d40bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d552f9640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d40b9b50 0x5624d40bbf40 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~921 out=921b] v0'0 uv5 ondisk = 0) v8 ==== 190+0+921 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 rados_obj.operate() r=0 bl.length=921 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 20 searching for the correct realm 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5624d40be820 0x5624d40c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c13b0 con 0x5624d40be820 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d54af8640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5624d40be820 0x5624d40c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d54af8640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5624d40be820 0x5624d40c0ce0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d54af8640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d4c0cb040 0x7f8d4c0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.376+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c13b0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d4c0cb040 0x7f8d4c0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d4c0cb040 0x7f8d4c0b6420 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.382 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.385 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x7f8d4c0cb040 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.380+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x7f8d4c0cb040 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.384+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v31'2 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x7f8d4c0cb040 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.391 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.394 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x7f8d4c0cb040 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.388+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40be820 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b9b50 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v31'5 uv5 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x5624d40b5280 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v31'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40b8e40 con 0x7f8d4c0cb040 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:30.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40bcc20 con 0x5624d40b5280 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40bcc20 con 0x5624d40b5280 2024-03-28T16:50:30.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7f8d500083f0 con 0x5624d40b5280 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40bcbf0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~421 out=421b] v0'0 uv2 ondisk = 0) v8 ==== 195+0+421 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados_obj.operate() r=0 bl.length=421 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 zone default found 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 4 Realm: () 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:30.397 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 zonegroup default 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40bd470 con 0x5624d40b9b50 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40bd8d0 con 0x5624d40be820 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c3320 con 0x5624d40be820 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.398 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 started sync module instance, tier type = 2024-03-28T16:50:30.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:30.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.392+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c2ab0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.396+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d500083f0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.396+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94716176377360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c2e60 con 0x7f8d4c0cb040 2024-03-28T16:50:30.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.396+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c4080 con 0x5624d40b5280 2024-03-28T16:50:30.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.396+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94716176377360] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x7f8d4c0cb040 2024-03-28T16:50:30.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.396+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.396+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94716176381760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c2e60 con 0x5624d40b5280 2024-03-28T16:50:30.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.396+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c4730 con 0x5624d40be820 2024-03-28T16:50:30.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.400+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94716176381760] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.400+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.400+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94716176384592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c4ae0 con 0x5624d40be820 2024-03-28T16:50:30.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.400+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c5450 con 0x7f8d4c0cb040 2024-03-28T16:50:30.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.400+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x7f8d4c0cb040 2024-03-28T16:50:30.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.400+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94716176388288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c5b40 con 0x7f8d4c0cb040 2024-03-28T16:50:30.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.400+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c65f0 con 0x5624d40b5280 2024-03-28T16:50:30.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94716176384592] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94716176388288] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x7f8d4c0cb040 2024-03-28T16:50:30.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94716176392432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c65b0 con 0x5624d40b5280 2024-03-28T16:50:30.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c3cd0 con 0x5624d40b9b50 2024-03-28T16:50:30.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94716176392432] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94716176389536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c6e70 con 0x5624d40b9b50 2024-03-28T16:50:30.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.404+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c7670 con 0x5624d40b9b50 2024-03-28T16:50:30.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.408+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(56 notify.6 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.408+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94716176389536] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.408+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94716176397232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c7e70 con 0x5624d40b9b50 2024-03-28T16:50:30.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.408+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c6a10 con 0x5624d40b9b50 2024-03-28T16:50:30.411 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94716176397232] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(58 notify.7 [create] v31'243 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94716176394768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c8420 con 0x5624d40b9b50 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=0 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=1 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=2 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=3 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=4 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=5 2024-03-28T16:50:30.412 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=6 2024-03-28T16:50:30.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94716176394768] v31'244 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 20 add_watcher() i=7 2024-03-28T16:50:30.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:30.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40cb840 con 0x5624d40b5280 2024-03-28T16:50:30.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.412+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94716176398544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c8d10 con 0x5624d40b5280 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94716176398544] v31'241 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8c0b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 5 note: GC not initialized 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8c0a7e4640 20 reqs_thread_entry: start 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:30.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 Filter name: none 2024-03-28T16:50:30.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8b7bfff640 20 reqs_thread_entry: start 2024-03-28T16:50:30.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-03-28T16:50:30.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.417 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40ed900 con 0x5624d40b9b50 2024-03-28T16:50:30.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-03-28T16:50:30.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:50:30.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-03-28T16:50:30.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 rados->read ofs=0 len=0 2024-03-28T16:50:30.418 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40ed900 con 0x5624d40b5280 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.416+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40ec5f0 con 0x5624d40b9b50 2024-03-28T16:50:30.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~346] v31'9 uv9 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d577de980 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-03-28T16:50:30.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d577de980 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:50:30.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d577de980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-03-28T16:50:30.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94716176550320 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40ec630 con 0x5624d40b9b50 2024-03-28T16:50:30.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== watch-notify(notify (1) cookie 94716176394768 notify 133143986228 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x5624d40a19b0 con 0x5624d40b9b50 2024-03-28T16:50:30.420 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8c38ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986228 cookie 94716176394768 notifier 4844 bl.length()=522 2024-03-28T16:50:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(65 notify.7 [notify cookie 94716176550320 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8c38ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-03-28T16:50:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8c38ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-03-28T16:50:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8c38ff9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f8c18003ea0 con 0x5624d40b9b50 2024-03-28T16:50:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== watch-notify(notify_complete (2) cookie 94716176550320 notify 133143986228 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5624d4077520 con 0x5624d40b9b50 2024-03-28T16:50:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.420+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40ec5f0 con 0x5624d40b5280 2024-03-28T16:50:30.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v31'4 uv4 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d577de980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-28T16:50:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d577de980 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d577de980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-03-28T16:50:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94716176548176 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40ec570 con 0x5624d40b9b50 2024-03-28T16:50:30.423 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== watch-notify(notify (1) cookie 94716176389536 notify 133143986229 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x5624d4077520 con 0x5624d40b9b50 2024-03-28T16:50:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8c38ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986229 cookie 94716176389536 notifier 4844 bl.length()=177 2024-03-28T16:50:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8c38ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-03-28T16:50:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8c38ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-03-28T16:50:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8c38ff9640 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x7f8c18003ea0 con 0x5624d40b9b50 2024-03-28T16:50:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(68 notify.5 [notify cookie 94716176548176 out=8b] v0'0 uv243 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== watch-notify(notify_complete (2) cookie 94716176548176 notify 133143986229 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x7f8d4802d070 con 0x5624d40b9b50 2024-03-28T16:50:30.424 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.425 INFO:teuthology.orchestra.run.smithi170.stdout:{"user_id":"bar","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bar","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]} 2024-03-28T16:50:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8c0b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:30.425 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.424+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94716176398544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40ec560 con 0x5624d40b5280 2024-03-28T16:50:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94716176398544] v31'242 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94716176377360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c78d0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94716176377360] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x7f8d4c0cb040 2024-03-28T16:50:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d577de980 20 remove_watcher() i=0 2024-03-28T16:50:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d577de980 2 removed watcher, disabling cache 2024-03-28T16:50:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94716176381760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c78d0 con 0x5624d40b5280 2024-03-28T16:50:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94716176381760] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d577de980 20 remove_watcher() i=1 2024-03-28T16:50:30.430 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.428+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94716176384592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c3740 con 0x5624d40be820 2024-03-28T16:50:30.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.432+0000 7f8d54af8640 1 -- 172.21.15.170:0/416667430 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94716176384592] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d4c0be1e0 con 0x5624d40be820 2024-03-28T16:50:30.432 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.432+0000 7f8d577de980 20 remove_watcher() i=2 2024-03-28T16:50:30.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.432+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94716176388288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c78d0 con 0x7f8d4c0cb040 2024-03-28T16:50:30.433 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.432+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94716176388288] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x7f8d4c0cb040 2024-03-28T16:50:30.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.432+0000 7f8d577de980 20 remove_watcher() i=3 2024-03-28T16:50:30.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.432+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94716176392432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c78d0 con 0x5624d40b5280 2024-03-28T16:50:30.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.436+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94716176392432] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d5003f040 con 0x5624d40b5280 2024-03-28T16:50:30.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.436+0000 7f8d577de980 20 remove_watcher() i=4 2024-03-28T16:50:30.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.436+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94716176389536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c60f0 con 0x5624d40b9b50 2024-03-28T16:50:30.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.436+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94716176389536] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.436+0000 7f8d577de980 20 remove_watcher() i=5 2024-03-28T16:50:30.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.436+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94716176397232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c78d0 con 0x5624d40b9b50 2024-03-28T16:50:30.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94716176397232] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 20 remove_watcher() i=6 2024-03-28T16:50:30.440 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94716176394768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x5624d40c78d0 con 0x5624d40b9b50 2024-03-28T16:50:30.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d552f9640 1 -- 172.21.15.170:0/416667430 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94716176394768] v31'245 uv243 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f8d48024020 con 0x5624d40b9b50 2024-03-28T16:50:30.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 20 remove_watcher() i=7 2024-03-28T16:50:30.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d40b9b50 msgr2=0x5624d40bbf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d40b9b50 0x5624d40bbf40 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d40b5280 msgr2=0x5624d40b7740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d40b5280 0x5624d40b7740 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d4c0cb040 msgr2=0x7f8d4c0b6420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d4c0cb040 0x7f8d4c0b6420 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5624d40be820 msgr2=0x5624d40c0ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x5624d40be820 0x5624d40c0ce0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c20034260 msgr2=0x7f8c20036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c20034260 0x7f8c20036720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0ae040 tx=0x7f8d4c0a6000 comp rx=0 tx=0).stop 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 msgr2=0x5624d3efc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d55afa640 1 -- 172.21.15.170:0/416667430 reap_dead start 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d3efc960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800a810 tx=0x7f8d4801bf20 comp rx=0 tx=0).stop 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 shutdown_connections 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 --2- 172.21.15.170:0/416667430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40a0fe0 0x5624d3efc960 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 >> 172.21.15.170:0/416667430 conn(0x5624d409d320 msgr2=0x5624d409a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 shutdown_connections 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.440+0000 7f8d577de980 1 -- 172.21.15.170:0/416667430 wait complete. 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c300342b0 msgr2=0x7f8c30036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c300342b0 0x7f8c30036770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5624d407f080 tx=0x7f8d4c005a10 comp rx=0 tx=0).stop 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40941e0 msgr2=0x5624d4094600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40941e0 0x5624d4094600 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8d48047ae0 tx=0x7f8d48011ed0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 shutdown_connections 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c300342b0 0x7f8c30036770 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/2319822859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d40941e0 0x5624d4094600 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 >> 172.21.15.170:0/2319822859 conn(0x5624d4088ff0 msgr2=0x5624d4089b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 shutdown_connections 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/2319822859 wait complete. 2024-03-28T16:50:30.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d407fdf0 msgr2=0x5624d40821e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x5624d407fdf0 0x5624d40821e0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d3f8a480 msgr2=0x5624d407d7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x5624d3f8a480 0x5624d407d7e0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d48044000 msgr2=0x5624d4085b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7f8d48044000 0x5624d4085b30 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c3c0342b0 msgr2=0x7f8c3c036770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f8c3c0342b0 0x7f8c3c036770 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0ad000 tx=0x7f8d4c0a5040 comp rx=0 tx=0).stop 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 msgr2=0x5624d4076bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 --2- 172.21.15.170:0/724846843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5624d3f85fc0 0x5624d4076bb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8d480012f0 tx=0x7f8d48001260 comp rx=0 tx=0).stop 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d55afa640 1 -- 172.21.15.170:0/724846843 reap_dead start 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 shutdown_connections 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 >> 172.21.15.170:0/724846843 conn(0x5624d3f81890 msgr2=0x5624d406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:30.447 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 shutdown_connections 2024-03-28T16:50:30.448 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:30.444+0000 7f8d577de980 1 -- 172.21.15.170:0/724846843 wait complete. 2024-03-28T16:50:30.455 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-03-28T16:50:35.483 INFO:tasks.radosgw_admin:RL: create_bucket myfoo fud bytes_out=0 bytes_in=0 failed=False 2024-03-28T16:50:35.483 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_acb3247c84f48e9ce91359973bf22ebb44ec5034/qa/tasks/radosgw_admin.py", line 1136, in task key.set_storage_class(storage_class) AttributeError: 'Key' object has no attribute 'set_storage_class' 2024-03-28T16:50:35.734 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=69cdb6a1ff204c52bf229833670ca014 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_acb3247c84f48e9ce91359973bf22ebb44ec5034/qa/tasks/radosgw_admin.py", line 1136, in task key.set_storage_class(storage_class) AttributeError: 'Key' object has no attribute 'set_storage_class' 2024-03-28T16:50:35.736 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-03-28T16:50:35.745 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-03-28T16:50:35.746 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T16:50:35.749 INFO:tasks.rgw.client.0.smithi170.stdout:2024-03-28T16:50:35.743+0000 7f40b6e21640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 27289) UID: 0 2024-03-28T16:50:35.749 INFO:tasks.rgw.client.0.smithi170.stdout:2024-03-28T16:50:35.743+0000 7f40ba669a40 -1 shutting down 2024-03-28T16:50:37.465 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-03-28T16:50:41.847 INFO:tasks.rgw.client.0:Stopped 2024-03-28T16:50:41.848 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-03-28T16:50:41.854 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/vault-root-token 2024-03-28T16:50:41.907 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-03-28T16:50:41.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-03-28T16:50:41.908 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-03-28T16:50:41.988 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setuser ceph since I am not root 2024-03-28T16:50:41.988 INFO:teuthology.orchestra.run.smithi170.stderr:ignoring --setgroup ceph since I am not root 2024-03-28T16:50:41.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22a833980 1 Processor -- start 2024-03-28T16:50:41.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22a833980 1 -- start start 2024-03-28T16:50:41.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22a833980 1 --2- >> v2:172.21.15.119:3300/0 conn(0x558712d3fb70 0x558712d3ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:41.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22a833980 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558712d15520 con 0x558712d40490 2024-03-28T16:50:41.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22a833980 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x558712cdc0f0 con 0x558712d3fb70 2024-03-28T16:50:41.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22834e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x558712d3fb70 0x558712d3ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:41.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22834e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x558712d3fb70 0x558712d3ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38844/0 (socket says 172.21.15.170:38844) 2024-03-28T16:50:41.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22834e640 1 -- 172.21.15.170:0/3961424819 learned_addr learned my addr 172.21.15.170:0/3961424819 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:41.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22834e640 1 -- 172.21.15.170:0/3961424819 >> v1:172.21.15.119:6789/0 conn(0x558712d40490 legacy=0x558712d40e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:41.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.987+0000 7fb22834e640 1 -- 172.21.15.170:0/3961424819 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558712d41680 con 0x558712d3fb70 2024-03-28T16:50:41.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22834e640 1 --2- 172.21.15.170:0/3961424819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712d3ff50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb21800a4e0 tx=0x7fb21801b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0881cf32e5d7b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:41.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22734c640 1 -- 172.21.15.170:0/3961424819 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180aa540 con 0x558712d3fb70 2024-03-28T16:50:41.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22734c640 1 -- 172.21.15.170:0/3961424819 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180a6070 con 0x558712d3fb70 2024-03-28T16:50:41.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22734c640 1 -- 172.21.15.170:0/3961424819 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180aa850 con 0x558712d3fb70 2024-03-28T16:50:41.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22a833980 1 -- 172.21.15.170:0/3961424819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 msgr2=0x558712d3ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:41.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22a833980 1 --2- 172.21.15.170:0/3961424819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712d3ff50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb21800a4e0 tx=0x7fb21801b090 comp rx=0 tx=0).stop 2024-03-28T16:50:41.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22a833980 1 -- 172.21.15.170:0/3961424819 shutdown_connections 2024-03-28T16:50:41.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22a833980 1 --2- 172.21.15.170:0/3961424819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712d3ff50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:41.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22a833980 1 -- 172.21.15.170:0/3961424819 >> 172.21.15.170:0/3961424819 conn(0x558712d3e850 msgr2=0x558712d3ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:41.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22a833980 1 -- 172.21.15.170:0/3961424819 shutdown_connections 2024-03-28T16:50:41.995 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:41.991+0000 7fb22a833980 1 -- 172.21.15.170:0/3961424819 wait complete. 2024-03-28T16:50:42.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22a833980 1 Processor -- start 2024-03-28T16:50:42.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22a833980 1 -- start start 2024-03-28T16:50:42.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22a833980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712eba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22a833980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558712cd9620 con 0x558712d3fb70 2024-03-28T16:50:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712eba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712eba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38854/0 (socket says 172.21.15.170:38854) 2024-03-28T16:50:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22834e640 1 -- 172.21.15.170:0/1384337364 learned_addr learned my addr 172.21.15.170:0/1384337364 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.007+0000 7fb22834e640 1 -- 172.21.15.170:0/1384337364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21800a1c0 con 0x558712d3fb70 2024-03-28T16:50:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22834e640 1 --2- 172.21.15.170:0/1384337364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712eba540 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb218017ca0 tx=0x7fb2180033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3170ee871824d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb11660c640 1 -- 172.21.15.170:0/1384337364 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180aa750 con 0x558712d3fb70 2024-03-28T16:50:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb11660c640 1 -- 172.21.15.170:0/1384337364 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180aa8b0 con 0x558712d3fb70 2024-03-28T16:50:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb11660c640 1 -- 172.21.15.170:0/1384337364 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180aab70 con 0x558712d3fb70 2024-03-28T16:50:42.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- 172.21.15.170:0/1384337364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 msgr2=0x558712eba540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 --2- 172.21.15.170:0/1384337364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712eba540 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb218017ca0 tx=0x7fb2180033b0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- 172.21.15.170:0/1384337364 shutdown_connections 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 --2- 172.21.15.170:0/1384337364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712eba540 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- 172.21.15.170:0/1384337364 >> 172.21.15.170:0/1384337364 conn(0x558712d3e850 msgr2=0x558712eb60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- 172.21.15.170:0/1384337364 shutdown_connections 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- 172.21.15.170:0/1384337364 wait complete. 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 Processor -- start 2024-03-28T16:50:42.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- start start 2024-03-28T16:50:42.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712ec1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558712cd4e00 con 0x558712d3fb70 2024-03-28T16:50:42.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712ec1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712ec1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38856/0 (socket says 172.21.15.170:38856) 2024-03-28T16:50:42.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22834e640 1 -- 172.21.15.170:0/3093734421 learned_addr learned my addr 172.21.15.170:0/3093734421 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:42.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22834e640 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558712d41680 con 0x558712d3fb70 2024-03-28T16:50:42.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22834e640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712ec1290 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb21801b930 tx=0x7fb218017e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb114e09640 1 -- 172.21.15.170:0/3093734421 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180c4070 con 0x558712d3fb70 2024-03-28T16:50:42.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb114e09640 1 -- 172.21.15.170:0/3093734421 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180c8580 con 0x558712d3fb70 2024-03-28T16:50:42.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb114e09640 1 -- 172.21.15.170:0/3093734421 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180c8820 con 0x558712d3fb70 2024-03-28T16:50:42.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.011+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558712ec17d0 con 0x558712d3fb70 2024-03-28T16:50:42.017 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558712ec1c00 con 0x558712d3fb70 2024-03-28T16:50:42.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb114e09640 1 -- 172.21.15.170:0/3093734421 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb2180a8070 con 0x558712d3fb70 2024-03-28T16:50:42.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb114e09640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb110034300 0x7fb1100367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb114e09640 1 -- 172.21.15.170:0/3093734421 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb2180e3780 con 0x558712d3fb70 2024-03-28T16:50:42.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712d40490 0x558712ec7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712ec8400 con 0x558712d40490 2024-03-28T16:50:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb110034300 0x7fb1100367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb228b4f640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712d40490 0x558712ec7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb110034300 0x7fb1100367c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb21c000f70 tx=0x7fb21c001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb228b4f640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712d40490 0x558712ec7ec0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb228b4f640 1 -- 172.21.15.170:0/3093734421 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2200083f0 con 0x558712d40490 2024-03-28T16:50:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712ec8760 con 0x558712d40490 2024-03-28T16:50:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb228b4f640 1 -- 172.21.15.170:0/3093734421 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb2200083f0 con 0x558712d40490 2024-03-28T16:50:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712eca0f0 0x558712ecc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.015+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:3 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712eccc80 con 0x558712eca0f0 2024-03-28T16:50:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22834e640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712eca0f0 0x558712ecc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22834e640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712eca0f0 0x558712ecc510 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22834e640 1 -- 172.21.15.170:0/3093734421 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(3 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [call out=48b,read 0~921 out=921b] v0'0 uv5 ondisk = 0) v8 ==== 232+0+969 (crc 0 0 0) 0x7fb2180d0390 con 0x558712eca0f0 2024-03-28T16:50:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712ec8f60 con 0x558712d40490 2024-03-28T16:50:42.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb228b4f640 1 -- 172.21.15.170:0/3093734421 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb2200083f0 con 0x558712d40490 2024-03-28T16:50:42.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2180d3000 0x558712ed0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:5 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712ed07f0 con 0x7fb2180d3000 2024-03-28T16:50:42.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2180d3000 0x558712ed0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.023 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2180d3000 0x558712ed0090 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb227b4d640 1 -- 172.21.15.170:0/3093734421 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(5 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [call out=48b,read 0~421 out=421b] v0'0 uv2 ondisk = 0) v8 ==== 237+0+469 (crc 0 0 0) 0x7fb21c0106b0 con 0x7fb2180d3000 2024-03-28T16:50:42.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22a833980 1 Processor -- start 2024-03-28T16:50:42.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22a833980 1 -- start start 2024-03-28T16:50:42.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22a833980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712ede780 0x558712edeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.019+0000 7fb22a833980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558712c6de30 con 0x558712ede780 2024-03-28T16:50:42.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712ede780 0x558712edeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712ede780 0x558712edeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38858/0 (socket says 172.21.15.170:38858) 2024-03-28T16:50:42.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22834e640 1 -- 172.21.15.170:0/2978763969 learned_addr learned my addr 172.21.15.170:0/2978763969 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:42.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22834e640 1 -- 172.21.15.170:0/2978763969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558712edf0e0 con 0x558712ede780 2024-03-28T16:50:42.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22834e640 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712ede780 0x558712edeba0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb2180e1d40 tx=0x7fb2180d0a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb10dffb640 1 -- 172.21.15.170:0/2978763969 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180d7040 con 0x558712ede780 2024-03-28T16:50:42.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb10dffb640 1 -- 172.21.15.170:0/2978763969 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180a95d0 con 0x558712ede780 2024-03-28T16:50:42.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb10dffb640 1 -- 172.21.15.170:0/2978763969 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180a98b0 con 0x558712ede780 2024-03-28T16:50:42.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558712edf330 con 0x558712ede780 2024-03-28T16:50:42.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558712edfe80 con 0x558712ede780 2024-03-28T16:50:42.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb10dffb640 1 -- 172.21.15.170:0/2978763969 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb2180e3a60 con 0x558712ede780 2024-03-28T16:50:42.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb10dffb640 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb100034260 0x7fb100036720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb10dffb640 1 -- 172.21.15.170:0/2978763969 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb2180edbb0 con 0x558712ede780 2024-03-28T16:50:42.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb227b4d640 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb100034260 0x7fb100036720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22a833980 1 Processor -- start 2024-03-28T16:50:42.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22a833980 1 -- start start 2024-03-28T16:50:42.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb227b4d640 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb100034260 0x7fb100036720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x558712ec92c0 tx=0x7fb21c006000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22a833980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712eebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.023+0000 7fb22a833980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558712c61790 con 0x558712eeb700 2024-03-28T16:50:42.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712eebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.031 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712eebb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38872/0 (socket says 172.21.15.170:38872) 2024-03-28T16:50:42.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 -- 172.21.15.170:0/2735532591 learned_addr learned my addr 172.21.15.170:0/2735532591 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:42.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 -- 172.21.15.170:0/2735532591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558712eec0f0 con 0x558712eeb700 2024-03-28T16:50:42.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 --2- 172.21.15.170:0/2735532591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712eebb20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb2180bd4b0 tx=0x7fb2180edf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3eb9b16d2ae4d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb0ff7fe640 1 -- 172.21.15.170:0/2735532591 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180d2560 con 0x558712eeb700 2024-03-28T16:50:42.033 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb0ff7fe640 1 -- 172.21.15.170:0/2735532591 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2180dec50 con 0x558712eeb700 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb0ff7fe640 1 -- 172.21.15.170:0/2735532591 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180e33d0 con 0x558712eeb700 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 -- 172.21.15.170:0/2735532591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 msgr2=0x558712eebb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 --2- 172.21.15.170:0/2735532591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712eebb20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb2180bd4b0 tx=0x7fb2180edf70 comp rx=0 tx=0).stop 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 -- 172.21.15.170:0/2735532591 shutdown_connections 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 --2- 172.21.15.170:0/2735532591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712eebb20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 -- 172.21.15.170:0/2735532591 >> 172.21.15.170:0/2735532591 conn(0x558712ee7a40 msgr2=0x558712ee5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 -- 172.21.15.170:0/2735532591 shutdown_connections 2024-03-28T16:50:42.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 -- 172.21.15.170:0/2735532591 wait complete. 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 Processor -- start 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 -- start start 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712ef23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22a833980 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x558712ef2900 con 0x558712eeb700 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712ef23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712ef23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.170:38884/0 (socket says 172.21.15.170:38884) 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 learned_addr learned my addr 172.21.15.170:0/3847699318 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-28T16:50:42.035 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558712edfe80 con 0x558712eeb700 2024-03-28T16:50:42.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.027+0000 7fb22834e640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712ef23c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb2180edfb0 tx=0x7fb2180d9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb0fdffb640 1 -- 172.21.15.170:0/3847699318 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180b4a20 con 0x558712eeb700 2024-03-28T16:50:42.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb0fdffb640 1 -- 172.21.15.170:0/3847699318 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218104070 con 0x558712eeb700 2024-03-28T16:50:42.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558712edfe80 con 0x558712eeb700 2024-03-28T16:50:42.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb0fdffb640 1 -- 172.21.15.170:0/3847699318 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2180b4d30 con 0x558712eeb700 2024-03-28T16:50:42.036 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558712edfe80 con 0x558712eeb700 2024-03-28T16:50:42.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb0fdffb640 1 -- 172.21.15.170:0/3847699318 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fb218106030 con 0x558712eeb700 2024-03-28T16:50:42.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb0fdffb640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb0f0034300 0x7fb0f00367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb0fdffb640 1 -- 172.21.15.170:0/3847699318 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb21810d910 con 0x558712eeb700 2024-03-28T16:50:42.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.042 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb0f0034300 0x7fb0f00367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712efd540 0x558712effa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00150 con 0x558712efd540 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb228b4f640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712efd540 0x558712effa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb0f0034300 0x7fb0f00367c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x558712ed0bb0 tx=0x7fb21c011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb228b4f640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712efd540 0x558712effa00 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:42.043 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 realm 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00520 con 0x558712efd540 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f010d0 con 0x558712efd540 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.031+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712f01a80 0x558712f03e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:5 5.17 5:ee71639f:::zone_info.b88c2392-6ef8-4660-b080-19480b748fc0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f04510 con 0x558712f01a80 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22834e640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712f01a80 0x558712f03e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22834e640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712f01a80 0x558712f03e70 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== osd_op_reply(5 zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 [read 0~921 out=921b] v0'0 uv5 ondisk = 0) v8 ==== 190+0+921 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 20 rados_obj.operate() r=0 bl.length=921 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 20 searching for the correct realm 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558712f06790 0x558712f08c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f092d0 con 0x558712f06790 2024-03-28T16:50:42.046 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558712f06790 0x558712f08c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558712f06790 0x558712f08c50 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb227b4d640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb21c00f3d0 0x7fb21c00f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f092d0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb228b4f640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb21c00f3d0 0x7fb21c00f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb228b4f640 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb21c00f3d0 0x7fb21c00f810 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=112b] v31'2 uv2 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.047 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.035+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 3 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 4 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.049 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.057 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.039+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 5 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v22'1 uv1 ondisk = 1) v8 ==== 144+0+79 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 3 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.058 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 4 ==== osd_op_reply(20 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 6 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 5 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 6 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.059 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v31'2 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.043+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 7 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 7 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.060 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 8 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 8 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.061 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 9 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f06790 2024-03-28T16:50:42.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 9 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.047+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712f01a80 2024-03-28T16:50:42.062 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 10 ==== osd_op_reply(35 [pgnls start_epoch 31 out=107b] v31'5 uv5 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x558712efd540 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v31'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f00d70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 RGWRados::pool_iterate: got zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 RGWRados::pool_iterate: got default.zone. 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 RGWRados::pool_iterate: got zone_info.b88c2392-6ef8-4660-b080-19480b748fc0 2024-03-28T16:50:42.063 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 RGWRados::pool_iterate: got zone_names.default 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f04b70 con 0x558712efd540 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f04b70 con 0x558712efd540 2024-03-28T16:50:42.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x7fb2200083f0 con 0x558712efd540 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados_obj.operate() r=0 bl.length=46 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:40 5.10 5:0ba8d62e:::zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f04b70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 7 ==== osd_op_reply(40 zonegroup_info.c7513231-bef6-43ae-ade6-ce57a95c9845 [read 0~421 out=421b] v0'0 uv2 ondisk = 0) v8 ==== 195+0+421 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados_obj.operate() r=0 bl.length=421 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 zone default found 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 4 Realm: () 2024-03-28T16:50:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 4 ZoneGroup: default (c7513231-bef6-43ae-ade6-ce57a95c9845) 2024-03-28T16:50:42.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 4 Zone: default (b88c2392-6ef8-4660-b080-19480b748fc0) 2024-03-28T16:50:42.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 10 cannot find current period zonegroup using local zonegroup configuration 2024-03-28T16:50:42.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 zonegroup default 2024-03-28T16:50:42.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f05410 con 0x558712f01a80 2024-03-28T16:50:42.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.066 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f05840 con 0x558712f06790 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 10 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados->read ofs=0 len=0 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0b220 con 0x558712f06790 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 11 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 rados_obj.operate() r=-2 bl.length=0 2024-03-28T16:50:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 started sync module instance, tier type = 2024-03-28T16:50:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 20 started zone id=b88c2392-6ef8-4660-b080-19480b748fc0 (name=default) with tier type = 2024-03-28T16:50:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.051+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0a9d0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.055+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 8 ==== osd_op_reply(44 notify.0 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2200083f0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.055+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94038626706736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0ad80 con 0x7fb21c00f3d0 2024-03-28T16:50:42.068 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.055+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0bf80 con 0x558712efd540 2024-03-28T16:50:42.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.055+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 9 ==== osd_op_reply(45 notify.0 [watch watch cookie 94038626706736] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.055+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 15 ==== osd_op_reply(46 notify.1 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.055+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94038626711104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0bf60 con 0x558712efd540 2024-03-28T16:50:42.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.055+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0ca20 con 0x558712f06790 2024-03-28T16:50:42.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.059+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94038626711104] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.071 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.059+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 12 ==== osd_op_reply(48 notify.2 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.059+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94038626714848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0cdd0 con 0x558712f06790 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.059+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0d740 con 0x7fb21c00f3d0 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.059+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 10 ==== osd_op_reply(50 notify.3 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.059+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94038626718640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0de30 con 0x7fb21c00f3d0 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.059+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0e900 con 0x558712efd540 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.063+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 13 ==== osd_op_reply(49 notify.2 [watch watch cookie 94038626714848] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.063+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 11 ==== osd_op_reply(51 notify.3 [watch watch cookie 94038626718640] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.072 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.063+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 17 ==== osd_op_reply(52 notify.4 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.063+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94038626722784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0e8c0 con 0x558712efd540 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.063+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0ecb0 con 0x558712f01a80 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.063+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94038626722784] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.067+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 12 ==== osd_op_reply(54 notify.5 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.067+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94038626712160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0f060 con 0x558712f01a80 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.067+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0f9d0 con 0x558712f01a80 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.067+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94038626712160] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.067+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 14 ==== osd_op_reply(56 notify.6 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.067+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94038626727216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0fdf0 con 0x558712f01a80 2024-03-28T16:50:42.074 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.067+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f10760 con 0x558712f01a80 2024-03-28T16:50:42.075 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 15 ==== osd_op_reply(58 notify.7 [create] v31'247 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94038626730688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f10b80 con 0x558712f01a80 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=0 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=1 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=2 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=3 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=4 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=5 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94038626727216] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=6 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94038626730688] v31'248 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 20 add_watcher() i=7 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 2 all 8 watchers are set, enabling cache 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0fa40 con 0x558712efd540 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94038626728464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f10790 con 0x558712efd540 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.071+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94038626728464] v31'244 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.075+0000 7fb0de7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:42.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.075+0000 7fb22a833980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-03-28T16:50:42.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.075+0000 7fb22a833980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-03-28T16:50:42.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.075+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1c150 con 0x558712efd540 2024-03-28T16:50:42.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.075+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'35 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.078 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.075+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1c590 con 0x558712f01a80 2024-03-28T16:50:42.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.079 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1ca30 con 0x558712f06790 2024-03-28T16:50:42.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 14 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.080 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1d230 con 0x558712efd540 2024-03-28T16:50:42.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'245 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.082 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1a1d0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 12 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'77 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.083 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.079+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1a1d0 con 0x558712f06790 2024-03-28T16:50:42.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.083+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 15 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'51 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.084 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.083+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1df70 con 0x558712efd540 2024-03-28T16:50:42.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.083+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'31 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.086 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.083+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1e000 con 0x558712f01a80 2024-03-28T16:50:42.088 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.087+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'52 uv34 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.087+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1e530 con 0x7fb21c00f3d0 2024-03-28T16:50:42.090 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.087+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 13 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.091 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.087+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1e530 con 0x558712f06790 2024-03-28T16:50:42.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.091+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 16 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'35 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.092 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.091+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1ea60 con 0x7fb21c00f3d0 2024-03-28T16:50:42.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.091+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 14 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'78 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.093 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.091+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1ea60 con 0x558712f01a80 2024-03-28T16:50:42.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.095+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'36 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.095 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.095+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1ef90 con 0x7fb21c00f3d0 2024-03-28T16:50:42.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.095+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 15 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.096 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.095+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1ef90 con 0x558712f01a80 2024-03-28T16:50:42.098 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.095+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'58 uv41 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.099 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.095+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1f0d0 con 0x558712efd540 2024-03-28T16:50:42.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.099+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'246 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.099+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f1ef00 con 0x558712f06790 2024-03-28T16:50:42.102 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.099+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 17 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'71 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.099+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.103+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 16 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'79 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.104 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.103+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f06790 2024-03-28T16:50:42.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.103+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 18 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.103+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712efd540 2024-03-28T16:50:42.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.107+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'37 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.107+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18ac0 con 0x558712efd540 2024-03-28T16:50:42.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.107+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'247 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.110 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.107+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f06790 2024-03-28T16:50:42.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.111+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 19 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'72 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.111+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18ac0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.111+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 17 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'39 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.111+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18ac0 con 0x558712f06790 2024-03-28T16:50:42.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.115+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 20 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'32 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.115+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18ac0 con 0x558712f06790 2024-03-28T16:50:42.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.115+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 21 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'35 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.115+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f06790 2024-03-28T16:50:42.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.115+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 22 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'25 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.115+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f06790 2024-03-28T16:50:42.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.119+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 23 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'52 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.119+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f01a80 2024-03-28T16:50:42.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.119+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'33 uv25 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.119+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f06790 2024-03-28T16:50:42.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.119+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 24 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'30 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.119+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f01a80 2024-03-28T16:50:42.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.123+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'53 uv36 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.123+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f06790 2024-03-28T16:50:42.127 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.127+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 25 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'73 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.127+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x558712f01a80 2024-03-28T16:50:42.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.127+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'59 uv43 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.127+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f191c0 con 0x7fb21c00f3d0 2024-03-28T16:50:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.131+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 18 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'36 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.131+0000 7fb0dd7e2640 20 reqs_thread_entry: start 2024-03-28T16:50:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.131+0000 7fb22a833980 20 init_complete bucket index max shards: 11 2024-03-28T16:50:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.131+0000 7fb22a833980 20 Filter name: none 2024-03-28T16:50:42.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.131+0000 7fb04effd640 20 reqs_thread_entry: start 2024-03-28T16:50:42.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.131+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-03-28T16:50:42.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.131+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 27 ==== osd_op_reply(94 gc.0 [call] v31'36 uv36 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 28 ==== osd_op_reply(95 gc.0 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 29 ==== osd_op_reply(96 gc.0 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:97 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 30 ==== osd_op_reply(97 gc.0 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:98 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 31 ==== osd_op_reply(98 gc.0 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.135+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:99 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 32 ==== osd_op_reply(99 gc.0 [call] v31'37 uv37 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-03-28T16:50:42.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:100 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 25 ==== osd_op_reply(100 gc.1 [call] v31'30 uv30 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.141 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:101 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 26 ==== osd_op_reply(101 gc.1 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.139+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:102 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 27 ==== osd_op_reply(102 gc.1 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:103 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 28 ==== osd_op_reply(103 gc.1 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:104 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 29 ==== osd_op_reply(104 gc.1 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.143+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:105 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 30 ==== osd_op_reply(105 gc.1 [call] v31'31 uv31 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-03-28T16:50:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:106 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 26 ==== osd_op_reply(106 gc.2 [call] v31'30 uv30 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:107 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 27 ==== osd_op_reply(107 gc.2 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:108 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 28 ==== osd_op_reply(108 gc.2 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.147+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:109 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 29 ==== osd_op_reply(109 gc.2 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:110 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 30 ==== osd_op_reply(110 gc.2 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:111 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 31 ==== osd_op_reply(111 gc.2 [call] v31'31 uv31 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-03-28T16:50:42.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:112 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 33 ==== osd_op_reply(112 gc.3 [call] v31'248 uv248 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.151+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:113 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 34 ==== osd_op_reply(113 gc.3 [call out=15b] v0'0 uv248 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:114 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 35 ==== osd_op_reply(114 gc.3 [call out=24b] v0'0 uv248 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:115 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 36 ==== osd_op_reply(115 gc.3 [call out=15b] v0'0 uv248 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:116 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 37 ==== osd_op_reply(116 gc.3 [call out=15b] v0'0 uv248 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:117 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 38 ==== osd_op_reply(117 gc.3 [call] v31'249 uv249 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-03-28T16:50:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.155+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:118 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 19 ==== osd_op_reply(118 gc.4 [call] v31'80 uv80 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:119 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 20 ==== osd_op_reply(119 gc.4 [call out=15b] v0'0 uv80 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:120 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 21 ==== osd_op_reply(120 gc.4 [call out=24b] v0'0 uv80 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.162 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:121 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 22 ==== osd_op_reply(121 gc.4 [call out=15b] v0'0 uv80 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:122 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 23 ==== osd_op_reply(122 gc.4 [call out=15b] v0'0 uv80 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.163 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.159+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:123 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 24 ==== osd_op_reply(123 gc.4 [call] v31'81 uv81 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-03-28T16:50:42.164 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:124 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 32 ==== osd_op_reply(124 gc.5 [call] v31'53 uv53 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.165 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:125 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 33 ==== osd_op_reply(125 gc.5 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:126 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 34 ==== osd_op_reply(126 gc.5 [call out=24b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.167 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:127 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 35 ==== osd_op_reply(127 gc.5 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:128 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 36 ==== osd_op_reply(128 gc.5 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.168 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.163+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:129 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.167+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 37 ==== osd_op_reply(129 gc.5 [call] v31'54 uv54 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.167+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-03-28T16:50:42.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.167+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:130 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.167+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 39 ==== osd_op_reply(130 gc.6 [call] v31'32 uv32 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.167+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:131 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 40 ==== osd_op_reply(131 gc.6 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:132 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 41 ==== osd_op_reply(132 gc.6 [call out=24b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:133 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 42 ==== osd_op_reply(133 gc.6 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:134 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 43 ==== osd_op_reply(134 gc.6 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:135 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.174 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 44 ==== osd_op_reply(135 gc.6 [call] v31'33 uv33 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-03-28T16:50:42.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.171+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:136 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.175+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 31 ==== osd_op_reply(136 gc.7 [call] v31'54 uv54 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.175+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:137 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.175+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 32 ==== osd_op_reply(137 gc.7 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.175+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.175+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:138 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 33 ==== osd_op_reply(138 gc.7 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:139 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 34 ==== osd_op_reply(139 gc.7 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:140 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 35 ==== osd_op_reply(140 gc.7 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.179+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:141 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.183+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 36 ==== osd_op_reply(141 gc.7 [call] v31'55 uv55 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.183+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-03-28T16:50:42.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.183+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:142 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 25 ==== osd_op_reply(142 gc.8 [call] v31'30 uv30 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:143 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 26 ==== osd_op_reply(143 gc.8 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:144 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 27 ==== osd_op_reply(144 gc.8 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:145 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 28 ==== osd_op_reply(145 gc.8 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:146 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 29 ==== osd_op_reply(146 gc.8 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.187+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:147 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 30 ==== osd_op_reply(147 gc.8 [call] v31'31 uv31 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-03-28T16:50:42.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:148 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 38 ==== osd_op_reply(148 gc.9 [call] v31'36 uv36 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:149 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 39 ==== osd_op_reply(149 gc.9 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:150 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 40 ==== osd_op_reply(150 gc.9 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.191+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:151 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 41 ==== osd_op_reply(151 gc.9 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:152 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 42 ==== osd_op_reply(152 gc.9 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:153 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 43 ==== osd_op_reply(153 gc.9 [call] v31'37 uv37 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-03-28T16:50:42.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.195+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:154 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 31 ==== osd_op_reply(154 gc.10 [call] v31'82 uv82 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:155 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 32 ==== osd_op_reply(155 gc.10 [call out=15b] v0'0 uv82 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:156 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 33 ==== osd_op_reply(156 gc.10 [call out=24b] v0'0 uv82 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:157 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 34 ==== osd_op_reply(157 gc.10 [call out=15b] v0'0 uv82 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:158 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 35 ==== osd_op_reply(158 gc.10 [call out=15b] v0'0 uv82 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.199+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:159 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 36 ==== osd_op_reply(159 gc.10 [call] v31'83 uv83 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-03-28T16:50:42.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:160 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 37 ==== osd_op_reply(160 gc.11 [call] v31'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:161 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 38 ==== osd_op_reply(161 gc.11 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.203+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:162 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.207+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 39 ==== osd_op_reply(162 gc.11 [call out=24b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.207+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:163 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.207+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 40 ==== osd_op_reply(163 gc.11 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.207+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.207+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:164 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.211+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 41 ==== osd_op_reply(164 gc.11 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.211+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.211+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:165 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.211+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 42 ==== osd_op_reply(165 gc.11 [call] v31'38 uv38 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.211+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-03-28T16:50:42.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.211+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:166 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 37 ==== osd_op_reply(166 gc.12 [call] v31'30 uv30 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:167 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 38 ==== osd_op_reply(167 gc.12 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:168 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 39 ==== osd_op_reply(168 gc.12 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:169 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 40 ==== osd_op_reply(169 gc.12 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:170 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 41 ==== osd_op_reply(170 gc.12 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.215+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:171 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.219+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 42 ==== osd_op_reply(171 gc.12 [call] v31'31 uv31 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.219+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-03-28T16:50:42.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.219+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:172 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 43 ==== osd_op_reply(172 gc.13 [call] v31'60 uv60 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:173 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 44 ==== osd_op_reply(173 gc.13 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:174 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 45 ==== osd_op_reply(174 gc.13 [call out=24b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:175 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 46 ==== osd_op_reply(175 gc.13 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.223+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:176 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.227+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 47 ==== osd_op_reply(176 gc.13 [call out=15b] v0'0 uv60 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.227+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.227+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:177 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.227+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 48 ==== osd_op_reply(177 gc.13 [call] v31'61 uv61 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.227+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-03-28T16:50:42.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.227+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:178 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 45 ==== osd_op_reply(178 gc.14 [call] v31'250 uv250 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:179 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 46 ==== osd_op_reply(179 gc.14 [call out=15b] v0'0 uv250 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:180 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 47 ==== osd_op_reply(180 gc.14 [call out=24b] v0'0 uv250 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:181 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 48 ==== osd_op_reply(181 gc.14 [call out=15b] v0'0 uv250 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:182 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 49 ==== osd_op_reply(182 gc.14 [call out=15b] v0'0 uv250 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.231+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:183 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.235+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 50 ==== osd_op_reply(183 gc.14 [call] v31'251 uv251 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.235+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-03-28T16:50:42.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.235+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:184 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 44 ==== osd_op_reply(184 gc.15 [call] v31'74 uv74 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:185 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 45 ==== osd_op_reply(185 gc.15 [call out=15b] v0'0 uv74 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:186 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 46 ==== osd_op_reply(186 gc.15 [call out=24b] v0'0 uv74 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:187 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 47 ==== osd_op_reply(187 gc.15 [call out=15b] v0'0 uv74 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:188 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 48 ==== osd_op_reply(188 gc.15 [call out=15b] v0'0 uv74 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.239+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:189 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.243+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 49 ==== osd_op_reply(189 gc.15 [call] v31'75 uv75 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.243+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-03-28T16:50:42.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.243+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:190 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.243+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 43 ==== osd_op_reply(190 gc.16 [call] v31'84 uv84 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.243+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:191 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 44 ==== osd_op_reply(191 gc.16 [call out=15b] v0'0 uv84 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:192 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 45 ==== osd_op_reply(192 gc.16 [call out=24b] v0'0 uv84 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:193 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 46 ==== osd_op_reply(193 gc.16 [call out=15b] v0'0 uv84 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:194 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 47 ==== osd_op_reply(194 gc.16 [call out=15b] v0'0 uv84 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:195 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 48 ==== osd_op_reply(195 gc.16 [call] v31'85 uv85 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-03-28T16:50:42.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.247+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:196 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 50 ==== osd_op_reply(196 gc.17 [call] v31'30 uv30 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:197 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 51 ==== osd_op_reply(197 gc.17 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:198 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 52 ==== osd_op_reply(198 gc.17 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:199 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 53 ==== osd_op_reply(199 gc.17 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:200 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 54 ==== osd_op_reply(200 gc.17 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.251+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:201 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.255+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 55 ==== osd_op_reply(201 gc.17 [call] v31'31 uv31 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.255+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-03-28T16:50:42.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.255+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:202 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.255+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 51 ==== osd_op_reply(202 gc.18 [call] v31'38 uv38 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.255+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:203 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 52 ==== osd_op_reply(203 gc.18 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:204 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 53 ==== osd_op_reply(204 gc.18 [call out=24b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:205 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 54 ==== osd_op_reply(205 gc.18 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:206 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 55 ==== osd_op_reply(206 gc.18 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.259+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:207 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.263+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 56 ==== osd_op_reply(207 gc.18 [call] v31'39 uv39 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.263+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-03-28T16:50:42.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.263+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:208 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.263+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 57 ==== osd_op_reply(208 gc.19 [call] v31'252 uv252 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.263+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:209 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 58 ==== osd_op_reply(209 gc.19 [call out=15b] v0'0 uv252 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:210 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 59 ==== osd_op_reply(210 gc.19 [call out=24b] v0'0 uv252 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:211 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 60 ==== osd_op_reply(211 gc.19 [call out=15b] v0'0 uv252 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:212 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 61 ==== osd_op_reply(212 gc.19 [call out=15b] v0'0 uv252 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.267+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:213 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712efd540 2024-03-28T16:50:42.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 62 ==== osd_op_reply(213 gc.19 [call] v31'253 uv253 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-03-28T16:50:42.272 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:214 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 56 ==== osd_op_reply(214 gc.20 [call] v31'76 uv76 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.274 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:215 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 57 ==== osd_op_reply(215 gc.20 [call out=15b] v0'0 uv76 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:216 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 58 ==== osd_op_reply(216 gc.20 [call out=24b] v0'0 uv76 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.271+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:217 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 59 ==== osd_op_reply(217 gc.20 [call out=15b] v0'0 uv76 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:218 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 60 ==== osd_op_reply(218 gc.20 [call out=15b] v0'0 uv76 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:219 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 61 ==== osd_op_reply(219 gc.20 [call] v31'77 uv77 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-03-28T16:50:42.279 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.275+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:220 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 49 ==== osd_op_reply(220 gc.21 [call] v31'40 uv40 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:221 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 50 ==== osd_op_reply(221 gc.21 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:222 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 51 ==== osd_op_reply(222 gc.21 [call out=24b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:223 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 52 ==== osd_op_reply(223 gc.21 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.279+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:224 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.283+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 53 ==== osd_op_reply(224 gc.21 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.283+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.283+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:225 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.283+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 54 ==== osd_op_reply(225 gc.21 [call] v31'41 uv41 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.283+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-03-28T16:50:42.285 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.283+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:226 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 62 ==== osd_op_reply(226 gc.22 [call] v31'33 uv33 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:227 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 63 ==== osd_op_reply(227 gc.22 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:228 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 64 ==== osd_op_reply(228 gc.22 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.289 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:229 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 65 ==== osd_op_reply(229 gc.22 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:230 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 66 ==== osd_op_reply(230 gc.22 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.287+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:231 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 67 ==== osd_op_reply(231 gc.22 [call] v31'34 uv34 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-03-28T16:50:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:232 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 68 ==== osd_op_reply(232 gc.23 [call] v31'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:233 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 69 ==== osd_op_reply(233 gc.23 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.291+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:234 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 70 ==== osd_op_reply(234 gc.23 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:235 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 71 ==== osd_op_reply(235 gc.23 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:236 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 72 ==== osd_op_reply(236 gc.23 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:237 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.295+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 73 ==== osd_op_reply(237 gc.23 [call] v31'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-03-28T16:50:42.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:238 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 74 ==== osd_op_reply(238 gc.24 [call] v31'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:239 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 75 ==== osd_op_reply(239 gc.24 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.302 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:240 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 76 ==== osd_op_reply(240 gc.24 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.303 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:241 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 77 ==== osd_op_reply(241 gc.24 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.299+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:242 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.303+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 78 ==== osd_op_reply(242 gc.24 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.303+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.304 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.303+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:243 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.303+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 79 ==== osd_op_reply(243 gc.24 [call] v31'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.303+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-03-28T16:50:42.306 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.303+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:244 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 80 ==== osd_op_reply(244 gc.25 [call] v31'55 uv55 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.308 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:245 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 81 ==== osd_op_reply(245 gc.25 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:246 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 82 ==== osd_op_reply(246 gc.25 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:247 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 83 ==== osd_op_reply(247 gc.25 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:248 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 84 ==== osd_op_reply(248 gc.25 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.311 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.307+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:249 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.312 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.311+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 85 ==== osd_op_reply(249 gc.25 [call] v31'56 uv56 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.311+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-03-28T16:50:42.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.311+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:250 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.311+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 49 ==== osd_op_reply(250 gc.26 [call] v31'34 uv34 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.311+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:251 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 50 ==== osd_op_reply(251 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:252 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 51 ==== osd_op_reply(252 gc.26 [call out=24b] v0'0 uv34 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:253 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 52 ==== osd_op_reply(253 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:254 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 53 ==== osd_op_reply(254 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.319 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.315+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:255 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.319+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 54 ==== osd_op_reply(255 gc.26 [call] v31'35 uv35 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.321 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.319+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-03-28T16:50:42.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.319+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:256 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 86 ==== osd_op_reply(256 gc.27 [call] v31'31 uv31 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:257 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 87 ==== osd_op_reply(257 gc.27 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:258 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 88 ==== osd_op_reply(258 gc.27 [call out=24b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:259 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 89 ==== osd_op_reply(259 gc.27 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:260 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 90 ==== osd_op_reply(260 gc.27 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.327 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.323+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:261 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.327+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 91 ==== osd_op_reply(261 gc.27 [call] v31'32 uv32 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.327+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-03-28T16:50:42.328 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.327+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:262 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 55 ==== osd_op_reply(262 gc.28 [call] v31'56 uv56 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.331 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:263 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 56 ==== osd_op_reply(263 gc.28 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:264 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 57 ==== osd_op_reply(264 gc.28 [call out=24b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.333 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:265 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 58 ==== osd_op_reply(265 gc.28 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.335 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.331+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:266 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.335+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 59 ==== osd_op_reply(266 gc.28 [call out=15b] v0'0 uv56 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.335+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.335+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:267 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.339+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 60 ==== osd_op_reply(267 gc.28 [call] v31'57 uv57 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.339+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-03-28T16:50:42.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.339+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:268 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.339+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 92 ==== osd_op_reply(268 gc.29 [call] v31'78 uv78 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.339+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:269 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 93 ==== osd_op_reply(269 gc.29 [call out=15b] v0'0 uv78 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:270 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 94 ==== osd_op_reply(270 gc.29 [call out=24b] v0'0 uv78 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:271 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 95 ==== osd_op_reply(271 gc.29 [call out=15b] v0'0 uv78 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.345 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:272 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 96 ==== osd_op_reply(272 gc.29 [call out=15b] v0'0 uv78 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.343+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:273 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f06790 2024-03-28T16:50:42.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.347+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 97 ==== osd_op_reply(273 gc.29 [call] v31'79 uv79 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.349 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.347+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-03-28T16:50:42.350 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.347+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:274 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.351+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 61 ==== osd_op_reply(274 gc.30 [call] v31'62 uv62 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.351+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:275 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.351+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 62 ==== osd_op_reply(275 gc.30 [call out=15b] v0'0 uv62 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.351+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.351+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:276 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.351+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 63 ==== osd_op_reply(276 gc.30 [call out=24b] v0'0 uv62 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.351+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:277 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.355+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 64 ==== osd_op_reply(277 gc.30 [call out=15b] v0'0 uv62 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.355+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.355+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:278 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.355+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 65 ==== osd_op_reply(278 gc.30 [call out=15b] v0'0 uv62 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.355+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.355+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:279 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x558712f01a80 2024-03-28T16:50:42.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.359+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 66 ==== osd_op_reply(279 gc.30 [call] v31'63 uv63 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.359+0000 7fb22a833980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-03-28T16:50:42.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.359+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:280 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.359+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 55 ==== osd_op_reply(280 gc.31 [call] v31'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.363 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.359+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:281 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 56 ==== osd_op_reply(281 gc.31 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:282 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 57 ==== osd_op_reply(282 gc.31 [call out=24b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:283 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 58 ==== osd_op_reply(283 gc.31 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-03-28T16:50:42.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:284 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 59 ==== osd_op_reply(284 gc.31 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb22a833980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-03-28T16:50:42.367 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.363+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:285 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f18a70 con 0x7fb21c00f3d0 2024-03-28T16:50:42.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.367+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 60 ==== osd_op_reply(285 gc.31 [call] v31'38 uv38 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.367+0000 7fb0de7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-03-28T16:50:42.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.367+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:286 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94038626728464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f11980 con 0x558712efd540 2024-03-28T16:50:42.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.371+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 63 ==== osd_op_reply(286 data_loggenerations_metadata [watch unwatch cookie 94038626728464] v31'254 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.373 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.371+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:287 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94038626706736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0fa10 con 0x7fb21c00f3d0 2024-03-28T16:50:42.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.375+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 61 ==== osd_op_reply(287 notify.0 [watch unwatch cookie 94038626706736] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.375+0000 7fb22a833980 20 remove_watcher() i=0 2024-03-28T16:50:42.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.375+0000 7fb22a833980 2 removed watcher, disabling cache 2024-03-28T16:50:42.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.375+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:288 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94038626711104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0fa10 con 0x558712efd540 2024-03-28T16:50:42.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.375+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 64 ==== osd_op_reply(288 notify.1 [watch unwatch cookie 94038626711104] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.375+0000 7fb22a833980 20 remove_watcher() i=1 2024-03-28T16:50:42.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.375+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- osd_op(unknown.0.0:289 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94038626714848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0b640 con 0x558712f06790 2024-03-28T16:50:42.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.379+0000 7fb227b4d640 1 -- 172.21.15.170:0/3847699318 <== osd.3 v2:172.21.15.170:6816/2102412426 98 ==== osd_op_reply(289 notify.2 [watch unwatch cookie 94038626714848] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb21c032040 con 0x558712f06790 2024-03-28T16:50:42.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.379+0000 7fb22a833980 20 remove_watcher() i=2 2024-03-28T16:50:42.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.379+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- osd_op(unknown.0.0:290 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94038626718640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0fa10 con 0x7fb21c00f3d0 2024-03-28T16:50:42.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.383+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.2 v2:172.21.15.170:6800/447751618 62 ==== osd_op_reply(290 notify.3 [watch unwatch cookie 94038626718640] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x7fb21c00f3d0 2024-03-28T16:50:42.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.383+0000 7fb22a833980 20 remove_watcher() i=3 2024-03-28T16:50:42.384 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.383+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- osd_op(unknown.0.0:291 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94038626722784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0b640 con 0x558712efd540 2024-03-28T16:50:42.386 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.383+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 <== osd.1 v2:172.21.15.170:6805/11349688 65 ==== osd_op_reply(291 notify.4 [watch unwatch cookie 94038626722784] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb22003e040 con 0x558712efd540 2024-03-28T16:50:42.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.383+0000 7fb22a833980 20 remove_watcher() i=4 2024-03-28T16:50:42.387 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.383+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:292 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94038626712160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0d3b0 con 0x558712f01a80 2024-03-28T16:50:42.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.387+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 67 ==== osd_op_reply(292 notify.5 [watch unwatch cookie 94038626712160] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.389 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.387+0000 7fb22a833980 20 remove_watcher() i=5 2024-03-28T16:50:42.390 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.387+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:293 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94038626727216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0bae0 con 0x558712f01a80 2024-03-28T16:50:42.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.391+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 68 ==== osd_op_reply(293 notify.6 [watch unwatch cookie 94038626727216] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.391+0000 7fb22a833980 20 remove_watcher() i=6 2024-03-28T16:50:42.392 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.391+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- osd_op(unknown.0.0:294 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94038626730688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) v8 -- 0x558712f0fa10 con 0x558712f01a80 2024-03-28T16:50:42.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.391+0000 7fb22834e640 1 -- 172.21.15.170:0/3847699318 <== osd.0 v2:172.21.15.119:6800/2028815515 69 ==== osd_op_reply(294 notify.7 [watch unwatch cookie 94038626730688] v31'249 uv247 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7fb2180e9070 con 0x558712f01a80 2024-03-28T16:50:42.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.391+0000 7fb22a833980 20 remove_watcher() i=7 2024-03-28T16:50:42.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712f01a80 msgr2=0x558712f03e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.395 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712f01a80 0x558712f03e70 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712efd540 msgr2=0x558712effa00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712efd540 0x558712effa00 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb21c00f3d0 msgr2=0x7fb21c00f810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb21c00f3d0 0x7fb21c00f810 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558712f06790 msgr2=0x558712f08c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.396 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x558712f06790 0x558712f08c50 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb0f0034300 msgr2=0x7fb0f00367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb0f0034300 0x7fb0f00367c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x558712ed0bb0 tx=0x7fb21c011000 comp rx=0 tx=0).stop 2024-03-28T16:50:42.399 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 msgr2=0x558712ef23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb228b4f640 1 -- 172.21.15.170:0/3847699318 reap_dead start 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712ef23c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb2180edfb0 tx=0x7fb2180d9f90 comp rx=0 tx=0).stop 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 shutdown_connections 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3847699318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712eeb700 0x558712ef23c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 >> 172.21.15.170:0/3847699318 conn(0x558712ee7a40 msgr2=0x558712ee50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 shutdown_connections 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3847699318 wait complete. 2024-03-28T16:50:42.400 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb100034260 msgr2=0x7fb100036720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb100034260 0x7fb100036720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x558712ec92c0 tx=0x7fb21c006000 comp rx=0 tx=0).stop 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712ede780 msgr2=0x558712edeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712ede780 0x558712edeba0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb2180e1d40 tx=0x7fb2180d0a10 comp rx=0 tx=0).stop 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 shutdown_connections 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb100034260 0x7fb100036720 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/2978763969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712ede780 0x558712edeba0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 >> 172.21.15.170:0/2978763969 conn(0x558712ed3590 msgr2=0x558712ed40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:42.401 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 shutdown_connections 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/2978763969 wait complete. 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712eca0f0 msgr2=0x558712ecc510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x558712eca0f0 0x558712ecc510 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712d40490 msgr2=0x558712ec7ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x558712d40490 0x558712ec7ec0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2180d3000 msgr2=0x558712ed0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fb2180d3000 0x558712ed0090 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb110034300 msgr2=0x7fb1100367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fb110034300 0x7fb1100367c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb21c000f70 tx=0x7fb21c001420 comp rx=0 tx=0).stop 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 msgr2=0x558712ec1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 --2- 172.21.15.170:0/3093734421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558712d3fb70 0x558712ec1290 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb21801b930 tx=0x7fb218017e30 comp rx=0 tx=0).stop 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb228b4f640 1 -- 172.21.15.170:0/3093734421 reap_dead start 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 shutdown_connections 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 >> 172.21.15.170:0/3093734421 conn(0x558712d3e850 msgr2=0x558712eb4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 shutdown_connections 2024-03-28T16:50:42.403 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-28T16:50:42.395+0000 7fb22a833980 1 -- 172.21.15.170:0/3093734421 wait complete. 2024-03-28T16:50:42.414 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-28T16:50:42.424 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-28T16:50:42.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:50:42.554 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f501e40b640 1 Processor -- start 2024-03-28T16:50:42.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f501e40b640 1 -- start start 2024-03-28T16:50:42.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f501e40b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5018130f30 0x7f5018133320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f501e40b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5018098c60 con 0x7f501812ac00 2024-03-28T16:50:42.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f501e40b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f501806e980 con 0x7f5018130f30 2024-03-28T16:50:42.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f5017fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f501812ac00 0x7f501812afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43780/0 (socket says 172.21.15.119:43780) 2024-03-28T16:50:42.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f5017fff640 1 -- 172.21.15.119:0/1277747588 learned_addr learned my addr 172.21.15.119:0/1277747588 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:42.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.550+0000 7f50177fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5018130f30 0x7f5018133320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f50177fe640 1 -- 172.21.15.119:0/1277747588 >> v1:172.21.15.119:6789/0 conn(0x7f501812ac00 legacy=0x7f501812afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f5016ffd640 1 -- 172.21.15.119:0/1277747588 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193663848 0 0) 0x7f5018098c60 con 0x7f501812ac00 2024-03-28T16:50:42.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f50177fe640 1 -- 172.21.15.119:0/1277747588 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501806f1f0 con 0x7f5018130f30 2024-03-28T16:50:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f50177fe640 1 --2- 172.21.15.119:0/1277747588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5018130f30 0x7f5018133320 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5008009740 tx=0x7f500801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3c1e248cd12881 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f5016ffd640 1 -- 172.21.15.119:0/1277747588 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50080ab530 con 0x7f5018130f30 2024-03-28T16:50:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f5016ffd640 1 -- 172.21.15.119:0/1277747588 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50080a7050 con 0x7f5018130f30 2024-03-28T16:50:42.558 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f5016ffd640 1 -- 172.21.15.119:0/1277747588 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50080ab7e0 con 0x7f5018130f30 2024-03-28T16:50:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f501e40b640 1 -- 172.21.15.119:0/1277747588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5018130f30 msgr2=0x7f5018133320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f501e40b640 1 --2- 172.21.15.119:0/1277747588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5018130f30 0x7f5018133320 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5008009740 tx=0x7f500801c370 comp rx=0 tx=0).stop 2024-03-28T16:50:42.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f501e40b640 1 -- 172.21.15.119:0/1277747588 shutdown_connections 2024-03-28T16:50:42.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f501e40b640 1 --2- 172.21.15.119:0/1277747588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5018130f30 0x7f5018133320 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f501e40b640 1 -- 172.21.15.119:0/1277747588 >> 172.21.15.119:0/1277747588 conn(0x7f501809c220 msgr2=0x7f5018071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:42.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f501e40b640 1 -- 172.21.15.119:0/1277747588 shutdown_connections 2024-03-28T16:50:42.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.554+0000 7f501e40b640 1 -- 172.21.15.119:0/1277747588 wait complete. 2024-03-28T16:50:42.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f501e40b640 1 Processor -- start 2024-03-28T16:50:42.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f501e40b640 1 -- start start 2024-03-28T16:50:42.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f501e40b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501812ac00 0x7f501812c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f501e40b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50180992b0 con 0x7f501812ac00 2024-03-28T16:50:42.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5017fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501812ac00 0x7f501812c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5017fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501812ac00 0x7f501812c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34090/0 (socket says 172.21.15.119:34090) 2024-03-28T16:50:42.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5017fff640 1 -- 172.21.15.119:0/2869450616 learned_addr learned my addr 172.21.15.119:0/2869450616 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:42.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5017fff640 1 -- 172.21.15.119:0/2869450616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008009420 con 0x7f501812ac00 2024-03-28T16:50:42.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5017fff640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501812ac00 0x7f501812c980 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f500000c420 tx=0x7f500000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5014ff9640 1 -- 172.21.15.119:0/2869450616 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50000074d0 con 0x7f501812ac00 2024-03-28T16:50:42.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5014ff9640 1 -- 172.21.15.119:0/2869450616 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5000007630 con 0x7f501812ac00 2024-03-28T16:50:42.563 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50180999d0 con 0x7f501812ac00 2024-03-28T16:50:42.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5014ff9640 1 -- 172.21.15.119:0/2869450616 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5000004960 con 0x7f501812ac00 2024-03-28T16:50:42.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5018098ec0 con 0x7f501812ac00 2024-03-28T16:50:42.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5014ff9640 1 -- 172.21.15.119:0/2869450616 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f5000004ac0 con 0x7f501812ac00 2024-03-28T16:50:42.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50180996f0 con 0x7f501812ac00 2024-03-28T16:50:42.564 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5014ff9640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ff8035a10 0x7f4ff8037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:42.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f5014ff9640 1 -- 172.21.15.119:0/2869450616 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f5000047c60 con 0x7f501812ac00 2024-03-28T16:50:42.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.558+0000 7f50177fe640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ff8035a10 0x7f4ff8037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:42.565 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.562+0000 7f50177fe640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ff8035a10 0x7f4ff8037ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f50080ad000 tx=0x7f5008008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:42.567 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.562+0000 7f5014ff9640 1 -- 172.21.15.119:0/2869450616 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f500001c8d0 con 0x7f501812ac00 2024-03-28T16:50:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.718+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5018062860 con 0x7f4ff8035a10 2024-03-28T16:50:42.887 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.886+0000 7f5014ff9640 1 -- 172.21.15.119:0/2869450616 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+673101 (secure 0 0 0) 0x7f5018062860 con 0x7f4ff8035a10 2024-03-28T16:50:42.895 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:50:42.901 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:50:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ff8035a10 msgr2=0x7f4ff8037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ff8035a10 0x7f4ff8037ed0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f50080ad000 tx=0x7f5008008040 comp rx=0 tx=0).stop 2024-03-28T16:50:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501812ac00 msgr2=0x7f501812c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501812ac00 0x7f501812c980 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f500000c420 tx=0x7f500000c8e0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 shutdown_connections 2024-03-28T16:50:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4ff8035a10 0x7f4ff8037ed0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 --2- 172.21.15.119:0/2869450616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f501812ac00 0x7f501812c980 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 >> 172.21.15.119:0/2869450616 conn(0x7f501809c220 msgr2=0x7f5018128370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 shutdown_connections 2024-03-28T16:50:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:42.894+0000 7f501e40b640 1 -- 172.21.15.119:0/2869450616 wait complete. 2024-03-28T16:50:42.929 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-03-28T16:50:42.533657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":3824,"num_read_kb":3471,"num_write":1479,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3745,"ondisk_log_size":3745,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":160768,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":157945,"kb_avail":377326592,"statfs":{"total":386547056640,"available":386382430208,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":161736470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":403,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":211,"num_read_kb":206,"num_write":80,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006262"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600005+0000","last_change":"2024-03-28T16:48:54.656021+0000","last_active":"2024-03-28T16:49:37.600005+0000","last_peered":"2024-03-28T16:49:37.600005+0000","last_clean":"2024-03-28T16:49:37.600005+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:49:37.600005+0000","last_undegraded":"2024-03-28T16:49:37.600005+0000","last_fullsized":"2024-03-28T16:49:37.600005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585246+0000","last_change":"2024-03-28T16:48:56.687475+0000","last_active":"2024-03-28T16:49:37.585246+0000","last_peered":"2024-03-28T16:49:37.585246+0000","last_clean":"2024-03-28T16:49:37.585246+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:49:37.585246+0000","last_undegraded":"2024-03-28T16:49:37.585246+0000","last_fullsized":"2024-03-28T16:49:37.585246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587869+0000","last_change":"2024-03-28T16:48:54.666372+0000","last_active":"2024-03-28T16:49:37.587869+0000","last_peered":"2024-03-28T16:49:37.587869+0000","last_clean":"2024-03-28T16:49:37.587869+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:49:37.587869+0000","last_undegraded":"2024-03-28T16:49:37.587869+0000","last_fullsized":"2024-03-28T16:49:37.587869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555560+0000","last_change":"2024-03-28T16:48:56.688050+0000","last_active":"2024-03-28T16:49:40.555560+0000","last_peered":"2024-03-28T16:49:40.555560+0000","last_clean":"2024-03-28T16:49:40.555560+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:49:40.555560+0000","last_undegraded":"2024-03-28T16:49:40.555560+0000","last_fullsized":"2024-03-28T16:49:40.555560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:32.377984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592873+0000","last_change":"2024-03-28T16:48:54.665976+0000","last_active":"2024-03-28T16:49:37.592873+0000","last_peered":"2024-03-28T16:49:37.592873+0000","last_clean":"2024-03-28T16:49:37.592873+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:49:37.592873+0000","last_undegraded":"2024-03-28T16:49:37.592873+0000","last_fullsized":"2024-03-28T16:49:37.592873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:38.265023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370127+0000","last_change":"2024-03-28T16:48:56.690734+0000","last_active":"2024-03-28T16:50:22.370127+0000","last_peered":"2024-03-28T16:50:22.370127+0000","last_clean":"2024-03-28T16:50:22.370127+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:50:22.370127+0000","last_undegraded":"2024-03-28T16:50:22.370127+0000","last_fullsized":"2024-03-28T16:50:22.370127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:26.690845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.374097+0000","last_change":"2024-03-28T16:48:54.656958+0000","last_active":"2024-03-28T16:50:22.374097+0000","last_peered":"2024-03-28T16:50:22.374097+0000","last_clean":"2024-03-28T16:50:22.374097+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:50:22.374097+0000","last_undegraded":"2024-03-28T16:50:22.374097+0000","last_fullsized":"2024-03-28T16:50:22.374097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.564230+0000","last_change":"2024-03-28T16:49:37.602262+0000","last_active":"2024-03-28T16:49:40.564230+0000","last_peered":"2024-03-28T16:49:40.564230+0000","last_clean":"2024-03-28T16:49:40.564230+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:49:40.564230+0000","last_undegraded":"2024-03-28T16:49:40.564230+0000","last_fullsized":"2024-03-28T16:49:40.564230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:12:52.595685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593536+0000","last_change":"2024-03-28T16:48:54.661971+0000","last_active":"2024-03-28T16:49:37.593536+0000","last_peered":"2024-03-28T16:49:37.593536+0000","last_clean":"2024-03-28T16:49:37.593536+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:49:37.593536+0000","last_undegraded":"2024-03-28T16:49:37.593536+0000","last_fullsized":"2024-03-28T16:49:37.593536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:29.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585408+0000","last_change":"2024-03-28T16:48:56.671464+0000","last_active":"2024-03-28T16:49:37.585408+0000","last_peered":"2024-03-28T16:49:37.585408+0000","last_clean":"2024-03-28T16:49:37.585408+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:49:37.585408+0000","last_undegraded":"2024-03-28T16:49:37.585408+0000","last_fullsized":"2024-03-28T16:49:37.585408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:18.173627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590098+0000","last_change":"2024-03-28T16:48:54.665454+0000","last_active":"2024-03-28T16:49:37.590098+0000","last_peered":"2024-03-28T16:49:37.590098+0000","last_clean":"2024-03-28T16:49:37.590098+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:49:37.590098+0000","last_undegraded":"2024-03-28T16:49:37.590098+0000","last_fullsized":"2024-03-28T16:49:37.590098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587928+0000","last_change":"2024-03-28T16:48:56.688375+0000","last_active":"2024-03-28T16:49:37.587928+0000","last_peered":"2024-03-28T16:49:37.587928+0000","last_clean":"2024-03-28T16:49:37.587928+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:49:37.587928+0000","last_undegraded":"2024-03-28T16:49:37.587928+0000","last_fullsized":"2024-03-28T16:49:37.587928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:56.236071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.859167+0000","last_change":"2024-03-28T16:48:54.672203+0000","last_active":"2024-03-28T16:49:39.859167+0000","last_peered":"2024-03-28T16:49:39.859167+0000","last_clean":"2024-03-28T16:49:39.859167+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:49:39.859167+0000","last_undegraded":"2024-03-28T16:49:39.859167+0000","last_fullsized":"2024-03-28T16:49:39.859167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:04:05.939269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555495+0000","last_change":"2024-03-28T16:48:56.695671+0000","last_active":"2024-03-28T16:49:40.555495+0000","last_peered":"2024-03-28T16:49:40.555495+0000","last_clean":"2024-03-28T16:49:40.555495+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:49:40.555495+0000","last_undegraded":"2024-03-28T16:49:40.555495+0000","last_fullsized":"2024-03-28T16:49:40.555495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585143+0000","last_change":"2024-03-28T16:48:54.658053+0000","last_active":"2024-03-28T16:49:37.585143+0000","last_peered":"2024-03-28T16:49:37.585143+0000","last_clean":"2024-03-28T16:49:37.585143+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:49:37.585143+0000","last_undegraded":"2024-03-28T16:49:37.585143+0000","last_fullsized":"2024-03-28T16:49:37.585143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839573+0000","last_change":"2024-03-28T16:48:56.681114+0000","last_active":"2024-03-28T16:50:28.839573+0000","last_peered":"2024-03-28T16:50:28.839573+0000","last_clean":"2024-03-28T16:50:28.839573+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:50:28.839573+0000","last_undegraded":"2024-03-28T16:50:28.839573+0000","last_fullsized":"2024-03-28T16:50:28.839573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527185+0000","last_change":"2024-03-28T16:48:56.677244+0000","last_active":"2024-03-28T16:49:39.527185+0000","last_peered":"2024-03-28T16:49:39.527185+0000","last_clean":"2024-03-28T16:49:39.527185+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:49:39.527185+0000","last_undegraded":"2024-03-28T16:49:39.527185+0000","last_fullsized":"2024-03-28T16:49:39.527185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:55.139244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602851+0000","last_change":"2024-03-28T16:48:54.671518+0000","last_active":"2024-03-28T16:49:37.602851+0000","last_peered":"2024-03-28T16:49:37.602851+0000","last_clean":"2024-03-28T16:49:37.602851+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:49:37.602851+0000","last_undegraded":"2024-03-28T16:49:37.602851+0000","last_fullsized":"2024-03-28T16:49:37.602851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'9","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.563949+0000","last_change":"2024-03-28T16:48:56.692955+0000","last_active":"2024-03-28T16:49:40.563949+0000","last_peered":"2024-03-28T16:49:40.563949+0000","last_clean":"2024-03-28T16:49:40.563949+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:49:40.563949+0000","last_undegraded":"2024-03-28T16:49:40.563949+0000","last_fullsized":"2024-03-28T16:49:40.563949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610918+0000","last_change":"2024-03-28T16:49:37.610918+0000","last_active":"2024-03-28T16:49:37.610918+0000","last_peered":"2024-03-28T16:49:37.610918+0000","last_clean":"2024-03-28T16:49:37.610918+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:49:37.610918+0000","last_undegraded":"2024-03-28T16:49:37.610918+0000","last_fullsized":"2024-03-28T16:49:37.610918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:21.127441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840116+0000","last_change":"2024-03-28T16:48:56.687940+0000","last_active":"2024-03-28T16:50:28.840116+0000","last_peered":"2024-03-28T16:50:28.840116+0000","last_clean":"2024-03-28T16:50:28.840116+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:50:28.840116+0000","last_undegraded":"2024-03-28T16:50:28.840116+0000","last_fullsized":"2024-03-28T16:50:28.840116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:40:18.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586038+0000","last_change":"2024-03-28T16:48:54.670148+0000","last_active":"2024-03-28T16:49:37.586038+0000","last_peered":"2024-03-28T16:49:37.586038+0000","last_clean":"2024-03-28T16:49:37.586038+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:49:37.586038+0000","last_undegraded":"2024-03-28T16:49:37.586038+0000","last_fullsized":"2024-03-28T16:49:37.586038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.865044+0000","last_change":"2024-03-28T16:48:56.678642+0000","last_active":"2024-03-28T16:49:39.865044+0000","last_peered":"2024-03-28T16:49:39.865044+0000","last_clean":"2024-03-28T16:49:39.865044+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:49:39.865044+0000","last_undegraded":"2024-03-28T16:49:39.865044+0000","last_fullsized":"2024-03-28T16:49:39.865044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586134+0000","last_change":"2024-03-28T16:48:54.655014+0000","last_active":"2024-03-28T16:49:37.586134+0000","last_peered":"2024-03-28T16:49:37.586134+0000","last_clean":"2024-03-28T16:49:37.586134+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:49:37.586134+0000","last_undegraded":"2024-03-28T16:49:37.586134+0000","last_fullsized":"2024-03-28T16:49:37.586134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.876760+0000","last_change":"2024-03-28T16:48:56.680884+0000","last_active":"2024-03-28T16:49:39.876760+0000","last_peered":"2024-03-28T16:49:39.876760+0000","last_clean":"2024-03-28T16:49:39.876760+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:49:39.876760+0000","last_undegraded":"2024-03-28T16:49:39.876760+0000","last_fullsized":"2024-03-28T16:49:39.876760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.842066+0000","last_change":"2024-03-28T16:48:54.670541+0000","last_active":"2024-03-28T16:49:39.842066+0000","last_peered":"2024-03-28T16:49:39.842066+0000","last_clean":"2024-03-28T16:49:39.842066+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:49:39.842066+0000","last_undegraded":"2024-03-28T16:49:39.842066+0000","last_fullsized":"2024-03-28T16:49:39.842066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854794+0000","last_change":"2024-03-28T16:48:56.671588+0000","last_active":"2024-03-28T16:50:28.854794+0000","last_peered":"2024-03-28T16:50:28.854794+0000","last_clean":"2024-03-28T16:50:28.854794+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:50:28.854794+0000","last_undegraded":"2024-03-28T16:50:28.854794+0000","last_fullsized":"2024-03-28T16:50:28.854794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:22.649572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582658+0000","last_change":"2024-03-28T16:48:54.673730+0000","last_active":"2024-03-28T16:49:37.582658+0000","last_peered":"2024-03-28T16:49:37.582658+0000","last_clean":"2024-03-28T16:49:37.582658+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:49:37.582658+0000","last_undegraded":"2024-03-28T16:49:37.582658+0000","last_fullsized":"2024-03-28T16:49:37.582658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:36:08.968340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'19","reported_seq":77,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.826736+0000","last_change":"2024-03-28T16:49:04.742579+0000","last_active":"2024-03-28T16:50:28.826736+0000","last_peered":"2024-03-28T16:50:28.826736+0000","last_clean":"2024-03-28T16:50:28.826736+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:50:28.826736+0000","last_undegraded":"2024-03-28T16:50:28.826736+0000","last_fullsized":"2024-03-28T16:50:28.826736+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:00.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839392+0000","last_change":"2024-03-28T16:48:56.683934+0000","last_active":"2024-03-28T16:50:28.839392+0000","last_peered":"2024-03-28T16:50:28.839392+0000","last_clean":"2024-03-28T16:50:28.839392+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:50:28.839392+0000","last_undegraded":"2024-03-28T16:50:28.839392+0000","last_fullsized":"2024-03-28T16:50:28.839392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:38.624478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592554+0000","last_change":"2024-03-28T16:48:54.670023+0000","last_active":"2024-03-28T16:49:37.592554+0000","last_peered":"2024-03-28T16:49:37.592554+0000","last_clean":"2024-03-28T16:49:37.592554+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:49:37.592554+0000","last_undegraded":"2024-03-28T16:49:37.592554+0000","last_fullsized":"2024-03-28T16:49:37.592554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587566+0000","last_change":"2024-03-28T16:48:58.699910+0000","last_active":"2024-03-28T16:49:37.587566+0000","last_peered":"2024-03-28T16:49:37.587566+0000","last_clean":"2024-03-28T16:49:37.587566+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:49:37.587566+0000","last_undegraded":"2024-03-28T16:49:37.587566+0000","last_fullsized":"2024-03-28T16:49:37.587566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:35.076776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.149453+0000","last_change":"2024-03-28T16:49:00.727669+0000","last_active":"2024-03-28T16:50:23.149453+0000","last_peered":"2024-03-28T16:50:23.149453+0000","last_clean":"2024-03-28T16:50:23.149453+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:50:23.149453+0000","last_undegraded":"2024-03-28T16:50:23.149453+0000","last_fullsized":"2024-03-28T16:50:23.149453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:28.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601464+0000","last_change":"2024-03-28T16:49:02.739439+0000","last_active":"2024-03-28T16:49:37.601464+0000","last_peered":"2024-03-28T16:49:37.601464+0000","last_clean":"2024-03-28T16:49:37.601464+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:49:37.601464+0000","last_undegraded":"2024-03-28T16:49:37.601464+0000","last_fullsized":"2024-03-28T16:49:37.601464+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'6","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.307391+0000","last_change":"2024-03-28T16:49:04.742984+0000","last_active":"2024-03-28T16:50:17.307391+0000","last_peered":"2024-03-28T16:50:17.307391+0000","last_clean":"2024-03-28T16:50:17.307391+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:50:17.307391+0000","last_undegraded":"2024-03-28T16:50:17.307391+0000","last_fullsized":"2024-03-28T16:50:17.307391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:30.200369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839461+0000","last_change":"2024-03-28T16:48:56.671488+0000","last_active":"2024-03-28T16:50:28.839461+0000","last_peered":"2024-03-28T16:50:28.839461+0000","last_clean":"2024-03-28T16:50:28.839461+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:50:28.839461+0000","last_undegraded":"2024-03-28T16:50:28.839461+0000","last_fullsized":"2024-03-28T16:50:28.839461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:39.819684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591228+0000","last_change":"2024-03-28T16:48:54.655701+0000","last_active":"2024-03-28T16:49:37.591228+0000","last_peered":"2024-03-28T16:49:37.591228+0000","last_clean":"2024-03-28T16:49:37.591228+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:49:37.591228+0000","last_undegraded":"2024-03-28T16:49:37.591228+0000","last_fullsized":"2024-03-28T16:49:37.591228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585880+0000","last_change":"2024-03-28T16:48:58.689758+0000","last_active":"2024-03-28T16:49:37.585880+0000","last_peered":"2024-03-28T16:49:37.585880+0000","last_clean":"2024-03-28T16:49:37.585880+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:49:37.585880+0000","last_undegraded":"2024-03-28T16:49:37.585880+0000","last_fullsized":"2024-03-28T16:49:37.585880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:08.600334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"29'7","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.605037+0000","last_change":"2024-03-28T16:49:00.717460+0000","last_active":"2024-03-28T16:49:37.605037+0000","last_peered":"2024-03-28T16:49:37.605037+0000","last_clean":"2024-03-28T16:49:37.605037+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:49:37.605037+0000","last_undegraded":"2024-03-28T16:49:37.605037+0000","last_fullsized":"2024-03-28T16:49:37.605037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:26.416340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586231+0000","last_change":"2024-03-28T16:49:02.730400+0000","last_active":"2024-03-28T16:49:37.586231+0000","last_peered":"2024-03-28T16:49:37.586231+0000","last_clean":"2024-03-28T16:49:37.586231+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:49:37.586231+0000","last_undegraded":"2024-03-28T16:49:37.586231+0000","last_fullsized":"2024-03-28T16:49:37.586231+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584337+0000","last_change":"2024-03-28T16:49:04.743161+0000","last_active":"2024-03-28T16:49:37.584337+0000","last_peered":"2024-03-28T16:49:37.584337+0000","last_clean":"2024-03-28T16:49:37.584337+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:49:37.584337+0000","last_undegraded":"2024-03-28T16:49:37.584337+0000","last_fullsized":"2024-03-28T16:49:37.584337+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189874+0000","last_change":"2024-03-28T16:48:56.676331+0000","last_active":"2024-03-28T16:50:23.189874+0000","last_peered":"2024-03-28T16:50:23.189874+0000","last_clean":"2024-03-28T16:50:23.189874+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:50:23.189874+0000","last_undegraded":"2024-03-28T16:50:23.189874+0000","last_fullsized":"2024-03-28T16:50:23.189874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:33.697214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592955+0000","last_change":"2024-03-28T16:48:54.659330+0000","last_active":"2024-03-28T16:49:37.592955+0000","last_peered":"2024-03-28T16:49:37.592955+0000","last_clean":"2024-03-28T16:49:37.592955+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:49:37.592955+0000","last_undegraded":"2024-03-28T16:49:37.592955+0000","last_fullsized":"2024-03-28T16:49:37.592955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:30:54.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'5","reported_seq":203,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.364471+0000","last_change":"2024-03-28T16:48:58.687156+0000","last_active":"2024-03-28T16:50:30.364471+0000","last_peered":"2024-03-28T16:50:30.364471+0000","last_clean":"2024-03-28T16:50:30.364471+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:50:30.364471+0000","last_undegraded":"2024-03-28T16:50:30.364471+0000","last_fullsized":"2024-03-28T16:50:30.364471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:25:52.416012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":267,"num_read_kb":266,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'11","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595817+0000","last_change":"2024-03-28T16:49:00.704257+0000","last_active":"2024-03-28T16:49:37.595817+0000","last_peered":"2024-03-28T16:49:37.595817+0000","last_clean":"2024-03-28T16:49:37.595817+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:49:37.595817+0000","last_undegraded":"2024-03-28T16:49:37.595817+0000","last_fullsized":"2024-03-28T16:49:37.595817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:59.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'246","reported_seq":329,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854124+0000","last_change":"2024-03-28T16:49:02.737451+0000","last_active":"2024-03-28T16:50:33.854124+0000","last_peered":"2024-03-28T16:50:33.854124+0000","last_clean":"2024-03-28T16:50:33.854124+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:50:33.854124+0000","last_undegraded":"2024-03-28T16:50:33.854124+0000","last_fullsized":"2024-03-28T16:50:33.854124+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:34.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591631+0000","last_change":"2024-03-28T16:49:04.741486+0000","last_active":"2024-03-28T16:49:37.591631+0000","last_peered":"2024-03-28T16:49:37.591631+0000","last_clean":"2024-03-28T16:49:37.591631+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:49:37.591631+0000","last_undegraded":"2024-03-28T16:49:37.591631+0000","last_fullsized":"2024-03-28T16:49:37.591631+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:51.610298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591048+0000","last_change":"2024-03-28T16:48:56.697774+0000","last_active":"2024-03-28T16:49:37.591048+0000","last_peered":"2024-03-28T16:49:37.591048+0000","last_clean":"2024-03-28T16:49:37.591048+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:49:37.591048+0000","last_undegraded":"2024-03-28T16:49:37.591048+0000","last_fullsized":"2024-03-28T16:49:37.591048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583578+0000","last_change":"2024-03-28T16:48:54.659035+0000","last_active":"2024-03-28T16:49:37.583578+0000","last_peered":"2024-03-28T16:49:37.583578+0000","last_clean":"2024-03-28T16:49:37.583578+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:49:37.583578+0000","last_undegraded":"2024-03-28T16:49:37.583578+0000","last_fullsized":"2024-03-28T16:49:37.583578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:59.516240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594556+0000","last_change":"2024-03-28T16:48:58.706776+0000","last_active":"2024-03-28T16:49:37.594556+0000","last_peered":"2024-03-28T16:49:37.594556+0000","last_clean":"2024-03-28T16:49:37.594556+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:49:37.594556+0000","last_undegraded":"2024-03-28T16:49:37.594556+0000","last_fullsized":"2024-03-28T16:49:37.594556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:18.066504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.142398+0000","last_change":"2024-03-28T16:49:00.710996+0000","last_active":"2024-03-28T16:50:23.142398+0000","last_peered":"2024-03-28T16:50:23.142398+0000","last_clean":"2024-03-28T16:50:23.142398+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:23.142398+0000","last_undegraded":"2024-03-28T16:50:23.142398+0000","last_fullsized":"2024-03-28T16:50:23.142398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:05.450361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'246","reported_seq":344,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.481011+0000","last_change":"2024-03-28T16:49:02.734376+0000","last_active":"2024-03-28T16:50:35.481011+0000","last_peered":"2024-03-28T16:50:35.481011+0000","last_clean":"2024-03-28T16:50:35.481011+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:50:35.481011+0000","last_undegraded":"2024-03-28T16:50:35.481011+0000","last_fullsized":"2024-03-28T16:50:35.481011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581627+0000","last_change":"2024-03-28T16:48:56.693376+0000","last_active":"2024-03-28T16:49:37.581627+0000","last_peered":"2024-03-28T16:49:37.581627+0000","last_clean":"2024-03-28T16:49:37.581627+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:49:37.581627+0000","last_undegraded":"2024-03-28T16:49:37.581627+0000","last_fullsized":"2024-03-28T16:49:37.581627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:27.504691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600235+0000","last_change":"2024-03-28T16:48:54.671982+0000","last_active":"2024-03-28T16:49:37.600235+0000","last_peered":"2024-03-28T16:49:37.600235+0000","last_clean":"2024-03-28T16:49:37.600235+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:49:37.600235+0000","last_undegraded":"2024-03-28T16:49:37.600235+0000","last_fullsized":"2024-03-28T16:49:37.600235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586619+0000","last_change":"2024-03-28T16:48:58.687997+0000","last_active":"2024-03-28T16:49:37.586619+0000","last_peered":"2024-03-28T16:49:37.586619+0000","last_clean":"2024-03-28T16:49:37.586619+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:49:37.586619+0000","last_undegraded":"2024-03-28T16:49:37.586619+0000","last_fullsized":"2024-03-28T16:49:37.586619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:29.519686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'38","reported_seq":84,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.155887+0000","last_change":"2024-03-28T16:49:00.713077+0000","last_active":"2024-03-28T16:50:23.155887+0000","last_peered":"2024-03-28T16:50:23.155887+0000","last_clean":"2024-03-28T16:50:23.155887+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:23.155887+0000","last_undegraded":"2024-03-28T16:50:23.155887+0000","last_fullsized":"2024-03-28T16:50:23.155887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:46.727245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587114+0000","last_change":"2024-03-28T16:49:02.730507+0000","last_active":"2024-03-28T16:49:37.587114+0000","last_peered":"2024-03-28T16:49:37.587114+0000","last_clean":"2024-03-28T16:49:37.587114+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:49:37.587114+0000","last_undegraded":"2024-03-28T16:49:37.587114+0000","last_fullsized":"2024-03-28T16:49:37.587114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.935069+0000","last_change":"2024-03-28T16:49:04.728316+0000","last_active":"2024-03-28T16:50:22.935069+0000","last_peered":"2024-03-28T16:50:22.935069+0000","last_clean":"2024-03-28T16:50:22.935069+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:22.935069+0000","last_undegraded":"2024-03-28T16:50:22.935069+0000","last_fullsized":"2024-03-28T16:50:22.935069+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:28.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'22","reported_seq":100,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729800+0000","last_change":"2024-03-28T16:48:56.678278+0000","last_active":"2024-03-28T16:50:22.729800+0000","last_peered":"2024-03-28T16:50:22.729800+0000","last_clean":"2024-03-28T16:50:22.729800+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:50:22.729800+0000","last_undegraded":"2024-03-28T16:50:22.729800+0000","last_fullsized":"2024-03-28T16:50:22.729800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.051186+0000","last_change":"2024-03-28T16:48:54.672277+0000","last_active":"2024-03-28T16:49:39.051186+0000","last_peered":"2024-03-28T16:49:39.051186+0000","last_clean":"2024-03-28T16:49:39.051186+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:49:39.051186+0000","last_undegraded":"2024-03-28T16:49:39.051186+0000","last_fullsized":"2024-03-28T16:49:39.051186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585905+0000","last_change":"2024-03-28T16:48:58.690263+0000","last_active":"2024-03-28T16:49:37.585905+0000","last_peered":"2024-03-28T16:49:37.585905+0000","last_clean":"2024-03-28T16:49:37.585905+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:49:37.585905+0000","last_undegraded":"2024-03-28T16:49:37.585905+0000","last_fullsized":"2024-03-28T16:49:37.585905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:25.224033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.129187+0000","last_change":"2024-03-28T16:49:00.727195+0000","last_active":"2024-03-28T16:50:23.129187+0000","last_peered":"2024-03-28T16:50:23.129187+0000","last_clean":"2024-03-28T16:50:23.129187+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:50:23.129187+0000","last_undegraded":"2024-03-28T16:50:23.129187+0000","last_fullsized":"2024-03-28T16:50:23.129187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:48.531527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585871+0000","last_change":"2024-03-28T16:49:02.714319+0000","last_active":"2024-03-28T16:49:37.585871+0000","last_peered":"2024-03-28T16:49:37.585871+0000","last_clean":"2024-03-28T16:49:37.585871+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:49:37.585871+0000","last_undegraded":"2024-03-28T16:49:37.585871+0000","last_fullsized":"2024-03-28T16:49:37.585871+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:49.522623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'3","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.157047+0000","last_change":"2024-03-28T16:49:04.738375+0000","last_active":"2024-03-28T16:50:17.157047+0000","last_peered":"2024-03-28T16:50:17.157047+0000","last_clean":"2024-03-28T16:50:17.157047+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:50:17.157047+0000","last_undegraded":"2024-03-28T16:50:17.157047+0000","last_fullsized":"2024-03-28T16:50:17.157047+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:31.198874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'3","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.537250+0000","last_change":"2024-03-28T16:48:56.681600+0000","last_active":"2024-03-28T16:49:39.537250+0000","last_peered":"2024-03-28T16:49:39.537250+0000","last_clean":"2024-03-28T16:49:39.537250+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:49:39.537250+0000","last_undegraded":"2024-03-28T16:49:39.537250+0000","last_fullsized":"2024-03-28T16:49:39.537250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588203+0000","last_change":"2024-03-28T16:48:54.676553+0000","last_active":"2024-03-28T16:49:37.588203+0000","last_peered":"2024-03-28T16:49:37.588203+0000","last_clean":"2024-03-28T16:49:37.588203+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:49:37.588203+0000","last_undegraded":"2024-03-28T16:49:37.588203+0000","last_fullsized":"2024-03-28T16:49:37.588203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:11.709923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592044+0000","last_change":"2024-03-28T16:48:58.682276+0000","last_active":"2024-03-28T16:49:37.592044+0000","last_peered":"2024-03-28T16:49:37.592044+0000","last_clean":"2024-03-28T16:49:37.592044+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:49:37.592044+0000","last_undegraded":"2024-03-28T16:49:37.592044+0000","last_fullsized":"2024-03-28T16:49:37.592044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:44.713820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.158232+0000","last_change":"2024-03-28T16:49:00.720840+0000","last_active":"2024-03-28T16:50:23.158232+0000","last_peered":"2024-03-28T16:50:23.158232+0000","last_clean":"2024-03-28T16:50:23.158232+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:50:23.158232+0000","last_undegraded":"2024-03-28T16:50:23.158232+0000","last_fullsized":"2024-03-28T16:50:23.158232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:28.746453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'246","reported_seq":339,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854481+0000","last_change":"2024-03-28T16:49:02.713412+0000","last_active":"2024-03-28T16:50:33.854481+0000","last_peered":"2024-03-28T16:50:33.854481+0000","last_clean":"2024-03-28T16:50:33.854481+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:33.854481+0000","last_undegraded":"2024-03-28T16:50:33.854481+0000","last_fullsized":"2024-03-28T16:50:33.854481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.828873+0000","last_change":"2024-03-28T16:49:04.743207+0000","last_active":"2024-03-28T16:50:28.828873+0000","last_peered":"2024-03-28T16:50:28.828873+0000","last_clean":"2024-03-28T16:50:28.828873+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:50:28.828873+0000","last_undegraded":"2024-03-28T16:50:28.828873+0000","last_fullsized":"2024-03-28T16:50:28.828873+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:12.429622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.864959+0000","last_change":"2024-03-28T16:48:56.685083+0000","last_active":"2024-03-28T16:49:39.864959+0000","last_peered":"2024-03-28T16:49:39.864959+0000","last_clean":"2024-03-28T16:49:39.864959+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:49:39.864959+0000","last_undegraded":"2024-03-28T16:49:39.864959+0000","last_fullsized":"2024-03-28T16:49:39.864959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587111+0000","last_change":"2024-03-28T16:48:54.658093+0000","last_active":"2024-03-28T16:49:37.587111+0000","last_peered":"2024-03-28T16:49:37.587111+0000","last_clean":"2024-03-28T16:49:37.587111+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:49:37.587111+0000","last_undegraded":"2024-03-28T16:49:37.587111+0000","last_fullsized":"2024-03-28T16:49:37.587111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:20.809568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586227+0000","last_change":"2024-03-28T16:48:58.682237+0000","last_active":"2024-03-28T16:49:37.586227+0000","last_peered":"2024-03-28T16:49:37.586227+0000","last_clean":"2024-03-28T16:49:37.586227+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:49:37.586227+0000","last_undegraded":"2024-03-28T16:49:37.586227+0000","last_fullsized":"2024-03-28T16:49:37.586227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:37.069394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'50","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.781121+0000","last_change":"2024-03-28T16:49:00.725827+0000","last_active":"2024-03-28T16:50:28.781121+0000","last_peered":"2024-03-28T16:50:28.781121+0000","last_clean":"2024-03-28T16:50:28.781121+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:50:28.781121+0000","last_undegraded":"2024-03-28T16:50:28.781121+0000","last_fullsized":"2024-03-28T16:50:28.781121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:56.316763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585610+0000","last_change":"2024-03-28T16:49:02.712054+0000","last_active":"2024-03-28T16:49:37.585610+0000","last_peered":"2024-03-28T16:49:37.585610+0000","last_clean":"2024-03-28T16:49:37.585610+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:49:37.585610+0000","last_undegraded":"2024-03-28T16:49:37.585610+0000","last_fullsized":"2024-03-28T16:49:37.585610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587403+0000","last_change":"2024-03-28T16:49:04.723184+0000","last_active":"2024-03-28T16:49:37.587403+0000","last_peered":"2024-03-28T16:49:37.587403+0000","last_clean":"2024-03-28T16:49:37.587403+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:49:37.587403+0000","last_undegraded":"2024-03-28T16:49:37.587403+0000","last_fullsized":"2024-03-28T16:49:37.587403+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:32.776436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877381+0000","last_change":"2024-03-28T16:48:56.683578+0000","last_active":"2024-03-28T16:49:39.877381+0000","last_peered":"2024-03-28T16:49:39.877381+0000","last_clean":"2024-03-28T16:49:39.877381+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:49:39.877381+0000","last_undegraded":"2024-03-28T16:49:39.877381+0000","last_fullsized":"2024-03-28T16:49:39.877381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:25.885438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588449+0000","last_change":"2024-03-28T16:48:54.664924+0000","last_active":"2024-03-28T16:49:37.588449+0000","last_peered":"2024-03-28T16:49:37.588449+0000","last_clean":"2024-03-28T16:49:37.588449+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:49:37.588449+0000","last_undegraded":"2024-03-28T16:49:37.588449+0000","last_fullsized":"2024-03-28T16:49:37.588449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:27:34.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595270+0000","last_change":"2024-03-28T16:48:58.692108+0000","last_active":"2024-03-28T16:49:37.595270+0000","last_peered":"2024-03-28T16:49:37.595270+0000","last_clean":"2024-03-28T16:49:37.595270+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:49:37.595270+0000","last_undegraded":"2024-03-28T16:49:37.595270+0000","last_fullsized":"2024-03-28T16:49:37.595270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:05.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'30","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.134000+0000","last_change":"2024-03-28T16:49:00.721579+0000","last_active":"2024-03-28T16:50:23.134000+0000","last_peered":"2024-03-28T16:50:23.134000+0000","last_clean":"2024-03-28T16:50:23.134000+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:23.134000+0000","last_undegraded":"2024-03-28T16:50:23.134000+0000","last_fullsized":"2024-03-28T16:50:23.134000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:13.394379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587827+0000","last_change":"2024-03-28T16:49:02.739236+0000","last_active":"2024-03-28T16:49:37.587827+0000","last_peered":"2024-03-28T16:49:37.587827+0000","last_clean":"2024-03-28T16:49:37.587827+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:49:37.587827+0000","last_undegraded":"2024-03-28T16:49:37.587827+0000","last_fullsized":"2024-03-28T16:49:37.587827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:34:20.723680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580108+0000","last_change":"2024-03-28T16:49:04.748645+0000","last_active":"2024-03-28T16:49:37.580108+0000","last_peered":"2024-03-28T16:49:37.580108+0000","last_clean":"2024-03-28T16:49:37.580108+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:49:37.580108+0000","last_undegraded":"2024-03-28T16:49:37.580108+0000","last_fullsized":"2024-03-28T16:49:37.580108+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:06:34.291471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593462+0000","last_change":"2024-03-28T16:48:56.695870+0000","last_active":"2024-03-28T16:49:37.593462+0000","last_peered":"2024-03-28T16:49:37.593462+0000","last_clean":"2024-03-28T16:49:37.593462+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:49:37.593462+0000","last_undegraded":"2024-03-28T16:49:37.593462+0000","last_fullsized":"2024-03-28T16:49:37.593462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:02.971815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589696+0000","last_change":"2024-03-28T16:48:54.673299+0000","last_active":"2024-03-28T16:49:37.589696+0000","last_peered":"2024-03-28T16:49:37.589696+0000","last_clean":"2024-03-28T16:49:37.589696+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:49:37.589696+0000","last_undegraded":"2024-03-28T16:49:37.589696+0000","last_fullsized":"2024-03-28T16:49:37.589696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:37.511744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588920+0000","last_change":"2024-03-28T16:48:58.683855+0000","last_active":"2024-03-28T16:49:37.588920+0000","last_peered":"2024-03-28T16:49:37.588920+0000","last_clean":"2024-03-28T16:49:37.588920+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:49:37.588920+0000","last_undegraded":"2024-03-28T16:49:37.588920+0000","last_fullsized":"2024-03-28T16:49:37.588920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:01.947384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'35","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.825556+0000","last_change":"2024-03-28T16:49:00.697730+0000","last_active":"2024-03-28T16:50:28.825556+0000","last_peered":"2024-03-28T16:50:28.825556+0000","last_clean":"2024-03-28T16:50:28.825556+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:50:28.825556+0000","last_undegraded":"2024-03-28T16:50:28.825556+0000","last_fullsized":"2024-03-28T16:50:28.825556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:27.038167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'246","reported_seq":322,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854703+0000","last_change":"2024-03-28T16:49:02.726890+0000","last_active":"2024-03-28T16:50:33.854703+0000","last_peered":"2024-03-28T16:50:33.854703+0000","last_clean":"2024-03-28T16:50:33.854703+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:33.854703+0000","last_undegraded":"2024-03-28T16:50:33.854703+0000","last_fullsized":"2024-03-28T16:50:33.854703+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:42.002449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'13","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.217530+0000","last_change":"2024-03-28T16:49:04.743257+0000","last_active":"2024-03-28T16:50:23.217530+0000","last_peered":"2024-03-28T16:50:23.217530+0000","last_clean":"2024-03-28T16:50:23.217530+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:50:23.217530+0000","last_undegraded":"2024-03-28T16:50:23.217530+0000","last_fullsized":"2024-03-28T16:50:23.217530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839667+0000","last_change":"2024-03-28T16:48:56.690053+0000","last_active":"2024-03-28T16:50:28.839667+0000","last_peered":"2024-03-28T16:50:28.839667+0000","last_clean":"2024-03-28T16:50:28.839667+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:50:28.839667+0000","last_undegraded":"2024-03-28T16:50:28.839667+0000","last_fullsized":"2024-03-28T16:50:28.839667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581989+0000","last_change":"2024-03-28T16:48:54.664571+0000","last_active":"2024-03-28T16:49:37.581989+0000","last_peered":"2024-03-28T16:49:37.581989+0000","last_clean":"2024-03-28T16:49:37.581989+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:49:37.581989+0000","last_undegraded":"2024-03-28T16:49:37.581989+0000","last_fullsized":"2024-03-28T16:49:37.581989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:56:22.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'3","reported_seq":378,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.377726+0000","last_change":"2024-03-28T16:49:37.622013+0000","last_active":"2024-03-28T16:50:30.377726+0000","last_peered":"2024-03-28T16:50:30.377726+0000","last_clean":"2024-03-28T16:50:30.377726+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:50:30.377726+0000","last_undegraded":"2024-03-28T16:50:30.377726+0000","last_fullsized":"2024-03-28T16:50:30.377726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:46.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":336,"num_read_kb":336,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587255+0000","last_change":"2024-03-28T16:49:00.725908+0000","last_active":"2024-03-28T16:49:37.587255+0000","last_peered":"2024-03-28T16:49:37.587255+0000","last_clean":"2024-03-28T16:49:37.587255+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:49:37.587255+0000","last_undegraded":"2024-03-28T16:49:37.587255+0000","last_fullsized":"2024-03-28T16:49:37.587255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:40.250784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'246","reported_seq":345,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854050+0000","last_change":"2024-03-28T16:49:02.715172+0000","last_active":"2024-03-28T16:50:33.854050+0000","last_peered":"2024-03-28T16:50:33.854050+0000","last_clean":"2024-03-28T16:50:33.854050+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:33.854050+0000","last_undegraded":"2024-03-28T16:50:33.854050+0000","last_fullsized":"2024-03-28T16:50:33.854050+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589680+0000","last_change":"2024-03-28T16:49:04.746465+0000","last_active":"2024-03-28T16:49:37.589680+0000","last_peered":"2024-03-28T16:49:37.589680+0000","last_clean":"2024-03-28T16:49:37.589680+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:49:37.589680+0000","last_undegraded":"2024-03-28T16:49:37.589680+0000","last_fullsized":"2024-03-28T16:49:37.589680+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:41.396912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189625+0000","last_change":"2024-03-28T16:48:56.681161+0000","last_active":"2024-03-28T16:50:23.189625+0000","last_peered":"2024-03-28T16:50:23.189625+0000","last_clean":"2024-03-28T16:50:23.189625+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:50:23.189625+0000","last_undegraded":"2024-03-28T16:50:23.189625+0000","last_fullsized":"2024-03-28T16:50:23.189625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:23.209038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580546+0000","last_change":"2024-03-28T16:48:54.676487+0000","last_active":"2024-03-28T16:49:37.580546+0000","last_peered":"2024-03-28T16:49:37.580546+0000","last_clean":"2024-03-28T16:49:37.580546+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:49:37.580546+0000","last_undegraded":"2024-03-28T16:49:37.580546+0000","last_fullsized":"2024-03-28T16:49:37.580546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:15.844441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588709+0000","last_change":"2024-03-28T16:48:58.703765+0000","last_active":"2024-03-28T16:49:37.588709+0000","last_peered":"2024-03-28T16:49:37.588709+0000","last_clean":"2024-03-28T16:49:37.588709+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:49:37.588709+0000","last_undegraded":"2024-03-28T16:49:37.588709+0000","last_fullsized":"2024-03-28T16:49:37.588709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:05:16.548217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'36","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.151478+0000","last_change":"2024-03-28T16:49:00.711767+0000","last_active":"2024-03-28T16:50:23.151478+0000","last_peered":"2024-03-28T16:50:23.151478+0000","last_clean":"2024-03-28T16:50:23.151478+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:23.151478+0000","last_undegraded":"2024-03-28T16:50:23.151478+0000","last_fullsized":"2024-03-28T16:50:23.151478+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:07.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590333+0000","last_change":"2024-03-28T16:49:02.720427+0000","last_active":"2024-03-28T16:49:37.590333+0000","last_peered":"2024-03-28T16:49:37.590333+0000","last_clean":"2024-03-28T16:49:37.590333+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:49:37.590333+0000","last_undegraded":"2024-03-28T16:49:37.590333+0000","last_fullsized":"2024-03-28T16:49:37.590333+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:02.281547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596396+0000","last_change":"2024-03-28T16:49:04.730860+0000","last_active":"2024-03-28T16:49:37.596396+0000","last_peered":"2024-03-28T16:49:37.596396+0000","last_clean":"2024-03-28T16:49:37.596396+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:49:37.596396+0000","last_undegraded":"2024-03-28T16:49:37.596396+0000","last_fullsized":"2024-03-28T16:49:37.596396+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189979+0000","last_change":"2024-03-28T16:48:56.693994+0000","last_active":"2024-03-28T16:50:23.189979+0000","last_peered":"2024-03-28T16:50:23.189979+0000","last_clean":"2024-03-28T16:50:23.189979+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:50:23.189979+0000","last_undegraded":"2024-03-28T16:50:23.189979+0000","last_fullsized":"2024-03-28T16:50:23.189979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:09.568996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":198,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583310+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:49:37.583310+0000","last_peered":"2024-03-28T16:49:37.583310+0000","last_clean":"2024-03-28T16:49:37.583310+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:49:37.583310+0000","last_undegraded":"2024-03-28T16:49:37.583310+0000","last_fullsized":"2024-03-28T16:49:37.583310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:35.744620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593784+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:49:37.593784+0000","last_peered":"2024-03-28T16:49:37.593784+0000","last_clean":"2024-03-28T16:49:37.593784+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:49:37.593784+0000","last_undegraded":"2024-03-28T16:49:37.593784+0000","last_fullsized":"2024-03-28T16:49:37.593784+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.194802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581636+0000","last_change":"2024-03-28T16:48:54.666099+0000","last_active":"2024-03-28T16:49:37.581636+0000","last_peered":"2024-03-28T16:49:37.581636+0000","last_clean":"2024-03-28T16:49:37.581636+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:49:37.581636+0000","last_undegraded":"2024-03-28T16:49:37.581636+0000","last_fullsized":"2024-03-28T16:49:37.581636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:03.153093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585495+0000","last_change":"2024-03-28T16:48:58.683594+0000","last_active":"2024-03-28T16:49:37.585495+0000","last_peered":"2024-03-28T16:49:37.585495+0000","last_clean":"2024-03-28T16:49:37.585495+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:49:37.585495+0000","last_undegraded":"2024-03-28T16:49:37.585495+0000","last_fullsized":"2024-03-28T16:49:37.585495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:14.457437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'70","reported_seq":138,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.807275+0000","last_change":"2024-03-28T16:49:00.715887+0000","last_active":"2024-03-28T16:50:28.807275+0000","last_peered":"2024-03-28T16:50:28.807275+0000","last_clean":"2024-03-28T16:50:28.807275+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:28.807275+0000","last_undegraded":"2024-03-28T16:50:28.807275+0000","last_fullsized":"2024-03-28T16:50:28.807275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:37:58.662402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591287+0000","last_change":"2024-03-28T16:49:02.731593+0000","last_active":"2024-03-28T16:49:37.591287+0000","last_peered":"2024-03-28T16:49:37.591287+0000","last_clean":"2024-03-28T16:49:37.591287+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:49:37.591287+0000","last_undegraded":"2024-03-28T16:49:37.591287+0000","last_fullsized":"2024-03-28T16:49:37.591287+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:36.899516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'52","reported_seq":114,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.870448+0000","last_change":"2024-03-28T16:49:04.752277+0000","last_active":"2024-03-28T16:50:28.870448+0000","last_peered":"2024-03-28T16:50:28.870448+0000","last_clean":"2024-03-28T16:50:28.870448+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:50:28.870448+0000","last_undegraded":"2024-03-28T16:50:28.870448+0000","last_fullsized":"2024-03-28T16:50:28.870448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:10.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'11","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.202723+0000","last_change":"2024-03-28T16:49:37.614693+0000","last_active":"2024-03-28T16:50:23.202723+0000","last_peered":"2024-03-28T16:50:23.202723+0000","last_clean":"2024-03-28T16:50:23.202723+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:50:23.202723+0000","last_undegraded":"2024-03-28T16:50:23.202723+0000","last_fullsized":"2024-03-28T16:50:23.202723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:53.196986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580404+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:49:37.580404+0000","last_peered":"2024-03-28T16:49:37.580404+0000","last_clean":"2024-03-28T16:49:37.580404+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:49:37.580404+0000","last_undegraded":"2024-03-28T16:49:37.580404+0000","last_fullsized":"2024-03-28T16:49:37.580404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:11.830948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585095+0000","last_change":"2024-03-28T16:48:54.662603+0000","last_active":"2024-03-28T16:49:37.585095+0000","last_peered":"2024-03-28T16:49:37.585095+0000","last_clean":"2024-03-28T16:49:37.585095+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:49:37.585095+0000","last_undegraded":"2024-03-28T16:49:37.585095+0000","last_fullsized":"2024-03-28T16:49:37.585095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594800+0000","last_change":"2024-03-28T16:48:58.706305+0000","last_active":"2024-03-28T16:49:37.594800+0000","last_peered":"2024-03-28T16:49:37.594800+0000","last_clean":"2024-03-28T16:49:37.594800+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:49:37.594800+0000","last_undegraded":"2024-03-28T16:49:37.594800+0000","last_fullsized":"2024-03-28T16:49:37.594800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:32.869035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'243","reported_seq":415,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854148+0000","last_change":"2024-03-28T16:49:00.718247+0000","last_active":"2024-03-28T16:50:33.854148+0000","last_peered":"2024-03-28T16:50:33.854148+0000","last_clean":"2024-03-28T16:50:33.854148+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:33.854148+0000","last_undegraded":"2024-03-28T16:50:33.854148+0000","last_fullsized":"2024-03-28T16:50:33.854148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:43.848104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":321,"num_read_kb":306,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582450+0000","last_change":"2024-03-28T16:49:02.740444+0000","last_active":"2024-03-28T16:49:37.582450+0000","last_peered":"2024-03-28T16:49:37.582450+0000","last_clean":"2024-03-28T16:49:37.582450+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:49:37.582450+0000","last_undegraded":"2024-03-28T16:49:37.582450+0000","last_fullsized":"2024-03-28T16:49:37.582450+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:41.589208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'4","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.938249+0000","last_change":"2024-03-28T16:49:04.744802+0000","last_active":"2024-03-28T16:50:22.938249+0000","last_peered":"2024-03-28T16:50:22.938249+0000","last_clean":"2024-03-28T16:50:22.938249+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:50:22.938249+0000","last_undegraded":"2024-03-28T16:50:22.938249+0000","last_fullsized":"2024-03-28T16:50:22.938249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.556077+0000","last_change":"2024-03-28T16:48:56.689836+0000","last_active":"2024-03-28T16:49:40.556077+0000","last_peered":"2024-03-28T16:49:40.556077+0000","last_clean":"2024-03-28T16:49:40.556077+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:49:40.556077+0000","last_undegraded":"2024-03-28T16:49:40.556077+0000","last_fullsized":"2024-03-28T16:49:40.556077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589208+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:49:37.589208+0000","last_peered":"2024-03-28T16:49:37.589208+0000","last_clean":"2024-03-28T16:49:37.589208+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:49:37.589208+0000","last_undegraded":"2024-03-28T16:49:37.589208+0000","last_fullsized":"2024-03-28T16:49:37.589208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584655+0000","last_change":"2024-03-28T16:48:54.663058+0000","last_active":"2024-03-28T16:49:37.584655+0000","last_peered":"2024-03-28T16:49:37.584655+0000","last_clean":"2024-03-28T16:49:37.584655+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:49:37.584655+0000","last_undegraded":"2024-03-28T16:49:37.584655+0000","last_fullsized":"2024-03-28T16:49:37.584655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"22'1","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.599332+0000","last_change":"2024-03-28T16:48:58.687188+0000","last_active":"2024-03-28T16:49:37.599332+0000","last_peered":"2024-03-28T16:49:37.599332+0000","last_clean":"2024-03-28T16:49:37.599332+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:49:37.599332+0000","last_undegraded":"2024-03-28T16:49:37.599332+0000","last_fullsized":"2024-03-28T16:49:37.599332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:27.900434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'16","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854777+0000","last_change":"2024-03-28T16:49:00.713728+0000","last_active":"2024-03-28T16:50:33.854777+0000","last_peered":"2024-03-28T16:50:33.854777+0000","last_clean":"2024-03-28T16:50:33.854777+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:33.854777+0000","last_undegraded":"2024-03-28T16:50:33.854777+0000","last_fullsized":"2024-03-28T16:50:33.854777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:03.960810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586565+0000","last_change":"2024-03-28T16:49:02.732230+0000","last_active":"2024-03-28T16:49:37.586565+0000","last_peered":"2024-03-28T16:49:37.586565+0000","last_clean":"2024-03-28T16:49:37.586565+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:49:37.586565+0000","last_undegraded":"2024-03-28T16:49:37.586565+0000","last_fullsized":"2024-03-28T16:49:37.586565+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:20.091845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581243+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:49:37.581243+0000","last_peered":"2024-03-28T16:49:37.581243+0000","last_clean":"2024-03-28T16:49:37.581243+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:49:37.581243+0000","last_undegraded":"2024-03-28T16:49:37.581243+0000","last_fullsized":"2024-03-28T16:49:37.581243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:51.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587517+0000","last_change":"2024-03-28T16:48:56.683769+0000","last_active":"2024-03-28T16:49:37.587517+0000","last_peered":"2024-03-28T16:49:37.587517+0000","last_clean":"2024-03-28T16:49:37.587517+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:49:37.587517+0000","last_undegraded":"2024-03-28T16:49:37.587517+0000","last_fullsized":"2024-03-28T16:49:37.587517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581400+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:49:37.581400+0000","last_peered":"2024-03-28T16:49:37.581400+0000","last_clean":"2024-03-28T16:49:37.581400+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:49:37.581400+0000","last_undegraded":"2024-03-28T16:49:37.581400+0000","last_fullsized":"2024-03-28T16:49:37.581400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:55:50.278254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591305+0000","last_change":"2024-03-28T16:48:54.666472+0000","last_active":"2024-03-28T16:49:37.591305+0000","last_peered":"2024-03-28T16:49:37.591305+0000","last_clean":"2024-03-28T16:49:37.591305+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:49:37.591305+0000","last_undegraded":"2024-03-28T16:49:37.591305+0000","last_fullsized":"2024-03-28T16:49:37.591305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587375+0000","last_change":"2024-03-28T16:48:58.688005+0000","last_active":"2024-03-28T16:49:37.587375+0000","last_peered":"2024-03-28T16:49:37.587375+0000","last_clean":"2024-03-28T16:49:37.587375+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:49:37.587375+0000","last_undegraded":"2024-03-28T16:49:37.587375+0000","last_fullsized":"2024-03-28T16:49:37.587375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:06.936540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'13","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595894+0000","last_change":"2024-03-28T16:49:00.713939+0000","last_active":"2024-03-28T16:49:37.595894+0000","last_peered":"2024-03-28T16:49:37.595894+0000","last_clean":"2024-03-28T16:49:37.595894+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:49:37.595894+0000","last_undegraded":"2024-03-28T16:49:37.595894+0000","last_fullsized":"2024-03-28T16:49:37.595894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:00:16.840810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588068+0000","last_change":"2024-03-28T16:49:02.741194+0000","last_active":"2024-03-28T16:49:37.588068+0000","last_peered":"2024-03-28T16:49:37.588068+0000","last_clean":"2024-03-28T16:49:37.588068+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:49:37.588068+0000","last_undegraded":"2024-03-28T16:49:37.588068+0000","last_fullsized":"2024-03-28T16:49:37.588068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:56.754215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585813+0000","last_change":"2024-03-28T16:49:04.741719+0000","last_active":"2024-03-28T16:49:37.585813+0000","last_peered":"2024-03-28T16:49:37.585813+0000","last_clean":"2024-03-28T16:49:37.585813+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:49:37.585813+0000","last_undegraded":"2024-03-28T16:49:37.585813+0000","last_fullsized":"2024-03-28T16:49:37.585813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:44.134615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555275+0000","last_change":"2024-03-28T16:48:56.687243+0000","last_active":"2024-03-28T16:49:40.555275+0000","last_peered":"2024-03-28T16:49:40.555275+0000","last_clean":"2024-03-28T16:49:40.555275+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:49:40.555275+0000","last_undegraded":"2024-03-28T16:49:40.555275+0000","last_fullsized":"2024-03-28T16:49:40.555275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596299+0000","last_change":"2024-03-28T16:48:54.657105+0000","last_active":"2024-03-28T16:49:37.596299+0000","last_peered":"2024-03-28T16:49:37.596299+0000","last_clean":"2024-03-28T16:49:37.596299+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:49:37.596299+0000","last_undegraded":"2024-03-28T16:49:37.596299+0000","last_fullsized":"2024-03-28T16:49:37.596299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586158+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:49:37.586158+0000","last_peered":"2024-03-28T16:49:37.586158+0000","last_clean":"2024-03-28T16:49:37.586158+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:49:37.586158+0000","last_undegraded":"2024-03-28T16:49:37.586158+0000","last_fullsized":"2024-03-28T16:49:37.586158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588917+0000","last_change":"2024-03-28T16:48:58.690031+0000","last_active":"2024-03-28T16:49:37.588917+0000","last_peered":"2024-03-28T16:49:37.588917+0000","last_clean":"2024-03-28T16:49:37.588917+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:49:37.588917+0000","last_undegraded":"2024-03-28T16:49:37.588917+0000","last_fullsized":"2024-03-28T16:49:37.588917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:29.762391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'57","reported_seq":115,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.796013+0000","last_change":"2024-03-28T16:49:00.706797+0000","last_active":"2024-03-28T16:50:28.796013+0000","last_peered":"2024-03-28T16:50:28.796013+0000","last_clean":"2024-03-28T16:50:28.796013+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:28.796013+0000","last_undegraded":"2024-03-28T16:50:28.796013+0000","last_fullsized":"2024-03-28T16:50:28.796013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:30.719266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590150+0000","last_change":"2024-03-28T16:49:02.720800+0000","last_active":"2024-03-28T16:49:37.590150+0000","last_peered":"2024-03-28T16:49:37.590150+0000","last_clean":"2024-03-28T16:49:37.590150+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:49:37.590150+0000","last_undegraded":"2024-03-28T16:49:37.590150+0000","last_fullsized":"2024-03-28T16:49:37.590150+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:07.770684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'9","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.211945+0000","last_change":"2024-03-28T16:49:04.741551+0000","last_active":"2024-03-28T16:50:30.211945+0000","last_peered":"2024-03-28T16:50:30.211945+0000","last_clean":"2024-03-28T16:50:30.211945+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:50:30.211945+0000","last_undegraded":"2024-03-28T16:50:30.211945+0000","last_fullsized":"2024-03-28T16:50:30.211945+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:55.072698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190216+0000","last_change":"2024-03-28T16:48:56.683849+0000","last_active":"2024-03-28T16:50:23.190216+0000","last_peered":"2024-03-28T16:50:23.190216+0000","last_clean":"2024-03-28T16:50:23.190216+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:50:23.190216+0000","last_undegraded":"2024-03-28T16:50:23.190216+0000","last_fullsized":"2024-03-28T16:50:23.190216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:57.646185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586877+0000","last_change":"2024-03-28T16:48:54.663076+0000","last_active":"2024-03-28T16:49:37.586877+0000","last_peered":"2024-03-28T16:49:37.586877+0000","last_clean":"2024-03-28T16:49:37.586877+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:49:37.586877+0000","last_undegraded":"2024-03-28T16:49:37.586877+0000","last_fullsized":"2024-03-28T16:49:37.586877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592775+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:49:37.592775+0000","last_peered":"2024-03-28T16:49:37.592775+0000","last_clean":"2024-03-28T16:49:37.592775+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:49:37.592775+0000","last_undegraded":"2024-03-28T16:49:37.592775+0000","last_fullsized":"2024-03-28T16:49:37.592775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:37.509224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582103+0000","last_change":"2024-03-28T16:48:58.683037+0000","last_active":"2024-03-28T16:49:37.582103+0000","last_peered":"2024-03-28T16:49:37.582103+0000","last_clean":"2024-03-28T16:49:37.582103+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:49:37.582103+0000","last_undegraded":"2024-03-28T16:49:37.582103+0000","last_fullsized":"2024-03-28T16:49:37.582103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:16.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'14","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595604+0000","last_change":"2024-03-28T16:49:00.720928+0000","last_active":"2024-03-28T16:49:37.595604+0000","last_peered":"2024-03-28T16:49:37.595604+0000","last_clean":"2024-03-28T16:49:37.595604+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:49:37.595604+0000","last_undegraded":"2024-03-28T16:49:37.595604+0000","last_fullsized":"2024-03-28T16:49:37.595604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:03.955318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'246","reported_seq":291,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.853994+0000","last_change":"2024-03-28T16:49:02.715814+0000","last_active":"2024-03-28T16:50:33.853994+0000","last_peered":"2024-03-28T16:50:33.853994+0000","last_clean":"2024-03-28T16:50:33.853994+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:33.853994+0000","last_undegraded":"2024-03-28T16:50:33.853994+0000","last_fullsized":"2024-03-28T16:50:33.853994+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.177166+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:23.177166+0000","last_peered":"2024-03-28T16:50:23.177166+0000","last_clean":"2024-03-28T16:50:23.177166+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:23.177166+0000","last_undegraded":"2024-03-28T16:50:23.177166+0000","last_fullsized":"2024-03-28T16:50:23.177166+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:14:04.371488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189696+0000","last_change":"2024-03-28T16:48:56.693531+0000","last_active":"2024-03-28T16:50:23.189696+0000","last_peered":"2024-03-28T16:50:23.189696+0000","last_clean":"2024-03-28T16:50:23.189696+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:50:23.189696+0000","last_undegraded":"2024-03-28T16:50:23.189696+0000","last_fullsized":"2024-03-28T16:50:23.189696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:22.327091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.850113+0000","last_change":"2024-03-28T16:48:54.662217+0000","last_active":"2024-03-28T16:49:39.850113+0000","last_peered":"2024-03-28T16:49:39.850113+0000","last_clean":"2024-03-28T16:49:39.850113+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:49:39.850113+0000","last_undegraded":"2024-03-28T16:49:39.850113+0000","last_fullsized":"2024-03-28T16:49:39.850113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:59.868215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591024+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:49:37.591024+0000","last_peered":"2024-03-28T16:49:37.591024+0000","last_clean":"2024-03-28T16:49:37.591024+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:49:37.591024+0000","last_undegraded":"2024-03-28T16:49:37.591024+0000","last_fullsized":"2024-03-28T16:49:37.591024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593505+0000","last_change":"2024-03-28T16:48:58.684369+0000","last_active":"2024-03-28T16:49:37.593505+0000","last_peered":"2024-03-28T16:49:37.593505+0000","last_clean":"2024-03-28T16:49:37.593505+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:49:37.593505+0000","last_undegraded":"2024-03-28T16:49:37.593505+0000","last_fullsized":"2024-03-28T16:49:37.593505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:58.333522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589611+0000","last_change":"2024-03-28T16:49:00.721989+0000","last_active":"2024-03-28T16:49:37.589611+0000","last_peered":"2024-03-28T16:49:37.589611+0000","last_clean":"2024-03-28T16:49:37.589611+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:49:37.589611+0000","last_undegraded":"2024-03-28T16:49:37.589611+0000","last_fullsized":"2024-03-28T16:49:37.589611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:27.568851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'246","reported_seq":324,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854449+0000","last_change":"2024-03-28T16:49:02.730370+0000","last_active":"2024-03-28T16:50:33.854449+0000","last_peered":"2024-03-28T16:50:33.854449+0000","last_clean":"2024-03-28T16:50:33.854449+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:33.854449+0000","last_undegraded":"2024-03-28T16:50:33.854449+0000","last_fullsized":"2024-03-28T16:50:33.854449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585106+0000","last_change":"2024-03-28T16:49:04.739259+0000","last_active":"2024-03-28T16:49:37.585106+0000","last_peered":"2024-03-28T16:49:37.585106+0000","last_clean":"2024-03-28T16:49:37.585106+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:49:37.585106+0000","last_undegraded":"2024-03-28T16:49:37.585106+0000","last_fullsized":"2024-03-28T16:49:37.585106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:50:01.255390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555184+0000","last_change":"2024-03-28T16:48:56.682608+0000","last_active":"2024-03-28T16:49:40.555184+0000","last_peered":"2024-03-28T16:49:40.555184+0000","last_clean":"2024-03-28T16:49:40.555184+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:49:40.555184+0000","last_undegraded":"2024-03-28T16:49:40.555184+0000","last_fullsized":"2024-03-28T16:49:40.555184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.855204+0000","last_change":"2024-03-28T16:48:54.675316+0000","last_active":"2024-03-28T16:49:39.855204+0000","last_peered":"2024-03-28T16:49:39.855204+0000","last_clean":"2024-03-28T16:49:39.855204+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:49:39.855204+0000","last_undegraded":"2024-03-28T16:49:39.855204+0000","last_fullsized":"2024-03-28T16:49:39.855204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588286+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:49:37.588286+0000","last_peered":"2024-03-28T16:49:37.588286+0000","last_clean":"2024-03-28T16:49:37.588286+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:49:37.588286+0000","last_undegraded":"2024-03-28T16:49:37.588286+0000","last_fullsized":"2024-03-28T16:49:37.588286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:45:04.316499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601449+0000","last_change":"2024-03-28T16:48:58.681176+0000","last_active":"2024-03-28T16:49:37.601449+0000","last_peered":"2024-03-28T16:49:37.601449+0000","last_clean":"2024-03-28T16:49:37.601449+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:49:37.601449+0000","last_undegraded":"2024-03-28T16:49:37.601449+0000","last_fullsized":"2024-03-28T16:49:37.601449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:49.369023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'76","reported_seq":147,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.790186+0000","last_change":"2024-03-28T16:49:00.709967+0000","last_active":"2024-03-28T16:50:28.790186+0000","last_peered":"2024-03-28T16:50:28.790186+0000","last_clean":"2024-03-28T16:50:28.790186+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:28.790186+0000","last_undegraded":"2024-03-28T16:50:28.790186+0000","last_fullsized":"2024-03-28T16:50:28.790186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:08.949965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585774+0000","last_change":"2024-03-28T16:49:02.715279+0000","last_active":"2024-03-28T16:49:37.585774+0000","last_peered":"2024-03-28T16:49:37.585774+0000","last_clean":"2024-03-28T16:49:37.585774+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:49:37.585774+0000","last_undegraded":"2024-03-28T16:49:37.585774+0000","last_fullsized":"2024-03-28T16:49:37.585774+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:02.364003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594229+0000","last_change":"2024-03-28T16:49:04.754190+0000","last_active":"2024-03-28T16:49:37.594229+0000","last_peered":"2024-03-28T16:49:37.594229+0000","last_clean":"2024-03-28T16:49:37.594229+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:49:37.594229+0000","last_undegraded":"2024-03-28T16:49:37.594229+0000","last_fullsized":"2024-03-28T16:49:37.594229+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:26.719819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594252+0000","last_change":"2024-03-28T16:49:04.740956+0000","last_active":"2024-03-28T16:49:37.594252+0000","last_peered":"2024-03-28T16:49:37.594252+0000","last_clean":"2024-03-28T16:49:37.594252+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:49:37.594252+0000","last_undegraded":"2024-03-28T16:49:37.594252+0000","last_fullsized":"2024-03-28T16:49:37.594252+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:55.907917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190179+0000","last_change":"2024-03-28T16:48:56.680906+0000","last_active":"2024-03-28T16:50:23.190179+0000","last_peered":"2024-03-28T16:50:23.190179+0000","last_clean":"2024-03-28T16:50:23.190179+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:50:23.190179+0000","last_undegraded":"2024-03-28T16:50:23.190179+0000","last_fullsized":"2024-03-28T16:50:23.190179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:05.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587082+0000","last_change":"2024-03-28T16:48:54.663088+0000","last_active":"2024-03-28T16:49:37.587082+0000","last_peered":"2024-03-28T16:49:37.587082+0000","last_clean":"2024-03-28T16:49:37.587082+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:49:37.587082+0000","last_undegraded":"2024-03-28T16:49:37.587082+0000","last_fullsized":"2024-03-28T16:49:37.587082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.659538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.606876+0000","last_change":"2024-03-28T16:49:37.606876+0000","last_active":"2024-03-28T16:49:37.606876+0000","last_peered":"2024-03-28T16:49:37.606876+0000","last_clean":"2024-03-28T16:49:37.606876+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:49:37.606876+0000","last_undegraded":"2024-03-28T16:49:37.606876+0000","last_fullsized":"2024-03-28T16:49:37.606876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:25.555426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'31","reported_seq":73,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.157002+0000","last_change":"2024-03-28T16:49:00.725615+0000","last_active":"2024-03-28T16:50:23.157002+0000","last_peered":"2024-03-28T16:50:23.157002+0000","last_clean":"2024-03-28T16:50:23.157002+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:50:23.157002+0000","last_undegraded":"2024-03-28T16:50:23.157002+0000","last_fullsized":"2024-03-28T16:50:23.157002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:56.763507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582514+0000","last_change":"2024-03-28T16:49:02.739259+0000","last_active":"2024-03-28T16:49:37.582514+0000","last_peered":"2024-03-28T16:49:37.582514+0000","last_clean":"2024-03-28T16:49:37.582514+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:49:37.582514+0000","last_undegraded":"2024-03-28T16:49:37.582514+0000","last_fullsized":"2024-03-28T16:49:37.582514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:33.761245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'1","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586987+0000","last_change":"2024-03-28T16:49:04.723410+0000","last_active":"2024-03-28T16:49:37.586987+0000","last_peered":"2024-03-28T16:49:37.586987+0000","last_clean":"2024-03-28T16:49:37.586987+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:49:37.586987+0000","last_undegraded":"2024-03-28T16:49:37.586987+0000","last_fullsized":"2024-03-28T16:49:37.586987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:56:19.278765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.603811+0000","last_change":"2024-03-28T16:49:37.603811+0000","last_active":"2024-03-28T16:49:37.603811+0000","last_peered":"2024-03-28T16:49:37.603811+0000","last_clean":"2024-03-28T16:49:37.603811+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:49:37.603811+0000","last_undegraded":"2024-03-28T16:49:37.603811+0000","last_fullsized":"2024-03-28T16:49:37.603811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:44.503106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.833936+0000","last_change":"2024-03-28T16:48:54.657586+0000","last_active":"2024-03-28T16:50:28.833936+0000","last_peered":"2024-03-28T16:50:28.833936+0000","last_clean":"2024-03-28T16:50:28.833936+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:50:28.833936+0000","last_undegraded":"2024-03-28T16:50:28.833936+0000","last_fullsized":"2024-03-28T16:50:28.833936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594338+0000","last_change":"2024-03-28T16:48:58.696407+0000","last_active":"2024-03-28T16:49:37.594338+0000","last_peered":"2024-03-28T16:49:37.594338+0000","last_clean":"2024-03-28T16:49:37.594338+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:49:37.594338+0000","last_undegraded":"2024-03-28T16:49:37.594338+0000","last_fullsized":"2024-03-28T16:49:37.594338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:45.492142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:15.707942+0000","last_change":"2024-03-28T16:49:00.701020+0000","last_active":"2024-03-28T16:50:15.707942+0000","last_peered":"2024-03-28T16:50:15.707942+0000","last_clean":"2024-03-28T16:50:15.707942+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:50:15.707942+0000","last_undegraded":"2024-03-28T16:50:15.707942+0000","last_fullsized":"2024-03-28T16:50:15.707942+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:42.652898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589597+0000","last_change":"2024-03-28T16:49:02.726748+0000","last_active":"2024-03-28T16:49:37.589597+0000","last_peered":"2024-03-28T16:49:37.589597+0000","last_clean":"2024-03-28T16:49:37.589597+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:49:37.589597+0000","last_undegraded":"2024-03-28T16:49:37.589597+0000","last_fullsized":"2024-03-28T16:49:37.589597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:39.742691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586140+0000","last_change":"2024-03-28T16:49:04.744040+0000","last_active":"2024-03-28T16:49:37.586140+0000","last_peered":"2024-03-28T16:49:37.586140+0000","last_clean":"2024-03-28T16:49:37.586140+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:49:37.586140+0000","last_undegraded":"2024-03-28T16:49:37.586140+0000","last_fullsized":"2024-03-28T16:49:37.586140+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:51:53.769493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.730425+0000","last_change":"2024-03-28T16:48:56.676841+0000","last_active":"2024-03-28T16:50:22.730425+0000","last_peered":"2024-03-28T16:50:22.730425+0000","last_clean":"2024-03-28T16:50:22.730425+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:50:22.730425+0000","last_undegraded":"2024-03-28T16:50:22.730425+0000","last_fullsized":"2024-03-28T16:50:22.730425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.183347+0000","last_change":"2024-03-28T16:48:54.659093+0000","last_active":"2024-03-28T16:50:23.183347+0000","last_peered":"2024-03-28T16:50:23.183347+0000","last_clean":"2024-03-28T16:50:23.183347+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:50:23.183347+0000","last_undegraded":"2024-03-28T16:50:23.183347+0000","last_fullsized":"2024-03-28T16:50:23.183347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'2","reported_seq":196,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.365615+0000","last_change":"2024-03-28T16:48:58.690200+0000","last_active":"2024-03-28T16:50:30.365615+0000","last_peered":"2024-03-28T16:50:30.365615+0000","last_clean":"2024-03-28T16:50:30.365615+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:50:30.365615+0000","last_undegraded":"2024-03-28T16:50:30.365615+0000","last_fullsized":"2024-03-28T16:50:30.365615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:59.661181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":253,"num_read_kb":252,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'35","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.577197+0000","last_change":"2024-03-28T16:49:00.718425+0000","last_active":"2024-03-28T16:50:28.577197+0000","last_peered":"2024-03-28T16:50:28.577197+0000","last_clean":"2024-03-28T16:50:28.577197+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:28.577197+0000","last_undegraded":"2024-03-28T16:50:28.577197+0000","last_fullsized":"2024-03-28T16:50:28.577197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:37.725635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":50,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585222+0000","last_change":"2024-03-28T16:49:02.715466+0000","last_active":"2024-03-28T16:49:37.585222+0000","last_peered":"2024-03-28T16:49:37.585222+0000","last_clean":"2024-03-28T16:49:37.585222+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:49:37.585222+0000","last_undegraded":"2024-03-28T16:49:37.585222+0000","last_fullsized":"2024-03-28T16:49:37.585222+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581973+0000","last_change":"2024-03-28T16:49:04.752570+0000","last_active":"2024-03-28T16:49:37.581973+0000","last_peered":"2024-03-28T16:49:37.581973+0000","last_clean":"2024-03-28T16:49:37.581973+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:49:37.581973+0000","last_undegraded":"2024-03-28T16:49:37.581973+0000","last_fullsized":"2024-03-28T16:49:37.581973+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:04:11.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189947+0000","last_change":"2024-03-28T16:48:56.683249+0000","last_active":"2024-03-28T16:50:23.189947+0000","last_peered":"2024-03-28T16:50:23.189947+0000","last_clean":"2024-03-28T16:50:23.189947+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:50:23.189947+0000","last_undegraded":"2024-03-28T16:50:23.189947+0000","last_fullsized":"2024-03-28T16:50:23.189947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:23.009747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602071+0000","last_change":"2024-03-28T16:48:54.674928+0000","last_active":"2024-03-28T16:49:37.602071+0000","last_peered":"2024-03-28T16:49:37.602071+0000","last_clean":"2024-03-28T16:49:37.602071+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:49:37.602071+0000","last_undegraded":"2024-03-28T16:49:37.602071+0000","last_fullsized":"2024-03-28T16:49:37.602071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593497+0000","last_change":"2024-03-28T16:48:58.680113+0000","last_active":"2024-03-28T16:49:37.593497+0000","last_peered":"2024-03-28T16:49:37.593497+0000","last_clean":"2024-03-28T16:49:37.593497+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:49:37.593497+0000","last_undegraded":"2024-03-28T16:49:37.593497+0000","last_fullsized":"2024-03-28T16:49:37.593497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:59.079328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.137620+0000","last_change":"2024-03-28T16:49:00.696647+0000","last_active":"2024-03-28T16:50:23.137620+0000","last_peered":"2024-03-28T16:50:23.137620+0000","last_clean":"2024-03-28T16:50:23.137620+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:50:23.137620+0000","last_undegraded":"2024-03-28T16:50:23.137620+0000","last_fullsized":"2024-03-28T16:50:23.137620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:15.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594972+0000","last_change":"2024-03-28T16:49:02.726334+0000","last_active":"2024-03-28T16:49:37.594972+0000","last_peered":"2024-03-28T16:49:37.594972+0000","last_clean":"2024-03-28T16:49:37.594972+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:49:37.594972+0000","last_undegraded":"2024-03-28T16:49:37.594972+0000","last_fullsized":"2024-03-28T16:49:37.594972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:58.829170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600011+0000","last_change":"2024-03-28T16:48:56.675359+0000","last_active":"2024-03-28T16:49:37.600011+0000","last_peered":"2024-03-28T16:49:37.600011+0000","last_clean":"2024-03-28T16:49:37.600011+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:49:37.600011+0000","last_undegraded":"2024-03-28T16:49:37.600011+0000","last_fullsized":"2024-03-28T16:49:37.600011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:35.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601915+0000","last_change":"2024-03-28T16:49:37.600567+0000","last_active":"2024-03-28T16:49:37.601915+0000","last_peered":"2024-03-28T16:49:37.601915+0000","last_clean":"2024-03-28T16:49:37.601915+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:49:37.601915+0000","last_undegraded":"2024-03-28T16:49:37.601915+0000","last_fullsized":"2024-03-28T16:49:37.601915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:40:41.090124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591166+0000","last_change":"2024-03-28T16:48:58.687213+0000","last_active":"2024-03-28T16:49:37.591166+0000","last_peered":"2024-03-28T16:49:37.591166+0000","last_clean":"2024-03-28T16:49:37.591166+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:49:37.591166+0000","last_undegraded":"2024-03-28T16:49:37.591166+0000","last_fullsized":"2024-03-28T16:49:37.591166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:13.652414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'51","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.785801+0000","last_change":"2024-03-28T16:49:00.716007+0000","last_active":"2024-03-28T16:50:28.785801+0000","last_peered":"2024-03-28T16:50:28.785801+0000","last_clean":"2024-03-28T16:50:28.785801+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:28.785801+0000","last_undegraded":"2024-03-28T16:50:28.785801+0000","last_fullsized":"2024-03-28T16:50:28.785801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:56.812114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595955+0000","last_change":"2024-03-28T16:49:02.730156+0000","last_active":"2024-03-28T16:49:37.595955+0000","last_peered":"2024-03-28T16:49:37.595955+0000","last_clean":"2024-03-28T16:49:37.595955+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:49:37.595955+0000","last_undegraded":"2024-03-28T16:49:37.595955+0000","last_fullsized":"2024-03-28T16:49:37.595955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:15.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'11","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.213440+0000","last_change":"2024-03-28T16:49:04.752525+0000","last_active":"2024-03-28T16:50:23.213440+0000","last_peered":"2024-03-28T16:50:23.213440+0000","last_clean":"2024-03-28T16:50:23.213440+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:50:23.213440+0000","last_undegraded":"2024-03-28T16:50:23.213440+0000","last_fullsized":"2024-03-28T16:50:23.213440+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:18.858161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588098+0000","last_change":"2024-03-28T16:48:56.697482+0000","last_active":"2024-03-28T16:49:37.588098+0000","last_peered":"2024-03-28T16:49:37.588098+0000","last_clean":"2024-03-28T16:49:37.588098+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:49:37.588098+0000","last_undegraded":"2024-03-28T16:49:37.588098+0000","last_fullsized":"2024-03-28T16:49:37.588098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:02.641119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584038+0000","last_change":"2024-03-28T16:48:54.668040+0000","last_active":"2024-03-28T16:49:37.584038+0000","last_peered":"2024-03-28T16:49:37.584038+0000","last_clean":"2024-03-28T16:49:37.584038+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:49:37.584038+0000","last_undegraded":"2024-03-28T16:49:37.584038+0000","last_fullsized":"2024-03-28T16:49:37.584038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596417+0000","last_change":"2024-03-28T16:48:58.686377+0000","last_active":"2024-03-28T16:49:37.596417+0000","last_peered":"2024-03-28T16:49:37.596417+0000","last_clean":"2024-03-28T16:49:37.596417+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:49:37.596417+0000","last_undegraded":"2024-03-28T16:49:37.596417+0000","last_fullsized":"2024-03-28T16:49:37.596417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:01.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.138592+0000","last_change":"2024-03-28T16:49:00.716668+0000","last_active":"2024-03-28T16:50:23.138592+0000","last_peered":"2024-03-28T16:50:23.138592+0000","last_clean":"2024-03-28T16:50:23.138592+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:23.138592+0000","last_undegraded":"2024-03-28T16:50:23.138592+0000","last_fullsized":"2024-03-28T16:50:23.138592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:48.604008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587828+0000","last_change":"2024-03-28T16:49:02.731621+0000","last_active":"2024-03-28T16:49:37.587828+0000","last_peered":"2024-03-28T16:49:37.587828+0000","last_clean":"2024-03-28T16:49:37.587828+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:49:37.587828+0000","last_undegraded":"2024-03-28T16:49:37.587828+0000","last_fullsized":"2024-03-28T16:49:37.587828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.557447+0000","last_change":"2024-03-28T16:49:04.754981+0000","last_active":"2024-03-28T16:49:40.557447+0000","last_peered":"2024-03-28T16:49:40.557447+0000","last_clean":"2024-03-28T16:49:40.557447+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:49:40.557447+0000","last_undegraded":"2024-03-28T16:49:40.557447+0000","last_fullsized":"2024-03-28T16:49:40.557447+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:33:25.577689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'3","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189773+0000","last_change":"2024-03-28T16:48:56.694984+0000","last_active":"2024-03-28T16:50:23.189773+0000","last_peered":"2024-03-28T16:50:23.189773+0000","last_clean":"2024-03-28T16:50:23.189773+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:50:23.189773+0000","last_undegraded":"2024-03-28T16:50:23.189773+0000","last_fullsized":"2024-03-28T16:50:23.189773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:09.365959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585942+0000","last_change":"2024-03-28T16:48:54.654527+0000","last_active":"2024-03-28T16:49:37.585942+0000","last_peered":"2024-03-28T16:49:37.585942+0000","last_clean":"2024-03-28T16:49:37.585942+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:49:37.585942+0000","last_undegraded":"2024-03-28T16:49:37.585942+0000","last_fullsized":"2024-03-28T16:49:37.585942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610755+0000","last_change":"2024-03-28T16:49:37.610755+0000","last_active":"2024-03-28T16:49:37.610755+0000","last_peered":"2024-03-28T16:49:37.610755+0000","last_clean":"2024-03-28T16:49:37.610755+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:49:37.610755+0000","last_undegraded":"2024-03-28T16:49:37.610755+0000","last_fullsized":"2024-03-28T16:49:37.610755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:24.156961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'32","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.163251+0000","last_change":"2024-03-28T16:49:00.706941+0000","last_active":"2024-03-28T16:50:23.163251+0000","last_peered":"2024-03-28T16:50:23.163251+0000","last_clean":"2024-03-28T16:50:23.163251+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:23.163251+0000","last_undegraded":"2024-03-28T16:50:23.163251+0000","last_fullsized":"2024-03-28T16:50:23.163251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:05.753628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":46,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587449+0000","last_change":"2024-03-28T16:49:02.734090+0000","last_active":"2024-03-28T16:49:37.587449+0000","last_peered":"2024-03-28T16:49:37.587449+0000","last_clean":"2024-03-28T16:49:37.587449+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:49:37.587449+0000","last_undegraded":"2024-03-28T16:49:37.587449+0000","last_fullsized":"2024-03-28T16:49:37.587449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:39.107257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.835263+0000","last_change":"2024-03-28T16:49:04.748086+0000","last_active":"2024-03-28T16:49:39.835263+0000","last_peered":"2024-03-28T16:49:39.835263+0000","last_clean":"2024-03-28T16:49:39.835263+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:49:39.835263+0000","last_undegraded":"2024-03-28T16:49:39.835263+0000","last_fullsized":"2024-03-28T16:49:39.835263+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:44.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588233+0000","last_change":"2024-03-28T16:48:56.682850+0000","last_active":"2024-03-28T16:49:37.588233+0000","last_peered":"2024-03-28T16:49:37.588233+0000","last_clean":"2024-03-28T16:49:37.588233+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:49:37.588233+0000","last_undegraded":"2024-03-28T16:49:37.588233+0000","last_fullsized":"2024-03-28T16:49:37.588233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592598+0000","last_change":"2024-03-28T16:48:54.674069+0000","last_active":"2024-03-28T16:49:37.592598+0000","last_peered":"2024-03-28T16:49:37.592598+0000","last_clean":"2024-03-28T16:49:37.592598+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:49:37.592598+0000","last_undegraded":"2024-03-28T16:49:37.592598+0000","last_fullsized":"2024-03-28T16:49:37.592598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.406243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587070+0000","last_change":"2024-03-28T16:48:58.681774+0000","last_active":"2024-03-28T16:49:37.587070+0000","last_peered":"2024-03-28T16:49:37.587070+0000","last_clean":"2024-03-28T16:49:37.587070+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:49:37.587070+0000","last_undegraded":"2024-03-28T16:49:37.587070+0000","last_fullsized":"2024-03-28T16:49:37.587070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:24.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589169+0000","last_change":"2024-03-28T16:49:00.720072+0000","last_active":"2024-03-28T16:49:37.589169+0000","last_peered":"2024-03-28T16:49:37.589169+0000","last_clean":"2024-03-28T16:49:37.589169+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:49:37.589169+0000","last_undegraded":"2024-03-28T16:49:37.589169+0000","last_fullsized":"2024-03-28T16:49:37.589169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:22:49.287785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'246","reported_seq":297,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854122+0000","last_change":"2024-03-28T16:49:02.734898+0000","last_active":"2024-03-28T16:50:33.854122+0000","last_peered":"2024-03-28T16:50:33.854122+0000","last_clean":"2024-03-28T16:50:33.854122+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:33.854122+0000","last_undegraded":"2024-03-28T16:50:33.854122+0000","last_fullsized":"2024-03-28T16:50:33.854122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:05.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.579919+0000","last_change":"2024-03-28T16:49:04.748224+0000","last_active":"2024-03-28T16:49:37.579919+0000","last_peered":"2024-03-28T16:49:37.579919+0000","last_clean":"2024-03-28T16:49:37.579919+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:49:37.579919+0000","last_undegraded":"2024-03-28T16:49:37.579919+0000","last_fullsized":"2024-03-28T16:49:37.579919+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:26.578458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'5","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370813+0000","last_change":"2024-03-28T16:48:56.685308+0000","last_active":"2024-03-28T16:50:22.370813+0000","last_peered":"2024-03-28T16:50:22.370813+0000","last_clean":"2024-03-28T16:50:22.370813+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:50:22.370813+0000","last_undegraded":"2024-03-28T16:50:22.370813+0000","last_fullsized":"2024-03-28T16:50:22.370813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:39.524773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586373+0000","last_change":"2024-03-28T16:48:54.667648+0000","last_active":"2024-03-28T16:49:37.586373+0000","last_peered":"2024-03-28T16:49:37.586373+0000","last_clean":"2024-03-28T16:49:37.586373+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:49:37.586373+0000","last_undegraded":"2024-03-28T16:49:37.586373+0000","last_fullsized":"2024-03-28T16:49:37.586373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598430+0000","last_change":"2024-03-28T16:48:58.684892+0000","last_active":"2024-03-28T16:49:37.598430+0000","last_peered":"2024-03-28T16:49:37.598430+0000","last_clean":"2024-03-28T16:49:37.598430+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:49:37.598430+0000","last_undegraded":"2024-03-28T16:49:37.598430+0000","last_fullsized":"2024-03-28T16:49:37.598430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:34.392979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.126243+0000","last_change":"2024-03-28T16:49:00.722124+0000","last_active":"2024-03-28T16:50:23.126243+0000","last_peered":"2024-03-28T16:50:23.126243+0000","last_clean":"2024-03-28T16:50:23.126243+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:23.126243+0000","last_undegraded":"2024-03-28T16:50:23.126243+0000","last_fullsized":"2024-03-28T16:50:23.126243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:39.985393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584276+0000","last_change":"2024-03-28T16:49:02.715093+0000","last_active":"2024-03-28T16:49:37.584276+0000","last_peered":"2024-03-28T16:49:37.584276+0000","last_clean":"2024-03-28T16:49:37.584276+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:49:37.584276+0000","last_undegraded":"2024-03-28T16:49:37.584276+0000","last_fullsized":"2024-03-28T16:49:37.584276+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:26.448039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601171+0000","last_change":"2024-03-28T16:49:04.745332+0000","last_active":"2024-03-28T16:49:37.601171+0000","last_peered":"2024-03-28T16:49:37.601171+0000","last_clean":"2024-03-28T16:49:37.601171+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:49:37.601171+0000","last_undegraded":"2024-03-28T16:49:37.601171+0000","last_fullsized":"2024-03-28T16:49:37.601171+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:11.521117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369950+0000","last_change":"2024-03-28T16:48:56.690217+0000","last_active":"2024-03-28T16:50:22.369950+0000","last_peered":"2024-03-28T16:50:22.369950+0000","last_clean":"2024-03-28T16:50:22.369950+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:50:22.369950+0000","last_undegraded":"2024-03-28T16:50:22.369950+0000","last_fullsized":"2024-03-28T16:50:22.369950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588654+0000","last_change":"2024-03-28T16:48:54.669871+0000","last_active":"2024-03-28T16:49:37.588654+0000","last_peered":"2024-03-28T16:49:37.588654+0000","last_clean":"2024-03-28T16:49:37.588654+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:49:37.588654+0000","last_undegraded":"2024-03-28T16:49:37.588654+0000","last_fullsized":"2024-03-28T16:49:37.588654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:58.495034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594708+0000","last_change":"2024-03-28T16:48:58.692783+0000","last_active":"2024-03-28T16:49:37.594708+0000","last_peered":"2024-03-28T16:49:37.594708+0000","last_clean":"2024-03-28T16:49:37.594708+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:49:37.594708+0000","last_undegraded":"2024-03-28T16:49:37.594708+0000","last_fullsized":"2024-03-28T16:49:37.594708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:20.188622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.128390+0000","last_change":"2024-03-28T16:49:00.706559+0000","last_active":"2024-03-28T16:50:23.128390+0000","last_peered":"2024-03-28T16:50:23.128390+0000","last_clean":"2024-03-28T16:50:23.128390+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:23.128390+0000","last_undegraded":"2024-03-28T16:50:23.128390+0000","last_fullsized":"2024-03-28T16:50:23.128390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:54.800679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581059+0000","last_change":"2024-03-28T16:49:02.739372+0000","last_active":"2024-03-28T16:49:37.581059+0000","last_peered":"2024-03-28T16:49:37.581059+0000","last_clean":"2024-03-28T16:49:37.581059+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:49:37.581059+0000","last_undegraded":"2024-03-28T16:49:37.581059+0000","last_fullsized":"2024-03-28T16:49:37.581059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:22:56.097286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582527+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:49:37.582527+0000","last_peered":"2024-03-28T16:49:37.582527+0000","last_clean":"2024-03-28T16:49:37.582527+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:49:37.582527+0000","last_undegraded":"2024-03-28T16:49:37.582527+0000","last_fullsized":"2024-03-28T16:49:37.582527+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:45:27.545220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593066+0000","last_change":"2024-03-28T16:48:56.666104+0000","last_active":"2024-03-28T16:49:37.593066+0000","last_peered":"2024-03-28T16:49:37.593066+0000","last_clean":"2024-03-28T16:49:37.593066+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:49:37.593066+0000","last_undegraded":"2024-03-28T16:49:37.593066+0000","last_fullsized":"2024-03-28T16:49:37.593066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:35:42.047564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588335+0000","last_change":"2024-03-28T16:48:54.665975+0000","last_active":"2024-03-28T16:49:37.588335+0000","last_peered":"2024-03-28T16:49:37.588335+0000","last_clean":"2024-03-28T16:49:37.588335+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:49:37.588335+0000","last_undegraded":"2024-03-28T16:49:37.588335+0000","last_fullsized":"2024-03-28T16:49:37.588335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585938+0000","last_change":"2024-03-28T16:48:58.688417+0000","last_active":"2024-03-28T16:49:37.585938+0000","last_peered":"2024-03-28T16:49:37.585938+0000","last_clean":"2024-03-28T16:49:37.585938+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:49:37.585938+0000","last_undegraded":"2024-03-28T16:49:37.585938+0000","last_fullsized":"2024-03-28T16:49:37.585938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:57.154239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'24","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.159574+0000","last_change":"2024-03-28T16:49:00.717188+0000","last_active":"2024-03-28T16:50:23.159574+0000","last_peered":"2024-03-28T16:50:23.159574+0000","last_clean":"2024-03-28T16:50:23.159574+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:23.159574+0000","last_undegraded":"2024-03-28T16:50:23.159574+0000","last_fullsized":"2024-03-28T16:50:23.159574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:40:19.790152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581817+0000","last_change":"2024-03-28T16:49:02.739568+0000","last_active":"2024-03-28T16:49:37.581817+0000","last_peered":"2024-03-28T16:49:37.581817+0000","last_clean":"2024-03-28T16:49:37.581817+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:49:37.581817+0000","last_undegraded":"2024-03-28T16:49:37.581817+0000","last_fullsized":"2024-03-28T16:49:37.581817+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:32.279507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587161+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:49:37.587161+0000","last_peered":"2024-03-28T16:49:37.587161+0000","last_clean":"2024-03-28T16:49:37.587161+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:49:37.587161+0000","last_undegraded":"2024-03-28T16:49:37.587161+0000","last_fullsized":"2024-03-28T16:49:37.587161+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:59.156256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'11","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854243+0000","last_change":"2024-03-28T16:48:56.678271+0000","last_active":"2024-03-28T16:50:28.854243+0000","last_peered":"2024-03-28T16:50:28.854243+0000","last_clean":"2024-03-28T16:50:28.854243+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:50:28.854243+0000","last_undegraded":"2024-03-28T16:50:28.854243+0000","last_fullsized":"2024-03-28T16:50:28.854243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:31.748236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602333+0000","last_change":"2024-03-28T16:48:54.655009+0000","last_active":"2024-03-28T16:49:37.602333+0000","last_peered":"2024-03-28T16:49:37.602333+0000","last_clean":"2024-03-28T16:49:37.602333+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:49:37.602333+0000","last_undegraded":"2024-03-28T16:49:37.602333+0000","last_fullsized":"2024-03-28T16:49:37.602333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'2","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595281+0000","last_change":"2024-03-28T16:48:58.697773+0000","last_active":"2024-03-28T16:49:37.595281+0000","last_peered":"2024-03-28T16:49:37.595281+0000","last_clean":"2024-03-28T16:49:37.595281+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:49:37.595281+0000","last_undegraded":"2024-03-28T16:49:37.595281+0000","last_fullsized":"2024-03-28T16:49:37.595281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:25.024742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'29","reported_seq":70,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.816906+0000","last_change":"2024-03-28T16:49:00.720335+0000","last_active":"2024-03-28T16:50:28.816906+0000","last_peered":"2024-03-28T16:50:28.816906+0000","last_clean":"2024-03-28T16:50:28.816906+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:50:28.816906+0000","last_undegraded":"2024-03-28T16:50:28.816906+0000","last_fullsized":"2024-03-28T16:50:28.816906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:53.514285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":26,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586838+0000","last_change":"2024-03-28T16:49:02.714786+0000","last_active":"2024-03-28T16:49:37.586838+0000","last_peered":"2024-03-28T16:49:37.586838+0000","last_clean":"2024-03-28T16:49:37.586838+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:49:37.586838+0000","last_undegraded":"2024-03-28T16:49:37.586838+0000","last_fullsized":"2024-03-28T16:49:37.586838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582310+0000","last_change":"2024-03-28T16:49:04.753745+0000","last_active":"2024-03-28T16:49:37.582310+0000","last_peered":"2024-03-28T16:49:37.582310+0000","last_clean":"2024-03-28T16:49:37.582310+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:49:37.582310+0000","last_undegraded":"2024-03-28T16:49:37.582310+0000","last_fullsized":"2024-03-28T16:49:37.582310+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:59.796190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.619610+0000","last_change":"2024-03-28T16:49:37.618664+0000","last_active":"2024-03-28T16:49:37.619610+0000","last_peered":"2024-03-28T16:49:37.619610+0000","last_clean":"2024-03-28T16:49:37.619610+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:49:37.619610+0000","last_undegraded":"2024-03-28T16:49:37.619610+0000","last_fullsized":"2024-03-28T16:49:37.619610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:36.076211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588848+0000","last_change":"2024-03-28T16:48:54.658051+0000","last_active":"2024-03-28T16:49:37.588848+0000","last_peered":"2024-03-28T16:49:37.588848+0000","last_clean":"2024-03-28T16:49:37.588848+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:49:37.588848+0000","last_undegraded":"2024-03-28T16:49:37.588848+0000","last_fullsized":"2024-03-28T16:49:37.588848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190542+0000","last_change":"2024-03-28T16:48:56.682367+0000","last_active":"2024-03-28T16:50:23.190542+0000","last_peered":"2024-03-28T16:50:23.190542+0000","last_clean":"2024-03-28T16:50:23.190542+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:50:23.190542+0000","last_undegraded":"2024-03-28T16:50:23.190542+0000","last_fullsized":"2024-03-28T16:50:23.190542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584176+0000","last_change":"2024-03-28T16:48:54.655298+0000","last_active":"2024-03-28T16:49:37.584176+0000","last_peered":"2024-03-28T16:49:37.584176+0000","last_clean":"2024-03-28T16:49:37.584176+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:49:37.584176+0000","last_undegraded":"2024-03-28T16:49:37.584176+0000","last_fullsized":"2024-03-28T16:49:37.584176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'3","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370059+0000","last_change":"2024-03-28T16:48:56.671798+0000","last_active":"2024-03-28T16:50:22.370059+0000","last_peered":"2024-03-28T16:50:22.370059+0000","last_clean":"2024-03-28T16:50:22.370059+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:50:22.370059+0000","last_undegraded":"2024-03-28T16:50:22.370059+0000","last_fullsized":"2024-03-28T16:50:22.370059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:59.926897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589042+0000","last_change":"2024-03-28T16:48:54.657491+0000","last_active":"2024-03-28T16:49:37.589042+0000","last_peered":"2024-03-28T16:49:37.589042+0000","last_clean":"2024-03-28T16:49:37.589042+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:49:37.589042+0000","last_undegraded":"2024-03-28T16:49:37.589042+0000","last_fullsized":"2024-03-28T16:49:37.589042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:40:58.160099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586819+0000","last_change":"2024-03-28T16:48:56.683483+0000","last_active":"2024-03-28T16:49:37.586819+0000","last_peered":"2024-03-28T16:49:37.586819+0000","last_clean":"2024-03-28T16:49:37.586819+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:49:37.586819+0000","last_undegraded":"2024-03-28T16:49:37.586819+0000","last_fullsized":"2024-03-28T16:49:37.586819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588791+0000","last_change":"2024-03-28T16:48:54.665811+0000","last_active":"2024-03-28T16:49:37.588791+0000","last_peered":"2024-03-28T16:49:37.588791+0000","last_clean":"2024-03-28T16:49:37.588791+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:49:37.588791+0000","last_undegraded":"2024-03-28T16:49:37.588791+0000","last_fullsized":"2024-03-28T16:49:37.588791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:38.364800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591495+0000","last_change":"2024-03-28T16:48:56.677917+0000","last_active":"2024-03-28T16:49:37.591495+0000","last_peered":"2024-03-28T16:49:37.591495+0000","last_clean":"2024-03-28T16:49:37.591495+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:49:37.591495+0000","last_undegraded":"2024-03-28T16:49:37.591495+0000","last_fullsized":"2024-03-28T16:49:37.591495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587233+0000","last_change":"2024-03-28T16:48:54.655391+0000","last_active":"2024-03-28T16:49:37.587233+0000","last_peered":"2024-03-28T16:49:37.587233+0000","last_clean":"2024-03-28T16:49:37.587233+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:49:37.587233+0000","last_undegraded":"2024-03-28T16:49:37.587233+0000","last_fullsized":"2024-03-28T16:49:37.587233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840006+0000","last_change":"2024-03-28T16:48:56.678419+0000","last_active":"2024-03-28T16:50:28.840006+0000","last_peered":"2024-03-28T16:50:28.840006+0000","last_clean":"2024-03-28T16:50:28.840006+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:50:28.840006+0000","last_undegraded":"2024-03-28T16:50:28.840006+0000","last_fullsized":"2024-03-28T16:50:28.840006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:03.478538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595229+0000","last_change":"2024-03-28T16:48:54.669640+0000","last_active":"2024-03-28T16:49:37.595229+0000","last_peered":"2024-03-28T16:49:37.595229+0000","last_clean":"2024-03-28T16:49:37.595229+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:49:37.595229+0000","last_undegraded":"2024-03-28T16:49:37.595229+0000","last_fullsized":"2024-03-28T16:49:37.595229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527969+0000","last_change":"2024-03-28T16:48:56.689880+0000","last_active":"2024-03-28T16:49:39.527969+0000","last_peered":"2024-03-28T16:49:39.527969+0000","last_clean":"2024-03-28T16:49:39.527969+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:49:39.527969+0000","last_undegraded":"2024-03-28T16:49:39.527969+0000","last_fullsized":"2024-03-28T16:49:39.527969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598088+0000","last_change":"2024-03-28T16:48:54.661594+0000","last_active":"2024-03-28T16:49:37.598088+0000","last_peered":"2024-03-28T16:49:37.598088+0000","last_clean":"2024-03-28T16:49:37.598088+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:49:37.598088+0000","last_undegraded":"2024-03-28T16:49:37.598088+0000","last_fullsized":"2024-03-28T16:49:37.598088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'5","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839320+0000","last_change":"2024-03-28T16:48:56.679523+0000","last_active":"2024-03-28T16:50:28.839320+0000","last_peered":"2024-03-28T16:50:28.839320+0000","last_clean":"2024-03-28T16:50:28.839320+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:50:28.839320+0000","last_undegraded":"2024-03-28T16:50:28.839320+0000","last_fullsized":"2024-03-28T16:50:28.839320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588478+0000","last_change":"2024-03-28T16:48:54.659870+0000","last_active":"2024-03-28T16:49:37.588478+0000","last_peered":"2024-03-28T16:49:37.588478+0000","last_clean":"2024-03-28T16:49:37.588478+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:49:37.588478+0000","last_undegraded":"2024-03-28T16:49:37.588478+0000","last_fullsized":"2024-03-28T16:49:37.588478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.526775+0000","last_change":"2024-03-28T16:48:56.687788+0000","last_active":"2024-03-28T16:49:39.526775+0000","last_peered":"2024-03-28T16:49:39.526775+0000","last_clean":"2024-03-28T16:49:39.526775+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:49:39.526775+0000","last_undegraded":"2024-03-28T16:49:39.526775+0000","last_fullsized":"2024-03-28T16:49:39.526775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:05.073812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581474+0000","last_change":"2024-03-28T16:48:54.657662+0000","last_active":"2024-03-28T16:49:37.581474+0000","last_peered":"2024-03-28T16:49:37.581474+0000","last_clean":"2024-03-28T16:49:37.581474+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:49:37.581474+0000","last_undegraded":"2024-03-28T16:49:37.581474+0000","last_fullsized":"2024-03-28T16:49:37.581474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:59.369957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'1","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369831+0000","last_change":"2024-03-28T16:48:56.682872+0000","last_active":"2024-03-28T16:50:22.369831+0000","last_peered":"2024-03-28T16:50:22.369831+0000","last_clean":"2024-03-28T16:50:22.369831+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:50:22.369831+0000","last_undegraded":"2024-03-28T16:50:22.369831+0000","last_fullsized":"2024-03-28T16:50:22.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:50.772258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584620+0000","last_change":"2024-03-28T16:48:54.663496+0000","last_active":"2024-03-28T16:49:37.584620+0000","last_peered":"2024-03-28T16:49:37.584620+0000","last_clean":"2024-03-28T16:49:37.584620+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:49:37.584620+0000","last_undegraded":"2024-03-28T16:49:37.584620+0000","last_fullsized":"2024-03-28T16:49:37.584620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587192+0000","last_change":"2024-03-28T16:48:56.678342+0000","last_active":"2024-03-28T16:49:37.587192+0000","last_peered":"2024-03-28T16:49:37.587192+0000","last_clean":"2024-03-28T16:49:37.587192+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:49:37.587192+0000","last_undegraded":"2024-03-28T16:49:37.587192+0000","last_fullsized":"2024-03-28T16:49:37.587192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585680+0000","last_change":"2024-03-28T16:48:54.675150+0000","last_active":"2024-03-28T16:49:37.585680+0000","last_peered":"2024-03-28T16:49:37.585680+0000","last_clean":"2024-03-28T16:49:37.585680+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:49:37.585680+0000","last_undegraded":"2024-03-28T16:49:37.585680+0000","last_fullsized":"2024-03-28T16:49:37.585680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190164+0000","last_change":"2024-03-28T16:48:56.671157+0000","last_active":"2024-03-28T16:50:23.190164+0000","last_peered":"2024-03-28T16:50:23.190164+0000","last_clean":"2024-03-28T16:50:23.190164+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:50:23.190164+0000","last_undegraded":"2024-03-28T16:50:23.190164+0000","last_fullsized":"2024-03-28T16:50:23.190164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:59.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585793+0000","last_change":"2024-03-28T16:48:54.674973+0000","last_active":"2024-03-28T16:49:37.585793+0000","last_peered":"2024-03-28T16:49:37.585793+0000","last_clean":"2024-03-28T16:49:37.585793+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:49:37.585793+0000","last_undegraded":"2024-03-28T16:49:37.585793+0000","last_fullsized":"2024-03-28T16:49:37.585793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840005+0000","last_change":"2024-03-28T16:48:56.683829+0000","last_active":"2024-03-28T16:50:28.840005+0000","last_peered":"2024-03-28T16:50:28.840005+0000","last_clean":"2024-03-28T16:50:28.840005+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:50:28.840005+0000","last_undegraded":"2024-03-28T16:50:28.840005+0000","last_fullsized":"2024-03-28T16:50:28.840005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587774+0000","last_change":"2024-03-28T16:48:54.676815+0000","last_active":"2024-03-28T16:49:37.587774+0000","last_peered":"2024-03-28T16:49:37.587774+0000","last_clean":"2024-03-28T16:49:37.587774+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:49:37.587774+0000","last_undegraded":"2024-03-28T16:49:37.587774+0000","last_fullsized":"2024-03-28T16:49:37.587774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:19.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839390+0000","last_change":"2024-03-28T16:48:56.693060+0000","last_active":"2024-03-28T16:50:28.839390+0000","last_peered":"2024-03-28T16:50:28.839390+0000","last_clean":"2024-03-28T16:50:28.839390+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:50:28.839390+0000","last_undegraded":"2024-03-28T16:50:28.839390+0000","last_fullsized":"2024-03-28T16:50:28.839390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589164+0000","last_change":"2024-03-28T16:48:54.663003+0000","last_active":"2024-03-28T16:49:37.589164+0000","last_peered":"2024-03-28T16:49:37.589164+0000","last_clean":"2024-03-28T16:49:37.589164+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:49:37.589164+0000","last_undegraded":"2024-03-28T16:49:37.589164+0000","last_fullsized":"2024-03-28T16:49:37.589164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:43.737071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.078594+0000","last_change":"2024-03-28T16:48:56.685753+0000","last_active":"2024-03-28T16:49:39.078594+0000","last_peered":"2024-03-28T16:49:39.078594+0000","last_clean":"2024-03-28T16:49:39.078594+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:49:39.078594+0000","last_undegraded":"2024-03-28T16:49:39.078594+0000","last_fullsized":"2024-03-28T16:49:39.078594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:12.442441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598216+0000","last_change":"2024-03-28T16:48:54.661896+0000","last_active":"2024-03-28T16:49:37.598216+0000","last_peered":"2024-03-28T16:49:37.598216+0000","last_clean":"2024-03-28T16:49:37.598216+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:49:37.598216+0000","last_undegraded":"2024-03-28T16:49:37.598216+0000","last_fullsized":"2024-03-28T16:49:37.598216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'9","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877960+0000","last_change":"2024-03-28T16:48:56.680119+0000","last_active":"2024-03-28T16:49:39.877960+0000","last_peered":"2024-03-28T16:49:39.877960+0000","last_clean":"2024-03-28T16:49:39.877960+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:49:39.877960+0000","last_undegraded":"2024-03-28T16:49:39.877960+0000","last_fullsized":"2024-03-28T16:49:39.877960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588667+0000","last_change":"2024-03-28T16:48:54.665812+0000","last_active":"2024-03-28T16:49:37.588667+0000","last_peered":"2024-03-28T16:49:37.588667+0000","last_clean":"2024-03-28T16:49:37.588667+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:49:37.588667+0000","last_undegraded":"2024-03-28T16:49:37.588667+0000","last_fullsized":"2024-03-28T16:49:37.588667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:31.951558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581570+0000","last_change":"2024-03-28T16:48:54.665260+0000","last_active":"2024-03-28T16:49:37.581570+0000","last_peered":"2024-03-28T16:49:37.581570+0000","last_clean":"2024-03-28T16:49:37.581570+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:49:37.581570+0000","last_undegraded":"2024-03-28T16:49:37.581570+0000","last_fullsized":"2024-03-28T16:49:37.581570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:37.463264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591429+0000","last_change":"2024-03-28T16:48:56.690793+0000","last_active":"2024-03-28T16:49:37.591429+0000","last_peered":"2024-03-28T16:49:37.591429+0000","last_clean":"2024-03-28T16:49:37.591429+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:49:37.591429+0000","last_undegraded":"2024-03-28T16:49:37.591429+0000","last_fullsized":"2024-03-28T16:49:37.591429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584402+0000","last_change":"2024-03-28T16:48:54.663735+0000","last_active":"2024-03-28T16:49:37.584402+0000","last_peered":"2024-03-28T16:49:37.584402+0000","last_clean":"2024-03-28T16:49:37.584402+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:49:37.584402+0000","last_undegraded":"2024-03-28T16:49:37.584402+0000","last_fullsized":"2024-03-28T16:49:37.584402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587762+0000","last_change":"2024-03-28T16:48:56.683832+0000","last_active":"2024-03-28T16:49:37.587762+0000","last_peered":"2024-03-28T16:49:37.587762+0000","last_clean":"2024-03-28T16:49:37.587762+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:49:37.587762+0000","last_undegraded":"2024-03-28T16:49:37.587762+0000","last_fullsized":"2024-03-28T16:49:37.587762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:38.808030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1968,"ondisk_log_size":1968,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1344,"num_read_kb":1127,"num_write":517,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1175,"ondisk_log_size":1175,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":854,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39916,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331924,"statfs":{"total":96636764160,"available":96595890176,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.752}]},{"osd":2,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.51900000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.75},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64200000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705691,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78500000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705691,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:50:42.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:50:43.070 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-03-28T16:50:43.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.094+0000 7f75ee57c640 1 Processor -- start 2024-03-28T16:50:43.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.094+0000 7f75ee57c640 1 -- start start 2024-03-28T16:50:43.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.094+0000 7f75ee57c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f75e8094500 0x7f75e8133510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.094+0000 7f75ee57c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e8095170 con 0x7f75e8093a10 2024-03-28T16:50:43.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.094+0000 7f75ee57c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e8072550 con 0x7f75e8094500 2024-03-28T16:50:43.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f75e8094500 0x7f75e8133510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.100 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e7fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f75e8093a10 0x7f75e8093df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43794/0 (socket says 172.21.15.119:43794) 2024-03-28T16:50:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e7fff640 1 -- 172.21.15.119:0/2950596212 learned_addr learned my addr 172.21.15.119:0/2950596212 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e77fe640 1 --2- 172.21.15.119:0/2950596212 >> v2:172.21.15.119:3300/0 conn(0x7f75e8094500 0x7f75e8133510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34106/0 (socket says 172.21.15.119:34106) 2024-03-28T16:50:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e77fe640 1 -- 172.21.15.119:0/2950596212 >> v1:172.21.15.119:6789/0 conn(0x7f75e8093a10 legacy=0x7f75e8093df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.101 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e77fe640 1 -- 172.21.15.119:0/2950596212 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8094bb0 con 0x7f75e8094500 2024-03-28T16:50:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e77fe640 1 --2- 172.21.15.119:0/2950596212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8094500 0x7f75e8133510 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009740 tx=0x7f75d801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b421e51a6a0b8f1f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e6ffd640 1 -- 172.21.15.119:0/2950596212 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d80ac570 con 0x7f75e8094500 2024-03-28T16:50:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e6ffd640 1 -- 172.21.15.119:0/2950596212 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d80ac6d0 con 0x7f75e8094500 2024-03-28T16:50:43.102 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75e6ffd640 1 -- 172.21.15.119:0/2950596212 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75d80ac990 con 0x7f75e8094500 2024-03-28T16:50:43.103 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75ee57c640 1 -- 172.21.15.119:0/2950596212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8094500 msgr2=0x7f75e8133510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75ee57c640 1 --2- 172.21.15.119:0/2950596212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8094500 0x7f75e8133510 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009740 tx=0x7f75d801c370 comp rx=0 tx=0).stop 2024-03-28T16:50:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75ee57c640 1 -- 172.21.15.119:0/2950596212 shutdown_connections 2024-03-28T16:50:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75ee57c640 1 --2- 172.21.15.119:0/2950596212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8094500 0x7f75e8133510 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75ee57c640 1 -- 172.21.15.119:0/2950596212 >> 172.21.15.119:0/2950596212 conn(0x7f75e809a2f0 msgr2=0x7f75e809a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75ee57c640 1 -- 172.21.15.119:0/2950596212 shutdown_connections 2024-03-28T16:50:43.104 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.098+0000 7f75ee57c640 1 -- 172.21.15.119:0/2950596212 wait complete. 2024-03-28T16:50:43.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75ee57c640 1 Processor -- start 2024-03-28T16:50:43.105 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75ee57c640 1 -- start start 2024-03-28T16:50:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75ee57c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8093a10 0x7f75e8131ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75ee57c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e8124430 con 0x7f75e8093a10 2024-03-28T16:50:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8093a10 0x7f75e8131ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8093a10 0x7f75e8131ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34120/0 (socket says 172.21.15.119:34120) 2024-03-28T16:50:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e7fff640 1 -- 172.21.15.119:0/2413870456 learned_addr learned my addr 172.21.15.119:0/2413870456 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e7fff640 1 -- 172.21.15.119:0/2413870456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8009420 con 0x7f75e8093a10 2024-03-28T16:50:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e7fff640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8093a10 0x7f75e8131ad0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c480 tx=0x7f75dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e4ff9640 1 -- 172.21.15.119:0/2413870456 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75dc005c20 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e4ff9640 1 -- 172.21.15.119:0/2413870456 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc005d80 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e81254b0 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e4ff9640 1 -- 172.21.15.119:0/2413870456 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75dc0071a0 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e8124a30 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e4ff9640 1 -- 172.21.15.119:0/2413870456 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f75dc007300 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e8133240 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e4ff9640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f75d4035a60 0x7f75d4037f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e4ff9640 1 -- 172.21.15.119:0/2413870456 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f75dc047c40 con 0x7f75e8093a10 2024-03-28T16:50:43.108 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e77fe640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f75d4035a60 0x7f75d4037f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.109 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.102+0000 7f75e77fe640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f75d4035a60 0x7f75d4037f20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f75d80ae000 tx=0x7f75d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.110 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.106+0000 7f75e4ff9640 1 -- 172.21.15.119:0/2413870456 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75dc0075b0 con 0x7f75e8093a10 2024-03-28T16:50:43.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.258+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75e81252f0 con 0x7f75d4035a60 2024-03-28T16:50:43.307 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.302+0000 7f75e4ff9640 1 -- 172.21.15.119:0/2413870456 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+673101 (secure 0 0 0) 0x7f75e81252f0 con 0x7f75d4035a60 2024-03-28T16:50:43.311 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:50:43.318 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:50:43.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f75d4035a60 msgr2=0x7f75d4037f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f75d4035a60 0x7f75d4037f20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f75d80ae000 tx=0x7f75d8008040 comp rx=0 tx=0).stop 2024-03-28T16:50:43.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8093a10 msgr2=0x7f75e8131ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8093a10 0x7f75e8131ad0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c480 tx=0x7f75dc00c940 comp rx=0 tx=0).stop 2024-03-28T16:50:43.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 shutdown_connections 2024-03-28T16:50:43.321 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f75d4035a60 0x7f75d4037f20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 --2- 172.21.15.119:0/2413870456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75e8093a10 0x7f75e8131ad0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 >> 172.21.15.119:0/2413870456 conn(0x7f75e809a2f0 msgr2=0x7f75e806c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:43.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 shutdown_connections 2024-03-28T16:50:43.322 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.314+0000 7f75ee57c640 1 -- 172.21.15.119:0/2413870456 wait complete. 2024-03-28T16:50:43.339 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-03-28T16:50:42.533657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":3824,"num_read_kb":3471,"num_write":1479,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3745,"ondisk_log_size":3745,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":160768,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":157945,"kb_avail":377326592,"statfs":{"total":386547056640,"available":386382430208,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":161736470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":403,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":211,"num_read_kb":206,"num_write":80,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006262"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600005+0000","last_change":"2024-03-28T16:48:54.656021+0000","last_active":"2024-03-28T16:49:37.600005+0000","last_peered":"2024-03-28T16:49:37.600005+0000","last_clean":"2024-03-28T16:49:37.600005+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:49:37.600005+0000","last_undegraded":"2024-03-28T16:49:37.600005+0000","last_fullsized":"2024-03-28T16:49:37.600005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585246+0000","last_change":"2024-03-28T16:48:56.687475+0000","last_active":"2024-03-28T16:49:37.585246+0000","last_peered":"2024-03-28T16:49:37.585246+0000","last_clean":"2024-03-28T16:49:37.585246+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:49:37.585246+0000","last_undegraded":"2024-03-28T16:49:37.585246+0000","last_fullsized":"2024-03-28T16:49:37.585246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587869+0000","last_change":"2024-03-28T16:48:54.666372+0000","last_active":"2024-03-28T16:49:37.587869+0000","last_peered":"2024-03-28T16:49:37.587869+0000","last_clean":"2024-03-28T16:49:37.587869+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:49:37.587869+0000","last_undegraded":"2024-03-28T16:49:37.587869+0000","last_fullsized":"2024-03-28T16:49:37.587869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555560+0000","last_change":"2024-03-28T16:48:56.688050+0000","last_active":"2024-03-28T16:49:40.555560+0000","last_peered":"2024-03-28T16:49:40.555560+0000","last_clean":"2024-03-28T16:49:40.555560+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:49:40.555560+0000","last_undegraded":"2024-03-28T16:49:40.555560+0000","last_fullsized":"2024-03-28T16:49:40.555560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:32.377984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592873+0000","last_change":"2024-03-28T16:48:54.665976+0000","last_active":"2024-03-28T16:49:37.592873+0000","last_peered":"2024-03-28T16:49:37.592873+0000","last_clean":"2024-03-28T16:49:37.592873+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:49:37.592873+0000","last_undegraded":"2024-03-28T16:49:37.592873+0000","last_fullsized":"2024-03-28T16:49:37.592873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:38.265023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370127+0000","last_change":"2024-03-28T16:48:56.690734+0000","last_active":"2024-03-28T16:50:22.370127+0000","last_peered":"2024-03-28T16:50:22.370127+0000","last_clean":"2024-03-28T16:50:22.370127+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:50:22.370127+0000","last_undegraded":"2024-03-28T16:50:22.370127+0000","last_fullsized":"2024-03-28T16:50:22.370127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:26.690845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.374097+0000","last_change":"2024-03-28T16:48:54.656958+0000","last_active":"2024-03-28T16:50:22.374097+0000","last_peered":"2024-03-28T16:50:22.374097+0000","last_clean":"2024-03-28T16:50:22.374097+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:50:22.374097+0000","last_undegraded":"2024-03-28T16:50:22.374097+0000","last_fullsized":"2024-03-28T16:50:22.374097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.564230+0000","last_change":"2024-03-28T16:49:37.602262+0000","last_active":"2024-03-28T16:49:40.564230+0000","last_peered":"2024-03-28T16:49:40.564230+0000","last_clean":"2024-03-28T16:49:40.564230+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:49:40.564230+0000","last_undegraded":"2024-03-28T16:49:40.564230+0000","last_fullsized":"2024-03-28T16:49:40.564230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:12:52.595685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593536+0000","last_change":"2024-03-28T16:48:54.661971+0000","last_active":"2024-03-28T16:49:37.593536+0000","last_peered":"2024-03-28T16:49:37.593536+0000","last_clean":"2024-03-28T16:49:37.593536+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:49:37.593536+0000","last_undegraded":"2024-03-28T16:49:37.593536+0000","last_fullsized":"2024-03-28T16:49:37.593536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:29.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585408+0000","last_change":"2024-03-28T16:48:56.671464+0000","last_active":"2024-03-28T16:49:37.585408+0000","last_peered":"2024-03-28T16:49:37.585408+0000","last_clean":"2024-03-28T16:49:37.585408+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:49:37.585408+0000","last_undegraded":"2024-03-28T16:49:37.585408+0000","last_fullsized":"2024-03-28T16:49:37.585408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:18.173627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590098+0000","last_change":"2024-03-28T16:48:54.665454+0000","last_active":"2024-03-28T16:49:37.590098+0000","last_peered":"2024-03-28T16:49:37.590098+0000","last_clean":"2024-03-28T16:49:37.590098+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:49:37.590098+0000","last_undegraded":"2024-03-28T16:49:37.590098+0000","last_fullsized":"2024-03-28T16:49:37.590098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587928+0000","last_change":"2024-03-28T16:48:56.688375+0000","last_active":"2024-03-28T16:49:37.587928+0000","last_peered":"2024-03-28T16:49:37.587928+0000","last_clean":"2024-03-28T16:49:37.587928+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:49:37.587928+0000","last_undegraded":"2024-03-28T16:49:37.587928+0000","last_fullsized":"2024-03-28T16:49:37.587928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:56.236071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.859167+0000","last_change":"2024-03-28T16:48:54.672203+0000","last_active":"2024-03-28T16:49:39.859167+0000","last_peered":"2024-03-28T16:49:39.859167+0000","last_clean":"2024-03-28T16:49:39.859167+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:49:39.859167+0000","last_undegraded":"2024-03-28T16:49:39.859167+0000","last_fullsized":"2024-03-28T16:49:39.859167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:04:05.939269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555495+0000","last_change":"2024-03-28T16:48:56.695671+0000","last_active":"2024-03-28T16:49:40.555495+0000","last_peered":"2024-03-28T16:49:40.555495+0000","last_clean":"2024-03-28T16:49:40.555495+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:49:40.555495+0000","last_undegraded":"2024-03-28T16:49:40.555495+0000","last_fullsized":"2024-03-28T16:49:40.555495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585143+0000","last_change":"2024-03-28T16:48:54.658053+0000","last_active":"2024-03-28T16:49:37.585143+0000","last_peered":"2024-03-28T16:49:37.585143+0000","last_clean":"2024-03-28T16:49:37.585143+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:49:37.585143+0000","last_undegraded":"2024-03-28T16:49:37.585143+0000","last_fullsized":"2024-03-28T16:49:37.585143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839573+0000","last_change":"2024-03-28T16:48:56.681114+0000","last_active":"2024-03-28T16:50:28.839573+0000","last_peered":"2024-03-28T16:50:28.839573+0000","last_clean":"2024-03-28T16:50:28.839573+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:50:28.839573+0000","last_undegraded":"2024-03-28T16:50:28.839573+0000","last_fullsized":"2024-03-28T16:50:28.839573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527185+0000","last_change":"2024-03-28T16:48:56.677244+0000","last_active":"2024-03-28T16:49:39.527185+0000","last_peered":"2024-03-28T16:49:39.527185+0000","last_clean":"2024-03-28T16:49:39.527185+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:49:39.527185+0000","last_undegraded":"2024-03-28T16:49:39.527185+0000","last_fullsized":"2024-03-28T16:49:39.527185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:55.139244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602851+0000","last_change":"2024-03-28T16:48:54.671518+0000","last_active":"2024-03-28T16:49:37.602851+0000","last_peered":"2024-03-28T16:49:37.602851+0000","last_clean":"2024-03-28T16:49:37.602851+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:49:37.602851+0000","last_undegraded":"2024-03-28T16:49:37.602851+0000","last_fullsized":"2024-03-28T16:49:37.602851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'9","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.563949+0000","last_change":"2024-03-28T16:48:56.692955+0000","last_active":"2024-03-28T16:49:40.563949+0000","last_peered":"2024-03-28T16:49:40.563949+0000","last_clean":"2024-03-28T16:49:40.563949+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:49:40.563949+0000","last_undegraded":"2024-03-28T16:49:40.563949+0000","last_fullsized":"2024-03-28T16:49:40.563949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610918+0000","last_change":"2024-03-28T16:49:37.610918+0000","last_active":"2024-03-28T16:49:37.610918+0000","last_peered":"2024-03-28T16:49:37.610918+0000","last_clean":"2024-03-28T16:49:37.610918+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:49:37.610918+0000","last_undegraded":"2024-03-28T16:49:37.610918+0000","last_fullsized":"2024-03-28T16:49:37.610918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:21.127441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840116+0000","last_change":"2024-03-28T16:48:56.687940+0000","last_active":"2024-03-28T16:50:28.840116+0000","last_peered":"2024-03-28T16:50:28.840116+0000","last_clean":"2024-03-28T16:50:28.840116+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:50:28.840116+0000","last_undegraded":"2024-03-28T16:50:28.840116+0000","last_fullsized":"2024-03-28T16:50:28.840116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:40:18.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586038+0000","last_change":"2024-03-28T16:48:54.670148+0000","last_active":"2024-03-28T16:49:37.586038+0000","last_peered":"2024-03-28T16:49:37.586038+0000","last_clean":"2024-03-28T16:49:37.586038+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:49:37.586038+0000","last_undegraded":"2024-03-28T16:49:37.586038+0000","last_fullsized":"2024-03-28T16:49:37.586038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.865044+0000","last_change":"2024-03-28T16:48:56.678642+0000","last_active":"2024-03-28T16:49:39.865044+0000","last_peered":"2024-03-28T16:49:39.865044+0000","last_clean":"2024-03-28T16:49:39.865044+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:49:39.865044+0000","last_undegraded":"2024-03-28T16:49:39.865044+0000","last_fullsized":"2024-03-28T16:49:39.865044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586134+0000","last_change":"2024-03-28T16:48:54.655014+0000","last_active":"2024-03-28T16:49:37.586134+0000","last_peered":"2024-03-28T16:49:37.586134+0000","last_clean":"2024-03-28T16:49:37.586134+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:49:37.586134+0000","last_undegraded":"2024-03-28T16:49:37.586134+0000","last_fullsized":"2024-03-28T16:49:37.586134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.876760+0000","last_change":"2024-03-28T16:48:56.680884+0000","last_active":"2024-03-28T16:49:39.876760+0000","last_peered":"2024-03-28T16:49:39.876760+0000","last_clean":"2024-03-28T16:49:39.876760+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:49:39.876760+0000","last_undegraded":"2024-03-28T16:49:39.876760+0000","last_fullsized":"2024-03-28T16:49:39.876760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.842066+0000","last_change":"2024-03-28T16:48:54.670541+0000","last_active":"2024-03-28T16:49:39.842066+0000","last_peered":"2024-03-28T16:49:39.842066+0000","last_clean":"2024-03-28T16:49:39.842066+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:49:39.842066+0000","last_undegraded":"2024-03-28T16:49:39.842066+0000","last_fullsized":"2024-03-28T16:49:39.842066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854794+0000","last_change":"2024-03-28T16:48:56.671588+0000","last_active":"2024-03-28T16:50:28.854794+0000","last_peered":"2024-03-28T16:50:28.854794+0000","last_clean":"2024-03-28T16:50:28.854794+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:50:28.854794+0000","last_undegraded":"2024-03-28T16:50:28.854794+0000","last_fullsized":"2024-03-28T16:50:28.854794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:22.649572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582658+0000","last_change":"2024-03-28T16:48:54.673730+0000","last_active":"2024-03-28T16:49:37.582658+0000","last_peered":"2024-03-28T16:49:37.582658+0000","last_clean":"2024-03-28T16:49:37.582658+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:49:37.582658+0000","last_undegraded":"2024-03-28T16:49:37.582658+0000","last_fullsized":"2024-03-28T16:49:37.582658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:36:08.968340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'19","reported_seq":77,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.826736+0000","last_change":"2024-03-28T16:49:04.742579+0000","last_active":"2024-03-28T16:50:28.826736+0000","last_peered":"2024-03-28T16:50:28.826736+0000","last_clean":"2024-03-28T16:50:28.826736+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:50:28.826736+0000","last_undegraded":"2024-03-28T16:50:28.826736+0000","last_fullsized":"2024-03-28T16:50:28.826736+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:00.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839392+0000","last_change":"2024-03-28T16:48:56.683934+0000","last_active":"2024-03-28T16:50:28.839392+0000","last_peered":"2024-03-28T16:50:28.839392+0000","last_clean":"2024-03-28T16:50:28.839392+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:50:28.839392+0000","last_undegraded":"2024-03-28T16:50:28.839392+0000","last_fullsized":"2024-03-28T16:50:28.839392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:38.624478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592554+0000","last_change":"2024-03-28T16:48:54.670023+0000","last_active":"2024-03-28T16:49:37.592554+0000","last_peered":"2024-03-28T16:49:37.592554+0000","last_clean":"2024-03-28T16:49:37.592554+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:49:37.592554+0000","last_undegraded":"2024-03-28T16:49:37.592554+0000","last_fullsized":"2024-03-28T16:49:37.592554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587566+0000","last_change":"2024-03-28T16:48:58.699910+0000","last_active":"2024-03-28T16:49:37.587566+0000","last_peered":"2024-03-28T16:49:37.587566+0000","last_clean":"2024-03-28T16:49:37.587566+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:49:37.587566+0000","last_undegraded":"2024-03-28T16:49:37.587566+0000","last_fullsized":"2024-03-28T16:49:37.587566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:35.076776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.149453+0000","last_change":"2024-03-28T16:49:00.727669+0000","last_active":"2024-03-28T16:50:23.149453+0000","last_peered":"2024-03-28T16:50:23.149453+0000","last_clean":"2024-03-28T16:50:23.149453+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:50:23.149453+0000","last_undegraded":"2024-03-28T16:50:23.149453+0000","last_fullsized":"2024-03-28T16:50:23.149453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:28.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601464+0000","last_change":"2024-03-28T16:49:02.739439+0000","last_active":"2024-03-28T16:49:37.601464+0000","last_peered":"2024-03-28T16:49:37.601464+0000","last_clean":"2024-03-28T16:49:37.601464+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:49:37.601464+0000","last_undegraded":"2024-03-28T16:49:37.601464+0000","last_fullsized":"2024-03-28T16:49:37.601464+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'6","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.307391+0000","last_change":"2024-03-28T16:49:04.742984+0000","last_active":"2024-03-28T16:50:17.307391+0000","last_peered":"2024-03-28T16:50:17.307391+0000","last_clean":"2024-03-28T16:50:17.307391+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:50:17.307391+0000","last_undegraded":"2024-03-28T16:50:17.307391+0000","last_fullsized":"2024-03-28T16:50:17.307391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:30.200369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839461+0000","last_change":"2024-03-28T16:48:56.671488+0000","last_active":"2024-03-28T16:50:28.839461+0000","last_peered":"2024-03-28T16:50:28.839461+0000","last_clean":"2024-03-28T16:50:28.839461+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:50:28.839461+0000","last_undegraded":"2024-03-28T16:50:28.839461+0000","last_fullsized":"2024-03-28T16:50:28.839461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:39.819684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591228+0000","last_change":"2024-03-28T16:48:54.655701+0000","last_active":"2024-03-28T16:49:37.591228+0000","last_peered":"2024-03-28T16:49:37.591228+0000","last_clean":"2024-03-28T16:49:37.591228+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:49:37.591228+0000","last_undegraded":"2024-03-28T16:49:37.591228+0000","last_fullsized":"2024-03-28T16:49:37.591228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585880+0000","last_change":"2024-03-28T16:48:58.689758+0000","last_active":"2024-03-28T16:49:37.585880+0000","last_peered":"2024-03-28T16:49:37.585880+0000","last_clean":"2024-03-28T16:49:37.585880+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:49:37.585880+0000","last_undegraded":"2024-03-28T16:49:37.585880+0000","last_fullsized":"2024-03-28T16:49:37.585880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:08.600334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"29'7","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.605037+0000","last_change":"2024-03-28T16:49:00.717460+0000","last_active":"2024-03-28T16:49:37.605037+0000","last_peered":"2024-03-28T16:49:37.605037+0000","last_clean":"2024-03-28T16:49:37.605037+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:49:37.605037+0000","last_undegraded":"2024-03-28T16:49:37.605037+0000","last_fullsized":"2024-03-28T16:49:37.605037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:26.416340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586231+0000","last_change":"2024-03-28T16:49:02.730400+0000","last_active":"2024-03-28T16:49:37.586231+0000","last_peered":"2024-03-28T16:49:37.586231+0000","last_clean":"2024-03-28T16:49:37.586231+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:49:37.586231+0000","last_undegraded":"2024-03-28T16:49:37.586231+0000","last_fullsized":"2024-03-28T16:49:37.586231+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584337+0000","last_change":"2024-03-28T16:49:04.743161+0000","last_active":"2024-03-28T16:49:37.584337+0000","last_peered":"2024-03-28T16:49:37.584337+0000","last_clean":"2024-03-28T16:49:37.584337+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:49:37.584337+0000","last_undegraded":"2024-03-28T16:49:37.584337+0000","last_fullsized":"2024-03-28T16:49:37.584337+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189874+0000","last_change":"2024-03-28T16:48:56.676331+0000","last_active":"2024-03-28T16:50:23.189874+0000","last_peered":"2024-03-28T16:50:23.189874+0000","last_clean":"2024-03-28T16:50:23.189874+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:50:23.189874+0000","last_undegraded":"2024-03-28T16:50:23.189874+0000","last_fullsized":"2024-03-28T16:50:23.189874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:33.697214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592955+0000","last_change":"2024-03-28T16:48:54.659330+0000","last_active":"2024-03-28T16:49:37.592955+0000","last_peered":"2024-03-28T16:49:37.592955+0000","last_clean":"2024-03-28T16:49:37.592955+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:49:37.592955+0000","last_undegraded":"2024-03-28T16:49:37.592955+0000","last_fullsized":"2024-03-28T16:49:37.592955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:30:54.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'5","reported_seq":203,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.364471+0000","last_change":"2024-03-28T16:48:58.687156+0000","last_active":"2024-03-28T16:50:30.364471+0000","last_peered":"2024-03-28T16:50:30.364471+0000","last_clean":"2024-03-28T16:50:30.364471+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:50:30.364471+0000","last_undegraded":"2024-03-28T16:50:30.364471+0000","last_fullsized":"2024-03-28T16:50:30.364471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:25:52.416012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":267,"num_read_kb":266,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'11","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595817+0000","last_change":"2024-03-28T16:49:00.704257+0000","last_active":"2024-03-28T16:49:37.595817+0000","last_peered":"2024-03-28T16:49:37.595817+0000","last_clean":"2024-03-28T16:49:37.595817+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:49:37.595817+0000","last_undegraded":"2024-03-28T16:49:37.595817+0000","last_fullsized":"2024-03-28T16:49:37.595817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:59.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'246","reported_seq":329,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854124+0000","last_change":"2024-03-28T16:49:02.737451+0000","last_active":"2024-03-28T16:50:33.854124+0000","last_peered":"2024-03-28T16:50:33.854124+0000","last_clean":"2024-03-28T16:50:33.854124+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:50:33.854124+0000","last_undegraded":"2024-03-28T16:50:33.854124+0000","last_fullsized":"2024-03-28T16:50:33.854124+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:34.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591631+0000","last_change":"2024-03-28T16:49:04.741486+0000","last_active":"2024-03-28T16:49:37.591631+0000","last_peered":"2024-03-28T16:49:37.591631+0000","last_clean":"2024-03-28T16:49:37.591631+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:49:37.591631+0000","last_undegraded":"2024-03-28T16:49:37.591631+0000","last_fullsized":"2024-03-28T16:49:37.591631+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:51.610298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591048+0000","last_change":"2024-03-28T16:48:56.697774+0000","last_active":"2024-03-28T16:49:37.591048+0000","last_peered":"2024-03-28T16:49:37.591048+0000","last_clean":"2024-03-28T16:49:37.591048+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:49:37.591048+0000","last_undegraded":"2024-03-28T16:49:37.591048+0000","last_fullsized":"2024-03-28T16:49:37.591048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583578+0000","last_change":"2024-03-28T16:48:54.659035+0000","last_active":"2024-03-28T16:49:37.583578+0000","last_peered":"2024-03-28T16:49:37.583578+0000","last_clean":"2024-03-28T16:49:37.583578+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:49:37.583578+0000","last_undegraded":"2024-03-28T16:49:37.583578+0000","last_fullsized":"2024-03-28T16:49:37.583578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:59.516240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594556+0000","last_change":"2024-03-28T16:48:58.706776+0000","last_active":"2024-03-28T16:49:37.594556+0000","last_peered":"2024-03-28T16:49:37.594556+0000","last_clean":"2024-03-28T16:49:37.594556+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:49:37.594556+0000","last_undegraded":"2024-03-28T16:49:37.594556+0000","last_fullsized":"2024-03-28T16:49:37.594556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:18.066504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.142398+0000","last_change":"2024-03-28T16:49:00.710996+0000","last_active":"2024-03-28T16:50:23.142398+0000","last_peered":"2024-03-28T16:50:23.142398+0000","last_clean":"2024-03-28T16:50:23.142398+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:23.142398+0000","last_undegraded":"2024-03-28T16:50:23.142398+0000","last_fullsized":"2024-03-28T16:50:23.142398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:05.450361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'246","reported_seq":344,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.481011+0000","last_change":"2024-03-28T16:49:02.734376+0000","last_active":"2024-03-28T16:50:35.481011+0000","last_peered":"2024-03-28T16:50:35.481011+0000","last_clean":"2024-03-28T16:50:35.481011+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:50:35.481011+0000","last_undegraded":"2024-03-28T16:50:35.481011+0000","last_fullsized":"2024-03-28T16:50:35.481011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581627+0000","last_change":"2024-03-28T16:48:56.693376+0000","last_active":"2024-03-28T16:49:37.581627+0000","last_peered":"2024-03-28T16:49:37.581627+0000","last_clean":"2024-03-28T16:49:37.581627+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:49:37.581627+0000","last_undegraded":"2024-03-28T16:49:37.581627+0000","last_fullsized":"2024-03-28T16:49:37.581627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:27.504691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600235+0000","last_change":"2024-03-28T16:48:54.671982+0000","last_active":"2024-03-28T16:49:37.600235+0000","last_peered":"2024-03-28T16:49:37.600235+0000","last_clean":"2024-03-28T16:49:37.600235+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:49:37.600235+0000","last_undegraded":"2024-03-28T16:49:37.600235+0000","last_fullsized":"2024-03-28T16:49:37.600235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586619+0000","last_change":"2024-03-28T16:48:58.687997+0000","last_active":"2024-03-28T16:49:37.586619+0000","last_peered":"2024-03-28T16:49:37.586619+0000","last_clean":"2024-03-28T16:49:37.586619+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:49:37.586619+0000","last_undegraded":"2024-03-28T16:49:37.586619+0000","last_fullsized":"2024-03-28T16:49:37.586619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:29.519686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'38","reported_seq":84,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.155887+0000","last_change":"2024-03-28T16:49:00.713077+0000","last_active":"2024-03-28T16:50:23.155887+0000","last_peered":"2024-03-28T16:50:23.155887+0000","last_clean":"2024-03-28T16:50:23.155887+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:23.155887+0000","last_undegraded":"2024-03-28T16:50:23.155887+0000","last_fullsized":"2024-03-28T16:50:23.155887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:46.727245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587114+0000","last_change":"2024-03-28T16:49:02.730507+0000","last_active":"2024-03-28T16:49:37.587114+0000","last_peered":"2024-03-28T16:49:37.587114+0000","last_clean":"2024-03-28T16:49:37.587114+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:49:37.587114+0000","last_undegraded":"2024-03-28T16:49:37.587114+0000","last_fullsized":"2024-03-28T16:49:37.587114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.935069+0000","last_change":"2024-03-28T16:49:04.728316+0000","last_active":"2024-03-28T16:50:22.935069+0000","last_peered":"2024-03-28T16:50:22.935069+0000","last_clean":"2024-03-28T16:50:22.935069+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:22.935069+0000","last_undegraded":"2024-03-28T16:50:22.935069+0000","last_fullsized":"2024-03-28T16:50:22.935069+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:28.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'22","reported_seq":100,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729800+0000","last_change":"2024-03-28T16:48:56.678278+0000","last_active":"2024-03-28T16:50:22.729800+0000","last_peered":"2024-03-28T16:50:22.729800+0000","last_clean":"2024-03-28T16:50:22.729800+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:50:22.729800+0000","last_undegraded":"2024-03-28T16:50:22.729800+0000","last_fullsized":"2024-03-28T16:50:22.729800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.051186+0000","last_change":"2024-03-28T16:48:54.672277+0000","last_active":"2024-03-28T16:49:39.051186+0000","last_peered":"2024-03-28T16:49:39.051186+0000","last_clean":"2024-03-28T16:49:39.051186+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:49:39.051186+0000","last_undegraded":"2024-03-28T16:49:39.051186+0000","last_fullsized":"2024-03-28T16:49:39.051186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585905+0000","last_change":"2024-03-28T16:48:58.690263+0000","last_active":"2024-03-28T16:49:37.585905+0000","last_peered":"2024-03-28T16:49:37.585905+0000","last_clean":"2024-03-28T16:49:37.585905+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:49:37.585905+0000","last_undegraded":"2024-03-28T16:49:37.585905+0000","last_fullsized":"2024-03-28T16:49:37.585905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:25.224033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.129187+0000","last_change":"2024-03-28T16:49:00.727195+0000","last_active":"2024-03-28T16:50:23.129187+0000","last_peered":"2024-03-28T16:50:23.129187+0000","last_clean":"2024-03-28T16:50:23.129187+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:50:23.129187+0000","last_undegraded":"2024-03-28T16:50:23.129187+0000","last_fullsized":"2024-03-28T16:50:23.129187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:48.531527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585871+0000","last_change":"2024-03-28T16:49:02.714319+0000","last_active":"2024-03-28T16:49:37.585871+0000","last_peered":"2024-03-28T16:49:37.585871+0000","last_clean":"2024-03-28T16:49:37.585871+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:49:37.585871+0000","last_undegraded":"2024-03-28T16:49:37.585871+0000","last_fullsized":"2024-03-28T16:49:37.585871+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:49.522623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'3","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.157047+0000","last_change":"2024-03-28T16:49:04.738375+0000","last_active":"2024-03-28T16:50:17.157047+0000","last_peered":"2024-03-28T16:50:17.157047+0000","last_clean":"2024-03-28T16:50:17.157047+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:50:17.157047+0000","last_undegraded":"2024-03-28T16:50:17.157047+0000","last_fullsized":"2024-03-28T16:50:17.157047+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:31.198874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'3","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.537250+0000","last_change":"2024-03-28T16:48:56.681600+0000","last_active":"2024-03-28T16:49:39.537250+0000","last_peered":"2024-03-28T16:49:39.537250+0000","last_clean":"2024-03-28T16:49:39.537250+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:49:39.537250+0000","last_undegraded":"2024-03-28T16:49:39.537250+0000","last_fullsized":"2024-03-28T16:49:39.537250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588203+0000","last_change":"2024-03-28T16:48:54.676553+0000","last_active":"2024-03-28T16:49:37.588203+0000","last_peered":"2024-03-28T16:49:37.588203+0000","last_clean":"2024-03-28T16:49:37.588203+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:49:37.588203+0000","last_undegraded":"2024-03-28T16:49:37.588203+0000","last_fullsized":"2024-03-28T16:49:37.588203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:11.709923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592044+0000","last_change":"2024-03-28T16:48:58.682276+0000","last_active":"2024-03-28T16:49:37.592044+0000","last_peered":"2024-03-28T16:49:37.592044+0000","last_clean":"2024-03-28T16:49:37.592044+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:49:37.592044+0000","last_undegraded":"2024-03-28T16:49:37.592044+0000","last_fullsized":"2024-03-28T16:49:37.592044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:44.713820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.158232+0000","last_change":"2024-03-28T16:49:00.720840+0000","last_active":"2024-03-28T16:50:23.158232+0000","last_peered":"2024-03-28T16:50:23.158232+0000","last_clean":"2024-03-28T16:50:23.158232+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:50:23.158232+0000","last_undegraded":"2024-03-28T16:50:23.158232+0000","last_fullsized":"2024-03-28T16:50:23.158232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:28.746453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'246","reported_seq":339,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854481+0000","last_change":"2024-03-28T16:49:02.713412+0000","last_active":"2024-03-28T16:50:33.854481+0000","last_peered":"2024-03-28T16:50:33.854481+0000","last_clean":"2024-03-28T16:50:33.854481+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:33.854481+0000","last_undegraded":"2024-03-28T16:50:33.854481+0000","last_fullsized":"2024-03-28T16:50:33.854481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.828873+0000","last_change":"2024-03-28T16:49:04.743207+0000","last_active":"2024-03-28T16:50:28.828873+0000","last_peered":"2024-03-28T16:50:28.828873+0000","last_clean":"2024-03-28T16:50:28.828873+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:50:28.828873+0000","last_undegraded":"2024-03-28T16:50:28.828873+0000","last_fullsized":"2024-03-28T16:50:28.828873+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:12.429622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.864959+0000","last_change":"2024-03-28T16:48:56.685083+0000","last_active":"2024-03-28T16:49:39.864959+0000","last_peered":"2024-03-28T16:49:39.864959+0000","last_clean":"2024-03-28T16:49:39.864959+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:49:39.864959+0000","last_undegraded":"2024-03-28T16:49:39.864959+0000","last_fullsized":"2024-03-28T16:49:39.864959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587111+0000","last_change":"2024-03-28T16:48:54.658093+0000","last_active":"2024-03-28T16:49:37.587111+0000","last_peered":"2024-03-28T16:49:37.587111+0000","last_clean":"2024-03-28T16:49:37.587111+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:49:37.587111+0000","last_undegraded":"2024-03-28T16:49:37.587111+0000","last_fullsized":"2024-03-28T16:49:37.587111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:20.809568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586227+0000","last_change":"2024-03-28T16:48:58.682237+0000","last_active":"2024-03-28T16:49:37.586227+0000","last_peered":"2024-03-28T16:49:37.586227+0000","last_clean":"2024-03-28T16:49:37.586227+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:49:37.586227+0000","last_undegraded":"2024-03-28T16:49:37.586227+0000","last_fullsized":"2024-03-28T16:49:37.586227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:37.069394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'50","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.781121+0000","last_change":"2024-03-28T16:49:00.725827+0000","last_active":"2024-03-28T16:50:28.781121+0000","last_peered":"2024-03-28T16:50:28.781121+0000","last_clean":"2024-03-28T16:50:28.781121+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:50:28.781121+0000","last_undegraded":"2024-03-28T16:50:28.781121+0000","last_fullsized":"2024-03-28T16:50:28.781121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:56.316763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585610+0000","last_change":"2024-03-28T16:49:02.712054+0000","last_active":"2024-03-28T16:49:37.585610+0000","last_peered":"2024-03-28T16:49:37.585610+0000","last_clean":"2024-03-28T16:49:37.585610+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:49:37.585610+0000","last_undegraded":"2024-03-28T16:49:37.585610+0000","last_fullsized":"2024-03-28T16:49:37.585610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587403+0000","last_change":"2024-03-28T16:49:04.723184+0000","last_active":"2024-03-28T16:49:37.587403+0000","last_peered":"2024-03-28T16:49:37.587403+0000","last_clean":"2024-03-28T16:49:37.587403+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:49:37.587403+0000","last_undegraded":"2024-03-28T16:49:37.587403+0000","last_fullsized":"2024-03-28T16:49:37.587403+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:32.776436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877381+0000","last_change":"2024-03-28T16:48:56.683578+0000","last_active":"2024-03-28T16:49:39.877381+0000","last_peered":"2024-03-28T16:49:39.877381+0000","last_clean":"2024-03-28T16:49:39.877381+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:49:39.877381+0000","last_undegraded":"2024-03-28T16:49:39.877381+0000","last_fullsized":"2024-03-28T16:49:39.877381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:25.885438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588449+0000","last_change":"2024-03-28T16:48:54.664924+0000","last_active":"2024-03-28T16:49:37.588449+0000","last_peered":"2024-03-28T16:49:37.588449+0000","last_clean":"2024-03-28T16:49:37.588449+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:49:37.588449+0000","last_undegraded":"2024-03-28T16:49:37.588449+0000","last_fullsized":"2024-03-28T16:49:37.588449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:27:34.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595270+0000","last_change":"2024-03-28T16:48:58.692108+0000","last_active":"2024-03-28T16:49:37.595270+0000","last_peered":"2024-03-28T16:49:37.595270+0000","last_clean":"2024-03-28T16:49:37.595270+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:49:37.595270+0000","last_undegraded":"2024-03-28T16:49:37.595270+0000","last_fullsized":"2024-03-28T16:49:37.595270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:05.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'30","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.134000+0000","last_change":"2024-03-28T16:49:00.721579+0000","last_active":"2024-03-28T16:50:23.134000+0000","last_peered":"2024-03-28T16:50:23.134000+0000","last_clean":"2024-03-28T16:50:23.134000+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:23.134000+0000","last_undegraded":"2024-03-28T16:50:23.134000+0000","last_fullsized":"2024-03-28T16:50:23.134000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:13.394379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587827+0000","last_change":"2024-03-28T16:49:02.739236+0000","last_active":"2024-03-28T16:49:37.587827+0000","last_peered":"2024-03-28T16:49:37.587827+0000","last_clean":"2024-03-28T16:49:37.587827+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:49:37.587827+0000","last_undegraded":"2024-03-28T16:49:37.587827+0000","last_fullsized":"2024-03-28T16:49:37.587827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:34:20.723680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580108+0000","last_change":"2024-03-28T16:49:04.748645+0000","last_active":"2024-03-28T16:49:37.580108+0000","last_peered":"2024-03-28T16:49:37.580108+0000","last_clean":"2024-03-28T16:49:37.580108+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:49:37.580108+0000","last_undegraded":"2024-03-28T16:49:37.580108+0000","last_fullsized":"2024-03-28T16:49:37.580108+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:06:34.291471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593462+0000","last_change":"2024-03-28T16:48:56.695870+0000","last_active":"2024-03-28T16:49:37.593462+0000","last_peered":"2024-03-28T16:49:37.593462+0000","last_clean":"2024-03-28T16:49:37.593462+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:49:37.593462+0000","last_undegraded":"2024-03-28T16:49:37.593462+0000","last_fullsized":"2024-03-28T16:49:37.593462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:02.971815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589696+0000","last_change":"2024-03-28T16:48:54.673299+0000","last_active":"2024-03-28T16:49:37.589696+0000","last_peered":"2024-03-28T16:49:37.589696+0000","last_clean":"2024-03-28T16:49:37.589696+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:49:37.589696+0000","last_undegraded":"2024-03-28T16:49:37.589696+0000","last_fullsized":"2024-03-28T16:49:37.589696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:37.511744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588920+0000","last_change":"2024-03-28T16:48:58.683855+0000","last_active":"2024-03-28T16:49:37.588920+0000","last_peered":"2024-03-28T16:49:37.588920+0000","last_clean":"2024-03-28T16:49:37.588920+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:49:37.588920+0000","last_undegraded":"2024-03-28T16:49:37.588920+0000","last_fullsized":"2024-03-28T16:49:37.588920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:01.947384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'35","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.825556+0000","last_change":"2024-03-28T16:49:00.697730+0000","last_active":"2024-03-28T16:50:28.825556+0000","last_peered":"2024-03-28T16:50:28.825556+0000","last_clean":"2024-03-28T16:50:28.825556+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:50:28.825556+0000","last_undegraded":"2024-03-28T16:50:28.825556+0000","last_fullsized":"2024-03-28T16:50:28.825556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:27.038167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'246","reported_seq":322,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854703+0000","last_change":"2024-03-28T16:49:02.726890+0000","last_active":"2024-03-28T16:50:33.854703+0000","last_peered":"2024-03-28T16:50:33.854703+0000","last_clean":"2024-03-28T16:50:33.854703+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:33.854703+0000","last_undegraded":"2024-03-28T16:50:33.854703+0000","last_fullsized":"2024-03-28T16:50:33.854703+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:42.002449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'13","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.217530+0000","last_change":"2024-03-28T16:49:04.743257+0000","last_active":"2024-03-28T16:50:23.217530+0000","last_peered":"2024-03-28T16:50:23.217530+0000","last_clean":"2024-03-28T16:50:23.217530+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:50:23.217530+0000","last_undegraded":"2024-03-28T16:50:23.217530+0000","last_fullsized":"2024-03-28T16:50:23.217530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839667+0000","last_change":"2024-03-28T16:48:56.690053+0000","last_active":"2024-03-28T16:50:28.839667+0000","last_peered":"2024-03-28T16:50:28.839667+0000","last_clean":"2024-03-28T16:50:28.839667+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:50:28.839667+0000","last_undegraded":"2024-03-28T16:50:28.839667+0000","last_fullsized":"2024-03-28T16:50:28.839667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581989+0000","last_change":"2024-03-28T16:48:54.664571+0000","last_active":"2024-03-28T16:49:37.581989+0000","last_peered":"2024-03-28T16:49:37.581989+0000","last_clean":"2024-03-28T16:49:37.581989+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:49:37.581989+0000","last_undegraded":"2024-03-28T16:49:37.581989+0000","last_fullsized":"2024-03-28T16:49:37.581989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:56:22.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'3","reported_seq":378,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.377726+0000","last_change":"2024-03-28T16:49:37.622013+0000","last_active":"2024-03-28T16:50:30.377726+0000","last_peered":"2024-03-28T16:50:30.377726+0000","last_clean":"2024-03-28T16:50:30.377726+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:50:30.377726+0000","last_undegraded":"2024-03-28T16:50:30.377726+0000","last_fullsized":"2024-03-28T16:50:30.377726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:46.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":336,"num_read_kb":336,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587255+0000","last_change":"2024-03-28T16:49:00.725908+0000","last_active":"2024-03-28T16:49:37.587255+0000","last_peered":"2024-03-28T16:49:37.587255+0000","last_clean":"2024-03-28T16:49:37.587255+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:49:37.587255+0000","last_undegraded":"2024-03-28T16:49:37.587255+0000","last_fullsized":"2024-03-28T16:49:37.587255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:40.250784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'246","reported_seq":345,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854050+0000","last_change":"2024-03-28T16:49:02.715172+0000","last_active":"2024-03-28T16:50:33.854050+0000","last_peered":"2024-03-28T16:50:33.854050+0000","last_clean":"2024-03-28T16:50:33.854050+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:33.854050+0000","last_undegraded":"2024-03-28T16:50:33.854050+0000","last_fullsized":"2024-03-28T16:50:33.854050+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589680+0000","last_change":"2024-03-28T16:49:04.746465+0000","last_active":"2024-03-28T16:49:37.589680+0000","last_peered":"2024-03-28T16:49:37.589680+0000","last_clean":"2024-03-28T16:49:37.589680+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:49:37.589680+0000","last_undegraded":"2024-03-28T16:49:37.589680+0000","last_fullsized":"2024-03-28T16:49:37.589680+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:41.396912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189625+0000","last_change":"2024-03-28T16:48:56.681161+0000","last_active":"2024-03-28T16:50:23.189625+0000","last_peered":"2024-03-28T16:50:23.189625+0000","last_clean":"2024-03-28T16:50:23.189625+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:50:23.189625+0000","last_undegraded":"2024-03-28T16:50:23.189625+0000","last_fullsized":"2024-03-28T16:50:23.189625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:23.209038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580546+0000","last_change":"2024-03-28T16:48:54.676487+0000","last_active":"2024-03-28T16:49:37.580546+0000","last_peered":"2024-03-28T16:49:37.580546+0000","last_clean":"2024-03-28T16:49:37.580546+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:49:37.580546+0000","last_undegraded":"2024-03-28T16:49:37.580546+0000","last_fullsized":"2024-03-28T16:49:37.580546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:15.844441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588709+0000","last_change":"2024-03-28T16:48:58.703765+0000","last_active":"2024-03-28T16:49:37.588709+0000","last_peered":"2024-03-28T16:49:37.588709+0000","last_clean":"2024-03-28T16:49:37.588709+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:49:37.588709+0000","last_undegraded":"2024-03-28T16:49:37.588709+0000","last_fullsized":"2024-03-28T16:49:37.588709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:05:16.548217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'36","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.151478+0000","last_change":"2024-03-28T16:49:00.711767+0000","last_active":"2024-03-28T16:50:23.151478+0000","last_peered":"2024-03-28T16:50:23.151478+0000","last_clean":"2024-03-28T16:50:23.151478+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:23.151478+0000","last_undegraded":"2024-03-28T16:50:23.151478+0000","last_fullsized":"2024-03-28T16:50:23.151478+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:07.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590333+0000","last_change":"2024-03-28T16:49:02.720427+0000","last_active":"2024-03-28T16:49:37.590333+0000","last_peered":"2024-03-28T16:49:37.590333+0000","last_clean":"2024-03-28T16:49:37.590333+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:49:37.590333+0000","last_undegraded":"2024-03-28T16:49:37.590333+0000","last_fullsized":"2024-03-28T16:49:37.590333+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:02.281547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596396+0000","last_change":"2024-03-28T16:49:04.730860+0000","last_active":"2024-03-28T16:49:37.596396+0000","last_peered":"2024-03-28T16:49:37.596396+0000","last_clean":"2024-03-28T16:49:37.596396+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:49:37.596396+0000","last_undegraded":"2024-03-28T16:49:37.596396+0000","last_fullsized":"2024-03-28T16:49:37.596396+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189979+0000","last_change":"2024-03-28T16:48:56.693994+0000","last_active":"2024-03-28T16:50:23.189979+0000","last_peered":"2024-03-28T16:50:23.189979+0000","last_clean":"2024-03-28T16:50:23.189979+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:50:23.189979+0000","last_undegraded":"2024-03-28T16:50:23.189979+0000","last_fullsized":"2024-03-28T16:50:23.189979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:09.568996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":198,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583310+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:49:37.583310+0000","last_peered":"2024-03-28T16:49:37.583310+0000","last_clean":"2024-03-28T16:49:37.583310+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:49:37.583310+0000","last_undegraded":"2024-03-28T16:49:37.583310+0000","last_fullsized":"2024-03-28T16:49:37.583310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:35.744620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593784+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:49:37.593784+0000","last_peered":"2024-03-28T16:49:37.593784+0000","last_clean":"2024-03-28T16:49:37.593784+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:49:37.593784+0000","last_undegraded":"2024-03-28T16:49:37.593784+0000","last_fullsized":"2024-03-28T16:49:37.593784+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.194802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581636+0000","last_change":"2024-03-28T16:48:54.666099+0000","last_active":"2024-03-28T16:49:37.581636+0000","last_peered":"2024-03-28T16:49:37.581636+0000","last_clean":"2024-03-28T16:49:37.581636+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:49:37.581636+0000","last_undegraded":"2024-03-28T16:49:37.581636+0000","last_fullsized":"2024-03-28T16:49:37.581636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:03.153093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585495+0000","last_change":"2024-03-28T16:48:58.683594+0000","last_active":"2024-03-28T16:49:37.585495+0000","last_peered":"2024-03-28T16:49:37.585495+0000","last_clean":"2024-03-28T16:49:37.585495+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:49:37.585495+0000","last_undegraded":"2024-03-28T16:49:37.585495+0000","last_fullsized":"2024-03-28T16:49:37.585495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:14.457437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'70","reported_seq":138,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.807275+0000","last_change":"2024-03-28T16:49:00.715887+0000","last_active":"2024-03-28T16:50:28.807275+0000","last_peered":"2024-03-28T16:50:28.807275+0000","last_clean":"2024-03-28T16:50:28.807275+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:28.807275+0000","last_undegraded":"2024-03-28T16:50:28.807275+0000","last_fullsized":"2024-03-28T16:50:28.807275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:37:58.662402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591287+0000","last_change":"2024-03-28T16:49:02.731593+0000","last_active":"2024-03-28T16:49:37.591287+0000","last_peered":"2024-03-28T16:49:37.591287+0000","last_clean":"2024-03-28T16:49:37.591287+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:49:37.591287+0000","last_undegraded":"2024-03-28T16:49:37.591287+0000","last_fullsized":"2024-03-28T16:49:37.591287+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:36.899516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'52","reported_seq":114,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.870448+0000","last_change":"2024-03-28T16:49:04.752277+0000","last_active":"2024-03-28T16:50:28.870448+0000","last_peered":"2024-03-28T16:50:28.870448+0000","last_clean":"2024-03-28T16:50:28.870448+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:50:28.870448+0000","last_undegraded":"2024-03-28T16:50:28.870448+0000","last_fullsized":"2024-03-28T16:50:28.870448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:10.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'11","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.202723+0000","last_change":"2024-03-28T16:49:37.614693+0000","last_active":"2024-03-28T16:50:23.202723+0000","last_peered":"2024-03-28T16:50:23.202723+0000","last_clean":"2024-03-28T16:50:23.202723+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:50:23.202723+0000","last_undegraded":"2024-03-28T16:50:23.202723+0000","last_fullsized":"2024-03-28T16:50:23.202723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:53.196986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580404+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:49:37.580404+0000","last_peered":"2024-03-28T16:49:37.580404+0000","last_clean":"2024-03-28T16:49:37.580404+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:49:37.580404+0000","last_undegraded":"2024-03-28T16:49:37.580404+0000","last_fullsized":"2024-03-28T16:49:37.580404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:11.830948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585095+0000","last_change":"2024-03-28T16:48:54.662603+0000","last_active":"2024-03-28T16:49:37.585095+0000","last_peered":"2024-03-28T16:49:37.585095+0000","last_clean":"2024-03-28T16:49:37.585095+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:49:37.585095+0000","last_undegraded":"2024-03-28T16:49:37.585095+0000","last_fullsized":"2024-03-28T16:49:37.585095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594800+0000","last_change":"2024-03-28T16:48:58.706305+0000","last_active":"2024-03-28T16:49:37.594800+0000","last_peered":"2024-03-28T16:49:37.594800+0000","last_clean":"2024-03-28T16:49:37.594800+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:49:37.594800+0000","last_undegraded":"2024-03-28T16:49:37.594800+0000","last_fullsized":"2024-03-28T16:49:37.594800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:32.869035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'243","reported_seq":415,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854148+0000","last_change":"2024-03-28T16:49:00.718247+0000","last_active":"2024-03-28T16:50:33.854148+0000","last_peered":"2024-03-28T16:50:33.854148+0000","last_clean":"2024-03-28T16:50:33.854148+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:33.854148+0000","last_undegraded":"2024-03-28T16:50:33.854148+0000","last_fullsized":"2024-03-28T16:50:33.854148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:43.848104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":321,"num_read_kb":306,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582450+0000","last_change":"2024-03-28T16:49:02.740444+0000","last_active":"2024-03-28T16:49:37.582450+0000","last_peered":"2024-03-28T16:49:37.582450+0000","last_clean":"2024-03-28T16:49:37.582450+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:49:37.582450+0000","last_undegraded":"2024-03-28T16:49:37.582450+0000","last_fullsized":"2024-03-28T16:49:37.582450+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:41.589208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'4","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.938249+0000","last_change":"2024-03-28T16:49:04.744802+0000","last_active":"2024-03-28T16:50:22.938249+0000","last_peered":"2024-03-28T16:50:22.938249+0000","last_clean":"2024-03-28T16:50:22.938249+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:50:22.938249+0000","last_undegraded":"2024-03-28T16:50:22.938249+0000","last_fullsized":"2024-03-28T16:50:22.938249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.556077+0000","last_change":"2024-03-28T16:48:56.689836+0000","last_active":"2024-03-28T16:49:40.556077+0000","last_peered":"2024-03-28T16:49:40.556077+0000","last_clean":"2024-03-28T16:49:40.556077+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:49:40.556077+0000","last_undegraded":"2024-03-28T16:49:40.556077+0000","last_fullsized":"2024-03-28T16:49:40.556077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589208+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:49:37.589208+0000","last_peered":"2024-03-28T16:49:37.589208+0000","last_clean":"2024-03-28T16:49:37.589208+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:49:37.589208+0000","last_undegraded":"2024-03-28T16:49:37.589208+0000","last_fullsized":"2024-03-28T16:49:37.589208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584655+0000","last_change":"2024-03-28T16:48:54.663058+0000","last_active":"2024-03-28T16:49:37.584655+0000","last_peered":"2024-03-28T16:49:37.584655+0000","last_clean":"2024-03-28T16:49:37.584655+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:49:37.584655+0000","last_undegraded":"2024-03-28T16:49:37.584655+0000","last_fullsized":"2024-03-28T16:49:37.584655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"22'1","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.599332+0000","last_change":"2024-03-28T16:48:58.687188+0000","last_active":"2024-03-28T16:49:37.599332+0000","last_peered":"2024-03-28T16:49:37.599332+0000","last_clean":"2024-03-28T16:49:37.599332+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:49:37.599332+0000","last_undegraded":"2024-03-28T16:49:37.599332+0000","last_fullsized":"2024-03-28T16:49:37.599332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:27.900434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'16","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854777+0000","last_change":"2024-03-28T16:49:00.713728+0000","last_active":"2024-03-28T16:50:33.854777+0000","last_peered":"2024-03-28T16:50:33.854777+0000","last_clean":"2024-03-28T16:50:33.854777+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:33.854777+0000","last_undegraded":"2024-03-28T16:50:33.854777+0000","last_fullsized":"2024-03-28T16:50:33.854777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:03.960810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586565+0000","last_change":"2024-03-28T16:49:02.732230+0000","last_active":"2024-03-28T16:49:37.586565+0000","last_peered":"2024-03-28T16:49:37.586565+0000","last_clean":"2024-03-28T16:49:37.586565+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:49:37.586565+0000","last_undegraded":"2024-03-28T16:49:37.586565+0000","last_fullsized":"2024-03-28T16:49:37.586565+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:20.091845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581243+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:49:37.581243+0000","last_peered":"2024-03-28T16:49:37.581243+0000","last_clean":"2024-03-28T16:49:37.581243+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:49:37.581243+0000","last_undegraded":"2024-03-28T16:49:37.581243+0000","last_fullsized":"2024-03-28T16:49:37.581243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:51.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587517+0000","last_change":"2024-03-28T16:48:56.683769+0000","last_active":"2024-03-28T16:49:37.587517+0000","last_peered":"2024-03-28T16:49:37.587517+0000","last_clean":"2024-03-28T16:49:37.587517+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:49:37.587517+0000","last_undegraded":"2024-03-28T16:49:37.587517+0000","last_fullsized":"2024-03-28T16:49:37.587517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581400+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:49:37.581400+0000","last_peered":"2024-03-28T16:49:37.581400+0000","last_clean":"2024-03-28T16:49:37.581400+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:49:37.581400+0000","last_undegraded":"2024-03-28T16:49:37.581400+0000","last_fullsized":"2024-03-28T16:49:37.581400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:55:50.278254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591305+0000","last_change":"2024-03-28T16:48:54.666472+0000","last_active":"2024-03-28T16:49:37.591305+0000","last_peered":"2024-03-28T16:49:37.591305+0000","last_clean":"2024-03-28T16:49:37.591305+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:49:37.591305+0000","last_undegraded":"2024-03-28T16:49:37.591305+0000","last_fullsized":"2024-03-28T16:49:37.591305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587375+0000","last_change":"2024-03-28T16:48:58.688005+0000","last_active":"2024-03-28T16:49:37.587375+0000","last_peered":"2024-03-28T16:49:37.587375+0000","last_clean":"2024-03-28T16:49:37.587375+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:49:37.587375+0000","last_undegraded":"2024-03-28T16:49:37.587375+0000","last_fullsized":"2024-03-28T16:49:37.587375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:06.936540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'13","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595894+0000","last_change":"2024-03-28T16:49:00.713939+0000","last_active":"2024-03-28T16:49:37.595894+0000","last_peered":"2024-03-28T16:49:37.595894+0000","last_clean":"2024-03-28T16:49:37.595894+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:49:37.595894+0000","last_undegraded":"2024-03-28T16:49:37.595894+0000","last_fullsized":"2024-03-28T16:49:37.595894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:00:16.840810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588068+0000","last_change":"2024-03-28T16:49:02.741194+0000","last_active":"2024-03-28T16:49:37.588068+0000","last_peered":"2024-03-28T16:49:37.588068+0000","last_clean":"2024-03-28T16:49:37.588068+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:49:37.588068+0000","last_undegraded":"2024-03-28T16:49:37.588068+0000","last_fullsized":"2024-03-28T16:49:37.588068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:56.754215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585813+0000","last_change":"2024-03-28T16:49:04.741719+0000","last_active":"2024-03-28T16:49:37.585813+0000","last_peered":"2024-03-28T16:49:37.585813+0000","last_clean":"2024-03-28T16:49:37.585813+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:49:37.585813+0000","last_undegraded":"2024-03-28T16:49:37.585813+0000","last_fullsized":"2024-03-28T16:49:37.585813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:44.134615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555275+0000","last_change":"2024-03-28T16:48:56.687243+0000","last_active":"2024-03-28T16:49:40.555275+0000","last_peered":"2024-03-28T16:49:40.555275+0000","last_clean":"2024-03-28T16:49:40.555275+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:49:40.555275+0000","last_undegraded":"2024-03-28T16:49:40.555275+0000","last_fullsized":"2024-03-28T16:49:40.555275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596299+0000","last_change":"2024-03-28T16:48:54.657105+0000","last_active":"2024-03-28T16:49:37.596299+0000","last_peered":"2024-03-28T16:49:37.596299+0000","last_clean":"2024-03-28T16:49:37.596299+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:49:37.596299+0000","last_undegraded":"2024-03-28T16:49:37.596299+0000","last_fullsized":"2024-03-28T16:49:37.596299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586158+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:49:37.586158+0000","last_peered":"2024-03-28T16:49:37.586158+0000","last_clean":"2024-03-28T16:49:37.586158+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:49:37.586158+0000","last_undegraded":"2024-03-28T16:49:37.586158+0000","last_fullsized":"2024-03-28T16:49:37.586158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588917+0000","last_change":"2024-03-28T16:48:58.690031+0000","last_active":"2024-03-28T16:49:37.588917+0000","last_peered":"2024-03-28T16:49:37.588917+0000","last_clean":"2024-03-28T16:49:37.588917+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:49:37.588917+0000","last_undegraded":"2024-03-28T16:49:37.588917+0000","last_fullsized":"2024-03-28T16:49:37.588917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:29.762391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'57","reported_seq":115,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.796013+0000","last_change":"2024-03-28T16:49:00.706797+0000","last_active":"2024-03-28T16:50:28.796013+0000","last_peered":"2024-03-28T16:50:28.796013+0000","last_clean":"2024-03-28T16:50:28.796013+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:28.796013+0000","last_undegraded":"2024-03-28T16:50:28.796013+0000","last_fullsized":"2024-03-28T16:50:28.796013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:30.719266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590150+0000","last_change":"2024-03-28T16:49:02.720800+0000","last_active":"2024-03-28T16:49:37.590150+0000","last_peered":"2024-03-28T16:49:37.590150+0000","last_clean":"2024-03-28T16:49:37.590150+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:49:37.590150+0000","last_undegraded":"2024-03-28T16:49:37.590150+0000","last_fullsized":"2024-03-28T16:49:37.590150+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:07.770684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'9","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.211945+0000","last_change":"2024-03-28T16:49:04.741551+0000","last_active":"2024-03-28T16:50:30.211945+0000","last_peered":"2024-03-28T16:50:30.211945+0000","last_clean":"2024-03-28T16:50:30.211945+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:50:30.211945+0000","last_undegraded":"2024-03-28T16:50:30.211945+0000","last_fullsized":"2024-03-28T16:50:30.211945+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:55.072698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190216+0000","last_change":"2024-03-28T16:48:56.683849+0000","last_active":"2024-03-28T16:50:23.190216+0000","last_peered":"2024-03-28T16:50:23.190216+0000","last_clean":"2024-03-28T16:50:23.190216+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:50:23.190216+0000","last_undegraded":"2024-03-28T16:50:23.190216+0000","last_fullsized":"2024-03-28T16:50:23.190216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:57.646185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586877+0000","last_change":"2024-03-28T16:48:54.663076+0000","last_active":"2024-03-28T16:49:37.586877+0000","last_peered":"2024-03-28T16:49:37.586877+0000","last_clean":"2024-03-28T16:49:37.586877+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:49:37.586877+0000","last_undegraded":"2024-03-28T16:49:37.586877+0000","last_fullsized":"2024-03-28T16:49:37.586877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592775+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:49:37.592775+0000","last_peered":"2024-03-28T16:49:37.592775+0000","last_clean":"2024-03-28T16:49:37.592775+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:49:37.592775+0000","last_undegraded":"2024-03-28T16:49:37.592775+0000","last_fullsized":"2024-03-28T16:49:37.592775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:37.509224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582103+0000","last_change":"2024-03-28T16:48:58.683037+0000","last_active":"2024-03-28T16:49:37.582103+0000","last_peered":"2024-03-28T16:49:37.582103+0000","last_clean":"2024-03-28T16:49:37.582103+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:49:37.582103+0000","last_undegraded":"2024-03-28T16:49:37.582103+0000","last_fullsized":"2024-03-28T16:49:37.582103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:16.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'14","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595604+0000","last_change":"2024-03-28T16:49:00.720928+0000","last_active":"2024-03-28T16:49:37.595604+0000","last_peered":"2024-03-28T16:49:37.595604+0000","last_clean":"2024-03-28T16:49:37.595604+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:49:37.595604+0000","last_undegraded":"2024-03-28T16:49:37.595604+0000","last_fullsized":"2024-03-28T16:49:37.595604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:03.955318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'246","reported_seq":291,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.853994+0000","last_change":"2024-03-28T16:49:02.715814+0000","last_active":"2024-03-28T16:50:33.853994+0000","last_peered":"2024-03-28T16:50:33.853994+0000","last_clean":"2024-03-28T16:50:33.853994+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:33.853994+0000","last_undegraded":"2024-03-28T16:50:33.853994+0000","last_fullsized":"2024-03-28T16:50:33.853994+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.177166+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:23.177166+0000","last_peered":"2024-03-28T16:50:23.177166+0000","last_clean":"2024-03-28T16:50:23.177166+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:23.177166+0000","last_undegraded":"2024-03-28T16:50:23.177166+0000","last_fullsized":"2024-03-28T16:50:23.177166+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:14:04.371488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189696+0000","last_change":"2024-03-28T16:48:56.693531+0000","last_active":"2024-03-28T16:50:23.189696+0000","last_peered":"2024-03-28T16:50:23.189696+0000","last_clean":"2024-03-28T16:50:23.189696+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:50:23.189696+0000","last_undegraded":"2024-03-28T16:50:23.189696+0000","last_fullsized":"2024-03-28T16:50:23.189696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:22.327091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.850113+0000","last_change":"2024-03-28T16:48:54.662217+0000","last_active":"2024-03-28T16:49:39.850113+0000","last_peered":"2024-03-28T16:49:39.850113+0000","last_clean":"2024-03-28T16:49:39.850113+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:49:39.850113+0000","last_undegraded":"2024-03-28T16:49:39.850113+0000","last_fullsized":"2024-03-28T16:49:39.850113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:59.868215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591024+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:49:37.591024+0000","last_peered":"2024-03-28T16:49:37.591024+0000","last_clean":"2024-03-28T16:49:37.591024+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:49:37.591024+0000","last_undegraded":"2024-03-28T16:49:37.591024+0000","last_fullsized":"2024-03-28T16:49:37.591024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593505+0000","last_change":"2024-03-28T16:48:58.684369+0000","last_active":"2024-03-28T16:49:37.593505+0000","last_peered":"2024-03-28T16:49:37.593505+0000","last_clean":"2024-03-28T16:49:37.593505+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:49:37.593505+0000","last_undegraded":"2024-03-28T16:49:37.593505+0000","last_fullsized":"2024-03-28T16:49:37.593505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:58.333522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589611+0000","last_change":"2024-03-28T16:49:00.721989+0000","last_active":"2024-03-28T16:49:37.589611+0000","last_peered":"2024-03-28T16:49:37.589611+0000","last_clean":"2024-03-28T16:49:37.589611+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:49:37.589611+0000","last_undegraded":"2024-03-28T16:49:37.589611+0000","last_fullsized":"2024-03-28T16:49:37.589611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:27.568851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'246","reported_seq":324,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854449+0000","last_change":"2024-03-28T16:49:02.730370+0000","last_active":"2024-03-28T16:50:33.854449+0000","last_peered":"2024-03-28T16:50:33.854449+0000","last_clean":"2024-03-28T16:50:33.854449+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:33.854449+0000","last_undegraded":"2024-03-28T16:50:33.854449+0000","last_fullsized":"2024-03-28T16:50:33.854449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585106+0000","last_change":"2024-03-28T16:49:04.739259+0000","last_active":"2024-03-28T16:49:37.585106+0000","last_peered":"2024-03-28T16:49:37.585106+0000","last_clean":"2024-03-28T16:49:37.585106+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:49:37.585106+0000","last_undegraded":"2024-03-28T16:49:37.585106+0000","last_fullsized":"2024-03-28T16:49:37.585106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:50:01.255390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555184+0000","last_change":"2024-03-28T16:48:56.682608+0000","last_active":"2024-03-28T16:49:40.555184+0000","last_peered":"2024-03-28T16:49:40.555184+0000","last_clean":"2024-03-28T16:49:40.555184+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:49:40.555184+0000","last_undegraded":"2024-03-28T16:49:40.555184+0000","last_fullsized":"2024-03-28T16:49:40.555184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.855204+0000","last_change":"2024-03-28T16:48:54.675316+0000","last_active":"2024-03-28T16:49:39.855204+0000","last_peered":"2024-03-28T16:49:39.855204+0000","last_clean":"2024-03-28T16:49:39.855204+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:49:39.855204+0000","last_undegraded":"2024-03-28T16:49:39.855204+0000","last_fullsized":"2024-03-28T16:49:39.855204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588286+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:49:37.588286+0000","last_peered":"2024-03-28T16:49:37.588286+0000","last_clean":"2024-03-28T16:49:37.588286+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:49:37.588286+0000","last_undegraded":"2024-03-28T16:49:37.588286+0000","last_fullsized":"2024-03-28T16:49:37.588286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:45:04.316499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601449+0000","last_change":"2024-03-28T16:48:58.681176+0000","last_active":"2024-03-28T16:49:37.601449+0000","last_peered":"2024-03-28T16:49:37.601449+0000","last_clean":"2024-03-28T16:49:37.601449+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:49:37.601449+0000","last_undegraded":"2024-03-28T16:49:37.601449+0000","last_fullsized":"2024-03-28T16:49:37.601449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:49.369023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'76","reported_seq":147,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.790186+0000","last_change":"2024-03-28T16:49:00.709967+0000","last_active":"2024-03-28T16:50:28.790186+0000","last_peered":"2024-03-28T16:50:28.790186+0000","last_clean":"2024-03-28T16:50:28.790186+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:28.790186+0000","last_undegraded":"2024-03-28T16:50:28.790186+0000","last_fullsized":"2024-03-28T16:50:28.790186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:08.949965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585774+0000","last_change":"2024-03-28T16:49:02.715279+0000","last_active":"2024-03-28T16:49:37.585774+0000","last_peered":"2024-03-28T16:49:37.585774+0000","last_clean":"2024-03-28T16:49:37.585774+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:49:37.585774+0000","last_undegraded":"2024-03-28T16:49:37.585774+0000","last_fullsized":"2024-03-28T16:49:37.585774+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:02.364003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594229+0000","last_change":"2024-03-28T16:49:04.754190+0000","last_active":"2024-03-28T16:49:37.594229+0000","last_peered":"2024-03-28T16:49:37.594229+0000","last_clean":"2024-03-28T16:49:37.594229+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:49:37.594229+0000","last_undegraded":"2024-03-28T16:49:37.594229+0000","last_fullsized":"2024-03-28T16:49:37.594229+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:26.719819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594252+0000","last_change":"2024-03-28T16:49:04.740956+0000","last_active":"2024-03-28T16:49:37.594252+0000","last_peered":"2024-03-28T16:49:37.594252+0000","last_clean":"2024-03-28T16:49:37.594252+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:49:37.594252+0000","last_undegraded":"2024-03-28T16:49:37.594252+0000","last_fullsized":"2024-03-28T16:49:37.594252+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:55.907917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190179+0000","last_change":"2024-03-28T16:48:56.680906+0000","last_active":"2024-03-28T16:50:23.190179+0000","last_peered":"2024-03-28T16:50:23.190179+0000","last_clean":"2024-03-28T16:50:23.190179+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:50:23.190179+0000","last_undegraded":"2024-03-28T16:50:23.190179+0000","last_fullsized":"2024-03-28T16:50:23.190179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:05.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587082+0000","last_change":"2024-03-28T16:48:54.663088+0000","last_active":"2024-03-28T16:49:37.587082+0000","last_peered":"2024-03-28T16:49:37.587082+0000","last_clean":"2024-03-28T16:49:37.587082+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:49:37.587082+0000","last_undegraded":"2024-03-28T16:49:37.587082+0000","last_fullsized":"2024-03-28T16:49:37.587082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.659538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.606876+0000","last_change":"2024-03-28T16:49:37.606876+0000","last_active":"2024-03-28T16:49:37.606876+0000","last_peered":"2024-03-28T16:49:37.606876+0000","last_clean":"2024-03-28T16:49:37.606876+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:49:37.606876+0000","last_undegraded":"2024-03-28T16:49:37.606876+0000","last_fullsized":"2024-03-28T16:49:37.606876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:25.555426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'31","reported_seq":73,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.157002+0000","last_change":"2024-03-28T16:49:00.725615+0000","last_active":"2024-03-28T16:50:23.157002+0000","last_peered":"2024-03-28T16:50:23.157002+0000","last_clean":"2024-03-28T16:50:23.157002+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:50:23.157002+0000","last_undegraded":"2024-03-28T16:50:23.157002+0000","last_fullsized":"2024-03-28T16:50:23.157002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:56.763507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582514+0000","last_change":"2024-03-28T16:49:02.739259+0000","last_active":"2024-03-28T16:49:37.582514+0000","last_peered":"2024-03-28T16:49:37.582514+0000","last_clean":"2024-03-28T16:49:37.582514+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:49:37.582514+0000","last_undegraded":"2024-03-28T16:49:37.582514+0000","last_fullsized":"2024-03-28T16:49:37.582514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:33.761245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'1","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586987+0000","last_change":"2024-03-28T16:49:04.723410+0000","last_active":"2024-03-28T16:49:37.586987+0000","last_peered":"2024-03-28T16:49:37.586987+0000","last_clean":"2024-03-28T16:49:37.586987+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:49:37.586987+0000","last_undegraded":"2024-03-28T16:49:37.586987+0000","last_fullsized":"2024-03-28T16:49:37.586987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:56:19.278765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.603811+0000","last_change":"2024-03-28T16:49:37.603811+0000","last_active":"2024-03-28T16:49:37.603811+0000","last_peered":"2024-03-28T16:49:37.603811+0000","last_clean":"2024-03-28T16:49:37.603811+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:49:37.603811+0000","last_undegraded":"2024-03-28T16:49:37.603811+0000","last_fullsized":"2024-03-28T16:49:37.603811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:44.503106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.833936+0000","last_change":"2024-03-28T16:48:54.657586+0000","last_active":"2024-03-28T16:50:28.833936+0000","last_peered":"2024-03-28T16:50:28.833936+0000","last_clean":"2024-03-28T16:50:28.833936+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:50:28.833936+0000","last_undegraded":"2024-03-28T16:50:28.833936+0000","last_fullsized":"2024-03-28T16:50:28.833936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594338+0000","last_change":"2024-03-28T16:48:58.696407+0000","last_active":"2024-03-28T16:49:37.594338+0000","last_peered":"2024-03-28T16:49:37.594338+0000","last_clean":"2024-03-28T16:49:37.594338+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:49:37.594338+0000","last_undegraded":"2024-03-28T16:49:37.594338+0000","last_fullsized":"2024-03-28T16:49:37.594338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:45.492142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:15.707942+0000","last_change":"2024-03-28T16:49:00.701020+0000","last_active":"2024-03-28T16:50:15.707942+0000","last_peered":"2024-03-28T16:50:15.707942+0000","last_clean":"2024-03-28T16:50:15.707942+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:50:15.707942+0000","last_undegraded":"2024-03-28T16:50:15.707942+0000","last_fullsized":"2024-03-28T16:50:15.707942+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:42.652898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589597+0000","last_change":"2024-03-28T16:49:02.726748+0000","last_active":"2024-03-28T16:49:37.589597+0000","last_peered":"2024-03-28T16:49:37.589597+0000","last_clean":"2024-03-28T16:49:37.589597+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:49:37.589597+0000","last_undegraded":"2024-03-28T16:49:37.589597+0000","last_fullsized":"2024-03-28T16:49:37.589597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:39.742691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586140+0000","last_change":"2024-03-28T16:49:04.744040+0000","last_active":"2024-03-28T16:49:37.586140+0000","last_peered":"2024-03-28T16:49:37.586140+0000","last_clean":"2024-03-28T16:49:37.586140+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:49:37.586140+0000","last_undegraded":"2024-03-28T16:49:37.586140+0000","last_fullsized":"2024-03-28T16:49:37.586140+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:51:53.769493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.730425+0000","last_change":"2024-03-28T16:48:56.676841+0000","last_active":"2024-03-28T16:50:22.730425+0000","last_peered":"2024-03-28T16:50:22.730425+0000","last_clean":"2024-03-28T16:50:22.730425+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:50:22.730425+0000","last_undegraded":"2024-03-28T16:50:22.730425+0000","last_fullsized":"2024-03-28T16:50:22.730425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.183347+0000","last_change":"2024-03-28T16:48:54.659093+0000","last_active":"2024-03-28T16:50:23.183347+0000","last_peered":"2024-03-28T16:50:23.183347+0000","last_clean":"2024-03-28T16:50:23.183347+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:50:23.183347+0000","last_undegraded":"2024-03-28T16:50:23.183347+0000","last_fullsized":"2024-03-28T16:50:23.183347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'2","reported_seq":196,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.365615+0000","last_change":"2024-03-28T16:48:58.690200+0000","last_active":"2024-03-28T16:50:30.365615+0000","last_peered":"2024-03-28T16:50:30.365615+0000","last_clean":"2024-03-28T16:50:30.365615+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:50:30.365615+0000","last_undegraded":"2024-03-28T16:50:30.365615+0000","last_fullsized":"2024-03-28T16:50:30.365615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:59.661181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":253,"num_read_kb":252,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'35","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.577197+0000","last_change":"2024-03-28T16:49:00.718425+0000","last_active":"2024-03-28T16:50:28.577197+0000","last_peered":"2024-03-28T16:50:28.577197+0000","last_clean":"2024-03-28T16:50:28.577197+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:28.577197+0000","last_undegraded":"2024-03-28T16:50:28.577197+0000","last_fullsized":"2024-03-28T16:50:28.577197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:37.725635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":50,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585222+0000","last_change":"2024-03-28T16:49:02.715466+0000","last_active":"2024-03-28T16:49:37.585222+0000","last_peered":"2024-03-28T16:49:37.585222+0000","last_clean":"2024-03-28T16:49:37.585222+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:49:37.585222+0000","last_undegraded":"2024-03-28T16:49:37.585222+0000","last_fullsized":"2024-03-28T16:49:37.585222+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581973+0000","last_change":"2024-03-28T16:49:04.752570+0000","last_active":"2024-03-28T16:49:37.581973+0000","last_peered":"2024-03-28T16:49:37.581973+0000","last_clean":"2024-03-28T16:49:37.581973+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:49:37.581973+0000","last_undegraded":"2024-03-28T16:49:37.581973+0000","last_fullsized":"2024-03-28T16:49:37.581973+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:04:11.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189947+0000","last_change":"2024-03-28T16:48:56.683249+0000","last_active":"2024-03-28T16:50:23.189947+0000","last_peered":"2024-03-28T16:50:23.189947+0000","last_clean":"2024-03-28T16:50:23.189947+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:50:23.189947+0000","last_undegraded":"2024-03-28T16:50:23.189947+0000","last_fullsized":"2024-03-28T16:50:23.189947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:23.009747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602071+0000","last_change":"2024-03-28T16:48:54.674928+0000","last_active":"2024-03-28T16:49:37.602071+0000","last_peered":"2024-03-28T16:49:37.602071+0000","last_clean":"2024-03-28T16:49:37.602071+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:49:37.602071+0000","last_undegraded":"2024-03-28T16:49:37.602071+0000","last_fullsized":"2024-03-28T16:49:37.602071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593497+0000","last_change":"2024-03-28T16:48:58.680113+0000","last_active":"2024-03-28T16:49:37.593497+0000","last_peered":"2024-03-28T16:49:37.593497+0000","last_clean":"2024-03-28T16:49:37.593497+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:49:37.593497+0000","last_undegraded":"2024-03-28T16:49:37.593497+0000","last_fullsized":"2024-03-28T16:49:37.593497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:59.079328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.137620+0000","last_change":"2024-03-28T16:49:00.696647+0000","last_active":"2024-03-28T16:50:23.137620+0000","last_peered":"2024-03-28T16:50:23.137620+0000","last_clean":"2024-03-28T16:50:23.137620+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:50:23.137620+0000","last_undegraded":"2024-03-28T16:50:23.137620+0000","last_fullsized":"2024-03-28T16:50:23.137620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:15.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594972+0000","last_change":"2024-03-28T16:49:02.726334+0000","last_active":"2024-03-28T16:49:37.594972+0000","last_peered":"2024-03-28T16:49:37.594972+0000","last_clean":"2024-03-28T16:49:37.594972+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:49:37.594972+0000","last_undegraded":"2024-03-28T16:49:37.594972+0000","last_fullsized":"2024-03-28T16:49:37.594972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:58.829170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600011+0000","last_change":"2024-03-28T16:48:56.675359+0000","last_active":"2024-03-28T16:49:37.600011+0000","last_peered":"2024-03-28T16:49:37.600011+0000","last_clean":"2024-03-28T16:49:37.600011+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:49:37.600011+0000","last_undegraded":"2024-03-28T16:49:37.600011+0000","last_fullsized":"2024-03-28T16:49:37.600011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:35.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601915+0000","last_change":"2024-03-28T16:49:37.600567+0000","last_active":"2024-03-28T16:49:37.601915+0000","last_peered":"2024-03-28T16:49:37.601915+0000","last_clean":"2024-03-28T16:49:37.601915+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:49:37.601915+0000","last_undegraded":"2024-03-28T16:49:37.601915+0000","last_fullsized":"2024-03-28T16:49:37.601915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:40:41.090124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591166+0000","last_change":"2024-03-28T16:48:58.687213+0000","last_active":"2024-03-28T16:49:37.591166+0000","last_peered":"2024-03-28T16:49:37.591166+0000","last_clean":"2024-03-28T16:49:37.591166+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:49:37.591166+0000","last_undegraded":"2024-03-28T16:49:37.591166+0000","last_fullsized":"2024-03-28T16:49:37.591166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:13.652414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'51","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.785801+0000","last_change":"2024-03-28T16:49:00.716007+0000","last_active":"2024-03-28T16:50:28.785801+0000","last_peered":"2024-03-28T16:50:28.785801+0000","last_clean":"2024-03-28T16:50:28.785801+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:28.785801+0000","last_undegraded":"2024-03-28T16:50:28.785801+0000","last_fullsized":"2024-03-28T16:50:28.785801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:56.812114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595955+0000","last_change":"2024-03-28T16:49:02.730156+0000","last_active":"2024-03-28T16:49:37.595955+0000","last_peered":"2024-03-28T16:49:37.595955+0000","last_clean":"2024-03-28T16:49:37.595955+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:49:37.595955+0000","last_undegraded":"2024-03-28T16:49:37.595955+0000","last_fullsized":"2024-03-28T16:49:37.595955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:15.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'11","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.213440+0000","last_change":"2024-03-28T16:49:04.752525+0000","last_active":"2024-03-28T16:50:23.213440+0000","last_peered":"2024-03-28T16:50:23.213440+0000","last_clean":"2024-03-28T16:50:23.213440+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:50:23.213440+0000","last_undegraded":"2024-03-28T16:50:23.213440+0000","last_fullsized":"2024-03-28T16:50:23.213440+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:18.858161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588098+0000","last_change":"2024-03-28T16:48:56.697482+0000","last_active":"2024-03-28T16:49:37.588098+0000","last_peered":"2024-03-28T16:49:37.588098+0000","last_clean":"2024-03-28T16:49:37.588098+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:49:37.588098+0000","last_undegraded":"2024-03-28T16:49:37.588098+0000","last_fullsized":"2024-03-28T16:49:37.588098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:02.641119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584038+0000","last_change":"2024-03-28T16:48:54.668040+0000","last_active":"2024-03-28T16:49:37.584038+0000","last_peered":"2024-03-28T16:49:37.584038+0000","last_clean":"2024-03-28T16:49:37.584038+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:49:37.584038+0000","last_undegraded":"2024-03-28T16:49:37.584038+0000","last_fullsized":"2024-03-28T16:49:37.584038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596417+0000","last_change":"2024-03-28T16:48:58.686377+0000","last_active":"2024-03-28T16:49:37.596417+0000","last_peered":"2024-03-28T16:49:37.596417+0000","last_clean":"2024-03-28T16:49:37.596417+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:49:37.596417+0000","last_undegraded":"2024-03-28T16:49:37.596417+0000","last_fullsized":"2024-03-28T16:49:37.596417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:01.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.138592+0000","last_change":"2024-03-28T16:49:00.716668+0000","last_active":"2024-03-28T16:50:23.138592+0000","last_peered":"2024-03-28T16:50:23.138592+0000","last_clean":"2024-03-28T16:50:23.138592+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:23.138592+0000","last_undegraded":"2024-03-28T16:50:23.138592+0000","last_fullsized":"2024-03-28T16:50:23.138592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:48.604008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587828+0000","last_change":"2024-03-28T16:49:02.731621+0000","last_active":"2024-03-28T16:49:37.587828+0000","last_peered":"2024-03-28T16:49:37.587828+0000","last_clean":"2024-03-28T16:49:37.587828+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:49:37.587828+0000","last_undegraded":"2024-03-28T16:49:37.587828+0000","last_fullsized":"2024-03-28T16:49:37.587828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.557447+0000","last_change":"2024-03-28T16:49:04.754981+0000","last_active":"2024-03-28T16:49:40.557447+0000","last_peered":"2024-03-28T16:49:40.557447+0000","last_clean":"2024-03-28T16:49:40.557447+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:49:40.557447+0000","last_undegraded":"2024-03-28T16:49:40.557447+0000","last_fullsized":"2024-03-28T16:49:40.557447+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:33:25.577689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'3","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189773+0000","last_change":"2024-03-28T16:48:56.694984+0000","last_active":"2024-03-28T16:50:23.189773+0000","last_peered":"2024-03-28T16:50:23.189773+0000","last_clean":"2024-03-28T16:50:23.189773+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:50:23.189773+0000","last_undegraded":"2024-03-28T16:50:23.189773+0000","last_fullsized":"2024-03-28T16:50:23.189773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:09.365959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585942+0000","last_change":"2024-03-28T16:48:54.654527+0000","last_active":"2024-03-28T16:49:37.585942+0000","last_peered":"2024-03-28T16:49:37.585942+0000","last_clean":"2024-03-28T16:49:37.585942+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:49:37.585942+0000","last_undegraded":"2024-03-28T16:49:37.585942+0000","last_fullsized":"2024-03-28T16:49:37.585942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610755+0000","last_change":"2024-03-28T16:49:37.610755+0000","last_active":"2024-03-28T16:49:37.610755+0000","last_peered":"2024-03-28T16:49:37.610755+0000","last_clean":"2024-03-28T16:49:37.610755+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:49:37.610755+0000","last_undegraded":"2024-03-28T16:49:37.610755+0000","last_fullsized":"2024-03-28T16:49:37.610755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:24.156961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'32","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.163251+0000","last_change":"2024-03-28T16:49:00.706941+0000","last_active":"2024-03-28T16:50:23.163251+0000","last_peered":"2024-03-28T16:50:23.163251+0000","last_clean":"2024-03-28T16:50:23.163251+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:23.163251+0000","last_undegraded":"2024-03-28T16:50:23.163251+0000","last_fullsized":"2024-03-28T16:50:23.163251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:05.753628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":46,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587449+0000","last_change":"2024-03-28T16:49:02.734090+0000","last_active":"2024-03-28T16:49:37.587449+0000","last_peered":"2024-03-28T16:49:37.587449+0000","last_clean":"2024-03-28T16:49:37.587449+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:49:37.587449+0000","last_undegraded":"2024-03-28T16:49:37.587449+0000","last_fullsized":"2024-03-28T16:49:37.587449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:39.107257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.835263+0000","last_change":"2024-03-28T16:49:04.748086+0000","last_active":"2024-03-28T16:49:39.835263+0000","last_peered":"2024-03-28T16:49:39.835263+0000","last_clean":"2024-03-28T16:49:39.835263+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:49:39.835263+0000","last_undegraded":"2024-03-28T16:49:39.835263+0000","last_fullsized":"2024-03-28T16:49:39.835263+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:44.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588233+0000","last_change":"2024-03-28T16:48:56.682850+0000","last_active":"2024-03-28T16:49:37.588233+0000","last_peered":"2024-03-28T16:49:37.588233+0000","last_clean":"2024-03-28T16:49:37.588233+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:49:37.588233+0000","last_undegraded":"2024-03-28T16:49:37.588233+0000","last_fullsized":"2024-03-28T16:49:37.588233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592598+0000","last_change":"2024-03-28T16:48:54.674069+0000","last_active":"2024-03-28T16:49:37.592598+0000","last_peered":"2024-03-28T16:49:37.592598+0000","last_clean":"2024-03-28T16:49:37.592598+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:49:37.592598+0000","last_undegraded":"2024-03-28T16:49:37.592598+0000","last_fullsized":"2024-03-28T16:49:37.592598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.406243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587070+0000","last_change":"2024-03-28T16:48:58.681774+0000","last_active":"2024-03-28T16:49:37.587070+0000","last_peered":"2024-03-28T16:49:37.587070+0000","last_clean":"2024-03-28T16:49:37.587070+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:49:37.587070+0000","last_undegraded":"2024-03-28T16:49:37.587070+0000","last_fullsized":"2024-03-28T16:49:37.587070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:24.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589169+0000","last_change":"2024-03-28T16:49:00.720072+0000","last_active":"2024-03-28T16:49:37.589169+0000","last_peered":"2024-03-28T16:49:37.589169+0000","last_clean":"2024-03-28T16:49:37.589169+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:49:37.589169+0000","last_undegraded":"2024-03-28T16:49:37.589169+0000","last_fullsized":"2024-03-28T16:49:37.589169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:22:49.287785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'246","reported_seq":297,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854122+0000","last_change":"2024-03-28T16:49:02.734898+0000","last_active":"2024-03-28T16:50:33.854122+0000","last_peered":"2024-03-28T16:50:33.854122+0000","last_clean":"2024-03-28T16:50:33.854122+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:33.854122+0000","last_undegraded":"2024-03-28T16:50:33.854122+0000","last_fullsized":"2024-03-28T16:50:33.854122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:05.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.579919+0000","last_change":"2024-03-28T16:49:04.748224+0000","last_active":"2024-03-28T16:49:37.579919+0000","last_peered":"2024-03-28T16:49:37.579919+0000","last_clean":"2024-03-28T16:49:37.579919+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:49:37.579919+0000","last_undegraded":"2024-03-28T16:49:37.579919+0000","last_fullsized":"2024-03-28T16:49:37.579919+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:26.578458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'5","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370813+0000","last_change":"2024-03-28T16:48:56.685308+0000","last_active":"2024-03-28T16:50:22.370813+0000","last_peered":"2024-03-28T16:50:22.370813+0000","last_clean":"2024-03-28T16:50:22.370813+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:50:22.370813+0000","last_undegraded":"2024-03-28T16:50:22.370813+0000","last_fullsized":"2024-03-28T16:50:22.370813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:39.524773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586373+0000","last_change":"2024-03-28T16:48:54.667648+0000","last_active":"2024-03-28T16:49:37.586373+0000","last_peered":"2024-03-28T16:49:37.586373+0000","last_clean":"2024-03-28T16:49:37.586373+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:49:37.586373+0000","last_undegraded":"2024-03-28T16:49:37.586373+0000","last_fullsized":"2024-03-28T16:49:37.586373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598430+0000","last_change":"2024-03-28T16:48:58.684892+0000","last_active":"2024-03-28T16:49:37.598430+0000","last_peered":"2024-03-28T16:49:37.598430+0000","last_clean":"2024-03-28T16:49:37.598430+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:49:37.598430+0000","last_undegraded":"2024-03-28T16:49:37.598430+0000","last_fullsized":"2024-03-28T16:49:37.598430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:34.392979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.126243+0000","last_change":"2024-03-28T16:49:00.722124+0000","last_active":"2024-03-28T16:50:23.126243+0000","last_peered":"2024-03-28T16:50:23.126243+0000","last_clean":"2024-03-28T16:50:23.126243+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:23.126243+0000","last_undegraded":"2024-03-28T16:50:23.126243+0000","last_fullsized":"2024-03-28T16:50:23.126243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:39.985393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584276+0000","last_change":"2024-03-28T16:49:02.715093+0000","last_active":"2024-03-28T16:49:37.584276+0000","last_peered":"2024-03-28T16:49:37.584276+0000","last_clean":"2024-03-28T16:49:37.584276+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:49:37.584276+0000","last_undegraded":"2024-03-28T16:49:37.584276+0000","last_fullsized":"2024-03-28T16:49:37.584276+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:26.448039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601171+0000","last_change":"2024-03-28T16:49:04.745332+0000","last_active":"2024-03-28T16:49:37.601171+0000","last_peered":"2024-03-28T16:49:37.601171+0000","last_clean":"2024-03-28T16:49:37.601171+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:49:37.601171+0000","last_undegraded":"2024-03-28T16:49:37.601171+0000","last_fullsized":"2024-03-28T16:49:37.601171+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:11.521117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369950+0000","last_change":"2024-03-28T16:48:56.690217+0000","last_active":"2024-03-28T16:50:22.369950+0000","last_peered":"2024-03-28T16:50:22.369950+0000","last_clean":"2024-03-28T16:50:22.369950+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:50:22.369950+0000","last_undegraded":"2024-03-28T16:50:22.369950+0000","last_fullsized":"2024-03-28T16:50:22.369950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588654+0000","last_change":"2024-03-28T16:48:54.669871+0000","last_active":"2024-03-28T16:49:37.588654+0000","last_peered":"2024-03-28T16:49:37.588654+0000","last_clean":"2024-03-28T16:49:37.588654+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:49:37.588654+0000","last_undegraded":"2024-03-28T16:49:37.588654+0000","last_fullsized":"2024-03-28T16:49:37.588654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:58.495034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594708+0000","last_change":"2024-03-28T16:48:58.692783+0000","last_active":"2024-03-28T16:49:37.594708+0000","last_peered":"2024-03-28T16:49:37.594708+0000","last_clean":"2024-03-28T16:49:37.594708+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:49:37.594708+0000","last_undegraded":"2024-03-28T16:49:37.594708+0000","last_fullsized":"2024-03-28T16:49:37.594708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:20.188622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.128390+0000","last_change":"2024-03-28T16:49:00.706559+0000","last_active":"2024-03-28T16:50:23.128390+0000","last_peered":"2024-03-28T16:50:23.128390+0000","last_clean":"2024-03-28T16:50:23.128390+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:23.128390+0000","last_undegraded":"2024-03-28T16:50:23.128390+0000","last_fullsized":"2024-03-28T16:50:23.128390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:54.800679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581059+0000","last_change":"2024-03-28T16:49:02.739372+0000","last_active":"2024-03-28T16:49:37.581059+0000","last_peered":"2024-03-28T16:49:37.581059+0000","last_clean":"2024-03-28T16:49:37.581059+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:49:37.581059+0000","last_undegraded":"2024-03-28T16:49:37.581059+0000","last_fullsized":"2024-03-28T16:49:37.581059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:22:56.097286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582527+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:49:37.582527+0000","last_peered":"2024-03-28T16:49:37.582527+0000","last_clean":"2024-03-28T16:49:37.582527+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:49:37.582527+0000","last_undegraded":"2024-03-28T16:49:37.582527+0000","last_fullsized":"2024-03-28T16:49:37.582527+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:45:27.545220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593066+0000","last_change":"2024-03-28T16:48:56.666104+0000","last_active":"2024-03-28T16:49:37.593066+0000","last_peered":"2024-03-28T16:49:37.593066+0000","last_clean":"2024-03-28T16:49:37.593066+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:49:37.593066+0000","last_undegraded":"2024-03-28T16:49:37.593066+0000","last_fullsized":"2024-03-28T16:49:37.593066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:35:42.047564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588335+0000","last_change":"2024-03-28T16:48:54.665975+0000","last_active":"2024-03-28T16:49:37.588335+0000","last_peered":"2024-03-28T16:49:37.588335+0000","last_clean":"2024-03-28T16:49:37.588335+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:49:37.588335+0000","last_undegraded":"2024-03-28T16:49:37.588335+0000","last_fullsized":"2024-03-28T16:49:37.588335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585938+0000","last_change":"2024-03-28T16:48:58.688417+0000","last_active":"2024-03-28T16:49:37.585938+0000","last_peered":"2024-03-28T16:49:37.585938+0000","last_clean":"2024-03-28T16:49:37.585938+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:49:37.585938+0000","last_undegraded":"2024-03-28T16:49:37.585938+0000","last_fullsized":"2024-03-28T16:49:37.585938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:57.154239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'24","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.159574+0000","last_change":"2024-03-28T16:49:00.717188+0000","last_active":"2024-03-28T16:50:23.159574+0000","last_peered":"2024-03-28T16:50:23.159574+0000","last_clean":"2024-03-28T16:50:23.159574+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:23.159574+0000","last_undegraded":"2024-03-28T16:50:23.159574+0000","last_fullsized":"2024-03-28T16:50:23.159574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:40:19.790152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581817+0000","last_change":"2024-03-28T16:49:02.739568+0000","last_active":"2024-03-28T16:49:37.581817+0000","last_peered":"2024-03-28T16:49:37.581817+0000","last_clean":"2024-03-28T16:49:37.581817+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:49:37.581817+0000","last_undegraded":"2024-03-28T16:49:37.581817+0000","last_fullsized":"2024-03-28T16:49:37.581817+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:32.279507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587161+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:49:37.587161+0000","last_peered":"2024-03-28T16:49:37.587161+0000","last_clean":"2024-03-28T16:49:37.587161+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:49:37.587161+0000","last_undegraded":"2024-03-28T16:49:37.587161+0000","last_fullsized":"2024-03-28T16:49:37.587161+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:59.156256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'11","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854243+0000","last_change":"2024-03-28T16:48:56.678271+0000","last_active":"2024-03-28T16:50:28.854243+0000","last_peered":"2024-03-28T16:50:28.854243+0000","last_clean":"2024-03-28T16:50:28.854243+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:50:28.854243+0000","last_undegraded":"2024-03-28T16:50:28.854243+0000","last_fullsized":"2024-03-28T16:50:28.854243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:31.748236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602333+0000","last_change":"2024-03-28T16:48:54.655009+0000","last_active":"2024-03-28T16:49:37.602333+0000","last_peered":"2024-03-28T16:49:37.602333+0000","last_clean":"2024-03-28T16:49:37.602333+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:49:37.602333+0000","last_undegraded":"2024-03-28T16:49:37.602333+0000","last_fullsized":"2024-03-28T16:49:37.602333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'2","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595281+0000","last_change":"2024-03-28T16:48:58.697773+0000","last_active":"2024-03-28T16:49:37.595281+0000","last_peered":"2024-03-28T16:49:37.595281+0000","last_clean":"2024-03-28T16:49:37.595281+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:49:37.595281+0000","last_undegraded":"2024-03-28T16:49:37.595281+0000","last_fullsized":"2024-03-28T16:49:37.595281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:25.024742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'29","reported_seq":70,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.816906+0000","last_change":"2024-03-28T16:49:00.720335+0000","last_active":"2024-03-28T16:50:28.816906+0000","last_peered":"2024-03-28T16:50:28.816906+0000","last_clean":"2024-03-28T16:50:28.816906+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:50:28.816906+0000","last_undegraded":"2024-03-28T16:50:28.816906+0000","last_fullsized":"2024-03-28T16:50:28.816906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:53.514285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":26,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586838+0000","last_change":"2024-03-28T16:49:02.714786+0000","last_active":"2024-03-28T16:49:37.586838+0000","last_peered":"2024-03-28T16:49:37.586838+0000","last_clean":"2024-03-28T16:49:37.586838+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:49:37.586838+0000","last_undegraded":"2024-03-28T16:49:37.586838+0000","last_fullsized":"2024-03-28T16:49:37.586838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582310+0000","last_change":"2024-03-28T16:49:04.753745+0000","last_active":"2024-03-28T16:49:37.582310+0000","last_peered":"2024-03-28T16:49:37.582310+0000","last_clean":"2024-03-28T16:49:37.582310+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:49:37.582310+0000","last_undegraded":"2024-03-28T16:49:37.582310+0000","last_fullsized":"2024-03-28T16:49:37.582310+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:59.796190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.619610+0000","last_change":"2024-03-28T16:49:37.618664+0000","last_active":"2024-03-28T16:49:37.619610+0000","last_peered":"2024-03-28T16:49:37.619610+0000","last_clean":"2024-03-28T16:49:37.619610+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:49:37.619610+0000","last_undegraded":"2024-03-28T16:49:37.619610+0000","last_fullsized":"2024-03-28T16:49:37.619610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:36.076211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588848+0000","last_change":"2024-03-28T16:48:54.658051+0000","last_active":"2024-03-28T16:49:37.588848+0000","last_peered":"2024-03-28T16:49:37.588848+0000","last_clean":"2024-03-28T16:49:37.588848+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:49:37.588848+0000","last_undegraded":"2024-03-28T16:49:37.588848+0000","last_fullsized":"2024-03-28T16:49:37.588848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190542+0000","last_change":"2024-03-28T16:48:56.682367+0000","last_active":"2024-03-28T16:50:23.190542+0000","last_peered":"2024-03-28T16:50:23.190542+0000","last_clean":"2024-03-28T16:50:23.190542+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:50:23.190542+0000","last_undegraded":"2024-03-28T16:50:23.190542+0000","last_fullsized":"2024-03-28T16:50:23.190542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584176+0000","last_change":"2024-03-28T16:48:54.655298+0000","last_active":"2024-03-28T16:49:37.584176+0000","last_peered":"2024-03-28T16:49:37.584176+0000","last_clean":"2024-03-28T16:49:37.584176+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:49:37.584176+0000","last_undegraded":"2024-03-28T16:49:37.584176+0000","last_fullsized":"2024-03-28T16:49:37.584176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'3","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370059+0000","last_change":"2024-03-28T16:48:56.671798+0000","last_active":"2024-03-28T16:50:22.370059+0000","last_peered":"2024-03-28T16:50:22.370059+0000","last_clean":"2024-03-28T16:50:22.370059+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:50:22.370059+0000","last_undegraded":"2024-03-28T16:50:22.370059+0000","last_fullsized":"2024-03-28T16:50:22.370059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:59.926897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589042+0000","last_change":"2024-03-28T16:48:54.657491+0000","last_active":"2024-03-28T16:49:37.589042+0000","last_peered":"2024-03-28T16:49:37.589042+0000","last_clean":"2024-03-28T16:49:37.589042+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:49:37.589042+0000","last_undegraded":"2024-03-28T16:49:37.589042+0000","last_fullsized":"2024-03-28T16:49:37.589042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:40:58.160099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586819+0000","last_change":"2024-03-28T16:48:56.683483+0000","last_active":"2024-03-28T16:49:37.586819+0000","last_peered":"2024-03-28T16:49:37.586819+0000","last_clean":"2024-03-28T16:49:37.586819+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:49:37.586819+0000","last_undegraded":"2024-03-28T16:49:37.586819+0000","last_fullsized":"2024-03-28T16:49:37.586819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588791+0000","last_change":"2024-03-28T16:48:54.665811+0000","last_active":"2024-03-28T16:49:37.588791+0000","last_peered":"2024-03-28T16:49:37.588791+0000","last_clean":"2024-03-28T16:49:37.588791+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:49:37.588791+0000","last_undegraded":"2024-03-28T16:49:37.588791+0000","last_fullsized":"2024-03-28T16:49:37.588791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:38.364800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591495+0000","last_change":"2024-03-28T16:48:56.677917+0000","last_active":"2024-03-28T16:49:37.591495+0000","last_peered":"2024-03-28T16:49:37.591495+0000","last_clean":"2024-03-28T16:49:37.591495+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:49:37.591495+0000","last_undegraded":"2024-03-28T16:49:37.591495+0000","last_fullsized":"2024-03-28T16:49:37.591495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587233+0000","last_change":"2024-03-28T16:48:54.655391+0000","last_active":"2024-03-28T16:49:37.587233+0000","last_peered":"2024-03-28T16:49:37.587233+0000","last_clean":"2024-03-28T16:49:37.587233+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:49:37.587233+0000","last_undegraded":"2024-03-28T16:49:37.587233+0000","last_fullsized":"2024-03-28T16:49:37.587233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840006+0000","last_change":"2024-03-28T16:48:56.678419+0000","last_active":"2024-03-28T16:50:28.840006+0000","last_peered":"2024-03-28T16:50:28.840006+0000","last_clean":"2024-03-28T16:50:28.840006+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:50:28.840006+0000","last_undegraded":"2024-03-28T16:50:28.840006+0000","last_fullsized":"2024-03-28T16:50:28.840006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:03.478538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595229+0000","last_change":"2024-03-28T16:48:54.669640+0000","last_active":"2024-03-28T16:49:37.595229+0000","last_peered":"2024-03-28T16:49:37.595229+0000","last_clean":"2024-03-28T16:49:37.595229+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:49:37.595229+0000","last_undegraded":"2024-03-28T16:49:37.595229+0000","last_fullsized":"2024-03-28T16:49:37.595229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527969+0000","last_change":"2024-03-28T16:48:56.689880+0000","last_active":"2024-03-28T16:49:39.527969+0000","last_peered":"2024-03-28T16:49:39.527969+0000","last_clean":"2024-03-28T16:49:39.527969+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:49:39.527969+0000","last_undegraded":"2024-03-28T16:49:39.527969+0000","last_fullsized":"2024-03-28T16:49:39.527969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598088+0000","last_change":"2024-03-28T16:48:54.661594+0000","last_active":"2024-03-28T16:49:37.598088+0000","last_peered":"2024-03-28T16:49:37.598088+0000","last_clean":"2024-03-28T16:49:37.598088+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:49:37.598088+0000","last_undegraded":"2024-03-28T16:49:37.598088+0000","last_fullsized":"2024-03-28T16:49:37.598088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'5","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839320+0000","last_change":"2024-03-28T16:48:56.679523+0000","last_active":"2024-03-28T16:50:28.839320+0000","last_peered":"2024-03-28T16:50:28.839320+0000","last_clean":"2024-03-28T16:50:28.839320+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:50:28.839320+0000","last_undegraded":"2024-03-28T16:50:28.839320+0000","last_fullsized":"2024-03-28T16:50:28.839320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588478+0000","last_change":"2024-03-28T16:48:54.659870+0000","last_active":"2024-03-28T16:49:37.588478+0000","last_peered":"2024-03-28T16:49:37.588478+0000","last_clean":"2024-03-28T16:49:37.588478+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:49:37.588478+0000","last_undegraded":"2024-03-28T16:49:37.588478+0000","last_fullsized":"2024-03-28T16:49:37.588478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.526775+0000","last_change":"2024-03-28T16:48:56.687788+0000","last_active":"2024-03-28T16:49:39.526775+0000","last_peered":"2024-03-28T16:49:39.526775+0000","last_clean":"2024-03-28T16:49:39.526775+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:49:39.526775+0000","last_undegraded":"2024-03-28T16:49:39.526775+0000","last_fullsized":"2024-03-28T16:49:39.526775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:05.073812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581474+0000","last_change":"2024-03-28T16:48:54.657662+0000","last_active":"2024-03-28T16:49:37.581474+0000","last_peered":"2024-03-28T16:49:37.581474+0000","last_clean":"2024-03-28T16:49:37.581474+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:49:37.581474+0000","last_undegraded":"2024-03-28T16:49:37.581474+0000","last_fullsized":"2024-03-28T16:49:37.581474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:59.369957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'1","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369831+0000","last_change":"2024-03-28T16:48:56.682872+0000","last_active":"2024-03-28T16:50:22.369831+0000","last_peered":"2024-03-28T16:50:22.369831+0000","last_clean":"2024-03-28T16:50:22.369831+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:50:22.369831+0000","last_undegraded":"2024-03-28T16:50:22.369831+0000","last_fullsized":"2024-03-28T16:50:22.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:50.772258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584620+0000","last_change":"2024-03-28T16:48:54.663496+0000","last_active":"2024-03-28T16:49:37.584620+0000","last_peered":"2024-03-28T16:49:37.584620+0000","last_clean":"2024-03-28T16:49:37.584620+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:49:37.584620+0000","last_undegraded":"2024-03-28T16:49:37.584620+0000","last_fullsized":"2024-03-28T16:49:37.584620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587192+0000","last_change":"2024-03-28T16:48:56.678342+0000","last_active":"2024-03-28T16:49:37.587192+0000","last_peered":"2024-03-28T16:49:37.587192+0000","last_clean":"2024-03-28T16:49:37.587192+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:49:37.587192+0000","last_undegraded":"2024-03-28T16:49:37.587192+0000","last_fullsized":"2024-03-28T16:49:37.587192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585680+0000","last_change":"2024-03-28T16:48:54.675150+0000","last_active":"2024-03-28T16:49:37.585680+0000","last_peered":"2024-03-28T16:49:37.585680+0000","last_clean":"2024-03-28T16:49:37.585680+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:49:37.585680+0000","last_undegraded":"2024-03-28T16:49:37.585680+0000","last_fullsized":"2024-03-28T16:49:37.585680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190164+0000","last_change":"2024-03-28T16:48:56.671157+0000","last_active":"2024-03-28T16:50:23.190164+0000","last_peered":"2024-03-28T16:50:23.190164+0000","last_clean":"2024-03-28T16:50:23.190164+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:50:23.190164+0000","last_undegraded":"2024-03-28T16:50:23.190164+0000","last_fullsized":"2024-03-28T16:50:23.190164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:59.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585793+0000","last_change":"2024-03-28T16:48:54.674973+0000","last_active":"2024-03-28T16:49:37.585793+0000","last_peered":"2024-03-28T16:49:37.585793+0000","last_clean":"2024-03-28T16:49:37.585793+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:49:37.585793+0000","last_undegraded":"2024-03-28T16:49:37.585793+0000","last_fullsized":"2024-03-28T16:49:37.585793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840005+0000","last_change":"2024-03-28T16:48:56.683829+0000","last_active":"2024-03-28T16:50:28.840005+0000","last_peered":"2024-03-28T16:50:28.840005+0000","last_clean":"2024-03-28T16:50:28.840005+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:50:28.840005+0000","last_undegraded":"2024-03-28T16:50:28.840005+0000","last_fullsized":"2024-03-28T16:50:28.840005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587774+0000","last_change":"2024-03-28T16:48:54.676815+0000","last_active":"2024-03-28T16:49:37.587774+0000","last_peered":"2024-03-28T16:49:37.587774+0000","last_clean":"2024-03-28T16:49:37.587774+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:49:37.587774+0000","last_undegraded":"2024-03-28T16:49:37.587774+0000","last_fullsized":"2024-03-28T16:49:37.587774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:19.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839390+0000","last_change":"2024-03-28T16:48:56.693060+0000","last_active":"2024-03-28T16:50:28.839390+0000","last_peered":"2024-03-28T16:50:28.839390+0000","last_clean":"2024-03-28T16:50:28.839390+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:50:28.839390+0000","last_undegraded":"2024-03-28T16:50:28.839390+0000","last_fullsized":"2024-03-28T16:50:28.839390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589164+0000","last_change":"2024-03-28T16:48:54.663003+0000","last_active":"2024-03-28T16:49:37.589164+0000","last_peered":"2024-03-28T16:49:37.589164+0000","last_clean":"2024-03-28T16:49:37.589164+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:49:37.589164+0000","last_undegraded":"2024-03-28T16:49:37.589164+0000","last_fullsized":"2024-03-28T16:49:37.589164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:43.737071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.078594+0000","last_change":"2024-03-28T16:48:56.685753+0000","last_active":"2024-03-28T16:49:39.078594+0000","last_peered":"2024-03-28T16:49:39.078594+0000","last_clean":"2024-03-28T16:49:39.078594+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:49:39.078594+0000","last_undegraded":"2024-03-28T16:49:39.078594+0000","last_fullsized":"2024-03-28T16:49:39.078594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:12.442441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598216+0000","last_change":"2024-03-28T16:48:54.661896+0000","last_active":"2024-03-28T16:49:37.598216+0000","last_peered":"2024-03-28T16:49:37.598216+0000","last_clean":"2024-03-28T16:49:37.598216+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:49:37.598216+0000","last_undegraded":"2024-03-28T16:49:37.598216+0000","last_fullsized":"2024-03-28T16:49:37.598216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'9","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877960+0000","last_change":"2024-03-28T16:48:56.680119+0000","last_active":"2024-03-28T16:49:39.877960+0000","last_peered":"2024-03-28T16:49:39.877960+0000","last_clean":"2024-03-28T16:49:39.877960+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:49:39.877960+0000","last_undegraded":"2024-03-28T16:49:39.877960+0000","last_fullsized":"2024-03-28T16:49:39.877960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588667+0000","last_change":"2024-03-28T16:48:54.665812+0000","last_active":"2024-03-28T16:49:37.588667+0000","last_peered":"2024-03-28T16:49:37.588667+0000","last_clean":"2024-03-28T16:49:37.588667+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:49:37.588667+0000","last_undegraded":"2024-03-28T16:49:37.588667+0000","last_fullsized":"2024-03-28T16:49:37.588667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:31.951558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581570+0000","last_change":"2024-03-28T16:48:54.665260+0000","last_active":"2024-03-28T16:49:37.581570+0000","last_peered":"2024-03-28T16:49:37.581570+0000","last_clean":"2024-03-28T16:49:37.581570+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:49:37.581570+0000","last_undegraded":"2024-03-28T16:49:37.581570+0000","last_fullsized":"2024-03-28T16:49:37.581570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:37.463264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591429+0000","last_change":"2024-03-28T16:48:56.690793+0000","last_active":"2024-03-28T16:49:37.591429+0000","last_peered":"2024-03-28T16:49:37.591429+0000","last_clean":"2024-03-28T16:49:37.591429+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:49:37.591429+0000","last_undegraded":"2024-03-28T16:49:37.591429+0000","last_fullsized":"2024-03-28T16:49:37.591429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584402+0000","last_change":"2024-03-28T16:48:54.663735+0000","last_active":"2024-03-28T16:49:37.584402+0000","last_peered":"2024-03-28T16:49:37.584402+0000","last_clean":"2024-03-28T16:49:37.584402+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:49:37.584402+0000","last_undegraded":"2024-03-28T16:49:37.584402+0000","last_fullsized":"2024-03-28T16:49:37.584402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587762+0000","last_change":"2024-03-28T16:48:56.683832+0000","last_active":"2024-03-28T16:49:37.587762+0000","last_peered":"2024-03-28T16:49:37.587762+0000","last_clean":"2024-03-28T16:49:37.587762+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:49:37.587762+0000","last_undegraded":"2024-03-28T16:49:37.587762+0000","last_fullsized":"2024-03-28T16:49:37.587762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:38.808030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1968,"ondisk_log_size":1968,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1344,"num_read_kb":1127,"num_write":517,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1175,"ondisk_log_size":1175,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":854,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39916,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331924,"statfs":{"total":96636764160,"available":96595890176,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.752}]},{"osd":2,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.51900000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.75},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64200000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705691,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78500000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705691,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:50:43.350 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-28T16:50:43.350 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:50:43.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.498+0000 7fd83c8ea640 1 Processor -- start 2024-03-28T16:50:43.503 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.498+0000 7fd83c8ea640 1 -- start start 2024-03-28T16:50:43.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd838128d10 0x7fd8381290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.504 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd838095ee0 con 0x7fd83812bb20 2024-03-28T16:50:43.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd838095b30 con 0x7fd838128d10 2024-03-28T16:50:43.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd838128d10 0x7fd8381290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd838128d10 0x7fd8381290f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34126/0 (socket says 172.21.15.119:34126) 2024-03-28T16:50:43.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd835d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd83812bb20 0x7fd83812bf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43800/0 (socket says 172.21.15.119:43800) 2024-03-28T16:50:43.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83659c640 1 -- 172.21.15.119:0/3549289005 learned_addr learned my addr 172.21.15.119:0/3549289005 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:43.505 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83659c640 1 -- 172.21.15.119:0/3549289005 >> v1:172.21.15.119:6789/0 conn(0x7fd83812bb20 legacy=0x7fd83812bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83659c640 1 -- 172.21.15.119:0/3549289005 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd838096320 con 0x7fd838128d10 2024-03-28T16:50:43.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83559a640 1 -- 172.21.15.119:0/3549289005 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99931738 0 0) 0x7fd838095ee0 con 0x7fd83812bb20 2024-03-28T16:50:43.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83659c640 1 --2- 172.21.15.119:0/3549289005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd8381290f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd828009e50 tx=0x7fd82801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cd3b818aca529 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.506 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83559a640 1 -- 172.21.15.119:0/3549289005 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd828009200 con 0x7fd838128d10 2024-03-28T16:50:43.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83559a640 1 -- 172.21.15.119:0/3549289005 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8280a5660 con 0x7fd838128d10 2024-03-28T16:50:43.507 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83559a640 1 -- 172.21.15.119:0/3549289005 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8280a5920 con 0x7fd838128d10 2024-03-28T16:50:43.510 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 -- 172.21.15.119:0/3549289005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 msgr2=0x7fd8381290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 --2- 172.21.15.119:0/3549289005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd8381290f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd828009e50 tx=0x7fd82801ca10 comp rx=0 tx=0).stop 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 -- 172.21.15.119:0/3549289005 shutdown_connections 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 --2- 172.21.15.119:0/3549289005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd8381290f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 -- 172.21.15.119:0/3549289005 >> 172.21.15.119:0/3549289005 conn(0x7fd83809a2f0 msgr2=0x7fd83809a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 -- 172.21.15.119:0/3549289005 shutdown_connections 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.502+0000 7fd83c8ea640 1 -- 172.21.15.119:0/3549289005 wait complete. 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83c8ea640 1 Processor -- start 2024-03-28T16:50:43.511 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83c8ea640 1 -- start start 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83c8ea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd83806c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83c8ea640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83806ded0 con 0x7fd838128d10 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd83806c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd83806c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34138/0 (socket says 172.21.15.119:34138) 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83659c640 1 -- 172.21.15.119:0/4073941077 learned_addr learned my addr 172.21.15.119:0/4073941077 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83659c640 1 -- 172.21.15.119:0/4073941077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828009b30 con 0x7fd838128d10 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83659c640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd83806c1e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd828019620 tx=0x7fd8280197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd82effd640 1 -- 172.21.15.119:0/4073941077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8280b8780 con 0x7fd838128d10 2024-03-28T16:50:43.512 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd82effd640 1 -- 172.21.15.119:0/4073941077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8280b88e0 con 0x7fd838128d10 2024-03-28T16:50:43.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd838128160 con 0x7fd838128d10 2024-03-28T16:50:43.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd82effd640 1 -- 172.21.15.119:0/4073941077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8280b8b80 con 0x7fd838128d10 2024-03-28T16:50:43.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83806d5f0 con 0x7fd838128d10 2024-03-28T16:50:43.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd82effd640 1 -- 172.21.15.119:0/4073941077 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd8280b8ce0 con 0x7fd838128d10 2024-03-28T16:50:43.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838067e00 con 0x7fd838128d10 2024-03-28T16:50:43.513 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd82effd640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd818035a10 0x7fd818037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd82effd640 1 -- 172.21.15.119:0/4073941077 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd8280e3c60 con 0x7fd838128d10 2024-03-28T16:50:43.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.506+0000 7fd835d9b640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd818035a10 0x7fd818037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.514 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.510+0000 7fd835d9b640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd818035a10 0x7fd818037ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd820007fd0 tx=0x7fd820007920 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.515 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.510+0000 7fd82effd640 1 -- 172.21.15.119:0/4073941077 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8280c8dc0 con 0x7fd838128d10 2024-03-28T16:50:43.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.666+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd83812ea70 con 0x7fd818035a10 2024-03-28T16:50:43.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.710+0000 7fd82effd640 1 -- 172.21.15.119:0/4073941077 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+673101 (secure 0 0 0) 0x7fd83812ea70 con 0x7fd818035a10 2024-03-28T16:50:43.719 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:50:43.725 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:50:43.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd818035a10 msgr2=0x7fd818037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd818035a10 0x7fd818037ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd820007fd0 tx=0x7fd820007920 comp rx=0 tx=0).stop 2024-03-28T16:50:43.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 msgr2=0x7fd83806c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd83806c1e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd828019620 tx=0x7fd8280197b0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 shutdown_connections 2024-03-28T16:50:43.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd818035a10 0x7fd818037ed0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 --2- 172.21.15.119:0/4073941077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd838128d10 0x7fd83806c1e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 >> 172.21.15.119:0/4073941077 conn(0x7fd83809a2f0 msgr2=0x7fd838121940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:43.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 shutdown_connections 2024-03-28T16:50:43.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.714+0000 7fd83c8ea640 1 -- 172.21.15.119:0/4073941077 wait complete. 2024-03-28T16:50:43.736 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-03-28T16:50:42.533657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":3824,"num_read_kb":3471,"num_write":1479,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3745,"ondisk_log_size":3745,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":160768,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":157945,"kb_avail":377326592,"statfs":{"total":386547056640,"available":386382430208,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":161736470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":403,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":211,"num_read_kb":206,"num_write":80,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006262"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600005+0000","last_change":"2024-03-28T16:48:54.656021+0000","last_active":"2024-03-28T16:49:37.600005+0000","last_peered":"2024-03-28T16:49:37.600005+0000","last_clean":"2024-03-28T16:49:37.600005+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:49:37.600005+0000","last_undegraded":"2024-03-28T16:49:37.600005+0000","last_fullsized":"2024-03-28T16:49:37.600005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585246+0000","last_change":"2024-03-28T16:48:56.687475+0000","last_active":"2024-03-28T16:49:37.585246+0000","last_peered":"2024-03-28T16:49:37.585246+0000","last_clean":"2024-03-28T16:49:37.585246+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:49:37.585246+0000","last_undegraded":"2024-03-28T16:49:37.585246+0000","last_fullsized":"2024-03-28T16:49:37.585246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587869+0000","last_change":"2024-03-28T16:48:54.666372+0000","last_active":"2024-03-28T16:49:37.587869+0000","last_peered":"2024-03-28T16:49:37.587869+0000","last_clean":"2024-03-28T16:49:37.587869+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:49:37.587869+0000","last_undegraded":"2024-03-28T16:49:37.587869+0000","last_fullsized":"2024-03-28T16:49:37.587869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555560+0000","last_change":"2024-03-28T16:48:56.688050+0000","last_active":"2024-03-28T16:49:40.555560+0000","last_peered":"2024-03-28T16:49:40.555560+0000","last_clean":"2024-03-28T16:49:40.555560+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:49:40.555560+0000","last_undegraded":"2024-03-28T16:49:40.555560+0000","last_fullsized":"2024-03-28T16:49:40.555560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:32.377984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592873+0000","last_change":"2024-03-28T16:48:54.665976+0000","last_active":"2024-03-28T16:49:37.592873+0000","last_peered":"2024-03-28T16:49:37.592873+0000","last_clean":"2024-03-28T16:49:37.592873+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:49:37.592873+0000","last_undegraded":"2024-03-28T16:49:37.592873+0000","last_fullsized":"2024-03-28T16:49:37.592873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:38.265023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370127+0000","last_change":"2024-03-28T16:48:56.690734+0000","last_active":"2024-03-28T16:50:22.370127+0000","last_peered":"2024-03-28T16:50:22.370127+0000","last_clean":"2024-03-28T16:50:22.370127+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:50:22.370127+0000","last_undegraded":"2024-03-28T16:50:22.370127+0000","last_fullsized":"2024-03-28T16:50:22.370127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:26.690845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.374097+0000","last_change":"2024-03-28T16:48:54.656958+0000","last_active":"2024-03-28T16:50:22.374097+0000","last_peered":"2024-03-28T16:50:22.374097+0000","last_clean":"2024-03-28T16:50:22.374097+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:50:22.374097+0000","last_undegraded":"2024-03-28T16:50:22.374097+0000","last_fullsized":"2024-03-28T16:50:22.374097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.564230+0000","last_change":"2024-03-28T16:49:37.602262+0000","last_active":"2024-03-28T16:49:40.564230+0000","last_peered":"2024-03-28T16:49:40.564230+0000","last_clean":"2024-03-28T16:49:40.564230+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:49:40.564230+0000","last_undegraded":"2024-03-28T16:49:40.564230+0000","last_fullsized":"2024-03-28T16:49:40.564230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:12:52.595685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593536+0000","last_change":"2024-03-28T16:48:54.661971+0000","last_active":"2024-03-28T16:49:37.593536+0000","last_peered":"2024-03-28T16:49:37.593536+0000","last_clean":"2024-03-28T16:49:37.593536+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:49:37.593536+0000","last_undegraded":"2024-03-28T16:49:37.593536+0000","last_fullsized":"2024-03-28T16:49:37.593536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:29.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585408+0000","last_change":"2024-03-28T16:48:56.671464+0000","last_active":"2024-03-28T16:49:37.585408+0000","last_peered":"2024-03-28T16:49:37.585408+0000","last_clean":"2024-03-28T16:49:37.585408+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:49:37.585408+0000","last_undegraded":"2024-03-28T16:49:37.585408+0000","last_fullsized":"2024-03-28T16:49:37.585408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:18.173627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590098+0000","last_change":"2024-03-28T16:48:54.665454+0000","last_active":"2024-03-28T16:49:37.590098+0000","last_peered":"2024-03-28T16:49:37.590098+0000","last_clean":"2024-03-28T16:49:37.590098+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:49:37.590098+0000","last_undegraded":"2024-03-28T16:49:37.590098+0000","last_fullsized":"2024-03-28T16:49:37.590098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587928+0000","last_change":"2024-03-28T16:48:56.688375+0000","last_active":"2024-03-28T16:49:37.587928+0000","last_peered":"2024-03-28T16:49:37.587928+0000","last_clean":"2024-03-28T16:49:37.587928+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:49:37.587928+0000","last_undegraded":"2024-03-28T16:49:37.587928+0000","last_fullsized":"2024-03-28T16:49:37.587928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:56.236071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.859167+0000","last_change":"2024-03-28T16:48:54.672203+0000","last_active":"2024-03-28T16:49:39.859167+0000","last_peered":"2024-03-28T16:49:39.859167+0000","last_clean":"2024-03-28T16:49:39.859167+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:49:39.859167+0000","last_undegraded":"2024-03-28T16:49:39.859167+0000","last_fullsized":"2024-03-28T16:49:39.859167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:04:05.939269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555495+0000","last_change":"2024-03-28T16:48:56.695671+0000","last_active":"2024-03-28T16:49:40.555495+0000","last_peered":"2024-03-28T16:49:40.555495+0000","last_clean":"2024-03-28T16:49:40.555495+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:49:40.555495+0000","last_undegraded":"2024-03-28T16:49:40.555495+0000","last_fullsized":"2024-03-28T16:49:40.555495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585143+0000","last_change":"2024-03-28T16:48:54.658053+0000","last_active":"2024-03-28T16:49:37.585143+0000","last_peered":"2024-03-28T16:49:37.585143+0000","last_clean":"2024-03-28T16:49:37.585143+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:49:37.585143+0000","last_undegraded":"2024-03-28T16:49:37.585143+0000","last_fullsized":"2024-03-28T16:49:37.585143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839573+0000","last_change":"2024-03-28T16:48:56.681114+0000","last_active":"2024-03-28T16:50:28.839573+0000","last_peered":"2024-03-28T16:50:28.839573+0000","last_clean":"2024-03-28T16:50:28.839573+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:50:28.839573+0000","last_undegraded":"2024-03-28T16:50:28.839573+0000","last_fullsized":"2024-03-28T16:50:28.839573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527185+0000","last_change":"2024-03-28T16:48:56.677244+0000","last_active":"2024-03-28T16:49:39.527185+0000","last_peered":"2024-03-28T16:49:39.527185+0000","last_clean":"2024-03-28T16:49:39.527185+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:49:39.527185+0000","last_undegraded":"2024-03-28T16:49:39.527185+0000","last_fullsized":"2024-03-28T16:49:39.527185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:55.139244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602851+0000","last_change":"2024-03-28T16:48:54.671518+0000","last_active":"2024-03-28T16:49:37.602851+0000","last_peered":"2024-03-28T16:49:37.602851+0000","last_clean":"2024-03-28T16:49:37.602851+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:49:37.602851+0000","last_undegraded":"2024-03-28T16:49:37.602851+0000","last_fullsized":"2024-03-28T16:49:37.602851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'9","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.563949+0000","last_change":"2024-03-28T16:48:56.692955+0000","last_active":"2024-03-28T16:49:40.563949+0000","last_peered":"2024-03-28T16:49:40.563949+0000","last_clean":"2024-03-28T16:49:40.563949+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:49:40.563949+0000","last_undegraded":"2024-03-28T16:49:40.563949+0000","last_fullsized":"2024-03-28T16:49:40.563949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610918+0000","last_change":"2024-03-28T16:49:37.610918+0000","last_active":"2024-03-28T16:49:37.610918+0000","last_peered":"2024-03-28T16:49:37.610918+0000","last_clean":"2024-03-28T16:49:37.610918+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:49:37.610918+0000","last_undegraded":"2024-03-28T16:49:37.610918+0000","last_fullsized":"2024-03-28T16:49:37.610918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:21.127441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840116+0000","last_change":"2024-03-28T16:48:56.687940+0000","last_active":"2024-03-28T16:50:28.840116+0000","last_peered":"2024-03-28T16:50:28.840116+0000","last_clean":"2024-03-28T16:50:28.840116+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:50:28.840116+0000","last_undegraded":"2024-03-28T16:50:28.840116+0000","last_fullsized":"2024-03-28T16:50:28.840116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:40:18.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586038+0000","last_change":"2024-03-28T16:48:54.670148+0000","last_active":"2024-03-28T16:49:37.586038+0000","last_peered":"2024-03-28T16:49:37.586038+0000","last_clean":"2024-03-28T16:49:37.586038+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:49:37.586038+0000","last_undegraded":"2024-03-28T16:49:37.586038+0000","last_fullsized":"2024-03-28T16:49:37.586038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.865044+0000","last_change":"2024-03-28T16:48:56.678642+0000","last_active":"2024-03-28T16:49:39.865044+0000","last_peered":"2024-03-28T16:49:39.865044+0000","last_clean":"2024-03-28T16:49:39.865044+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:49:39.865044+0000","last_undegraded":"2024-03-28T16:49:39.865044+0000","last_fullsized":"2024-03-28T16:49:39.865044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586134+0000","last_change":"2024-03-28T16:48:54.655014+0000","last_active":"2024-03-28T16:49:37.586134+0000","last_peered":"2024-03-28T16:49:37.586134+0000","last_clean":"2024-03-28T16:49:37.586134+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:49:37.586134+0000","last_undegraded":"2024-03-28T16:49:37.586134+0000","last_fullsized":"2024-03-28T16:49:37.586134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.876760+0000","last_change":"2024-03-28T16:48:56.680884+0000","last_active":"2024-03-28T16:49:39.876760+0000","last_peered":"2024-03-28T16:49:39.876760+0000","last_clean":"2024-03-28T16:49:39.876760+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:49:39.876760+0000","last_undegraded":"2024-03-28T16:49:39.876760+0000","last_fullsized":"2024-03-28T16:49:39.876760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.842066+0000","last_change":"2024-03-28T16:48:54.670541+0000","last_active":"2024-03-28T16:49:39.842066+0000","last_peered":"2024-03-28T16:49:39.842066+0000","last_clean":"2024-03-28T16:49:39.842066+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:49:39.842066+0000","last_undegraded":"2024-03-28T16:49:39.842066+0000","last_fullsized":"2024-03-28T16:49:39.842066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854794+0000","last_change":"2024-03-28T16:48:56.671588+0000","last_active":"2024-03-28T16:50:28.854794+0000","last_peered":"2024-03-28T16:50:28.854794+0000","last_clean":"2024-03-28T16:50:28.854794+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:50:28.854794+0000","last_undegraded":"2024-03-28T16:50:28.854794+0000","last_fullsized":"2024-03-28T16:50:28.854794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:22.649572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582658+0000","last_change":"2024-03-28T16:48:54.673730+0000","last_active":"2024-03-28T16:49:37.582658+0000","last_peered":"2024-03-28T16:49:37.582658+0000","last_clean":"2024-03-28T16:49:37.582658+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:49:37.582658+0000","last_undegraded":"2024-03-28T16:49:37.582658+0000","last_fullsized":"2024-03-28T16:49:37.582658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:36:08.968340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'19","reported_seq":77,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.826736+0000","last_change":"2024-03-28T16:49:04.742579+0000","last_active":"2024-03-28T16:50:28.826736+0000","last_peered":"2024-03-28T16:50:28.826736+0000","last_clean":"2024-03-28T16:50:28.826736+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:50:28.826736+0000","last_undegraded":"2024-03-28T16:50:28.826736+0000","last_fullsized":"2024-03-28T16:50:28.826736+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:00.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839392+0000","last_change":"2024-03-28T16:48:56.683934+0000","last_active":"2024-03-28T16:50:28.839392+0000","last_peered":"2024-03-28T16:50:28.839392+0000","last_clean":"2024-03-28T16:50:28.839392+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:50:28.839392+0000","last_undegraded":"2024-03-28T16:50:28.839392+0000","last_fullsized":"2024-03-28T16:50:28.839392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:38.624478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592554+0000","last_change":"2024-03-28T16:48:54.670023+0000","last_active":"2024-03-28T16:49:37.592554+0000","last_peered":"2024-03-28T16:49:37.592554+0000","last_clean":"2024-03-28T16:49:37.592554+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:49:37.592554+0000","last_undegraded":"2024-03-28T16:49:37.592554+0000","last_fullsized":"2024-03-28T16:49:37.592554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587566+0000","last_change":"2024-03-28T16:48:58.699910+0000","last_active":"2024-03-28T16:49:37.587566+0000","last_peered":"2024-03-28T16:49:37.587566+0000","last_clean":"2024-03-28T16:49:37.587566+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:49:37.587566+0000","last_undegraded":"2024-03-28T16:49:37.587566+0000","last_fullsized":"2024-03-28T16:49:37.587566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:35.076776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.149453+0000","last_change":"2024-03-28T16:49:00.727669+0000","last_active":"2024-03-28T16:50:23.149453+0000","last_peered":"2024-03-28T16:50:23.149453+0000","last_clean":"2024-03-28T16:50:23.149453+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:50:23.149453+0000","last_undegraded":"2024-03-28T16:50:23.149453+0000","last_fullsized":"2024-03-28T16:50:23.149453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:28.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601464+0000","last_change":"2024-03-28T16:49:02.739439+0000","last_active":"2024-03-28T16:49:37.601464+0000","last_peered":"2024-03-28T16:49:37.601464+0000","last_clean":"2024-03-28T16:49:37.601464+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:49:37.601464+0000","last_undegraded":"2024-03-28T16:49:37.601464+0000","last_fullsized":"2024-03-28T16:49:37.601464+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'6","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.307391+0000","last_change":"2024-03-28T16:49:04.742984+0000","last_active":"2024-03-28T16:50:17.307391+0000","last_peered":"2024-03-28T16:50:17.307391+0000","last_clean":"2024-03-28T16:50:17.307391+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:50:17.307391+0000","last_undegraded":"2024-03-28T16:50:17.307391+0000","last_fullsized":"2024-03-28T16:50:17.307391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:30.200369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839461+0000","last_change":"2024-03-28T16:48:56.671488+0000","last_active":"2024-03-28T16:50:28.839461+0000","last_peered":"2024-03-28T16:50:28.839461+0000","last_clean":"2024-03-28T16:50:28.839461+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:50:28.839461+0000","last_undegraded":"2024-03-28T16:50:28.839461+0000","last_fullsized":"2024-03-28T16:50:28.839461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:39.819684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591228+0000","last_change":"2024-03-28T16:48:54.655701+0000","last_active":"2024-03-28T16:49:37.591228+0000","last_peered":"2024-03-28T16:49:37.591228+0000","last_clean":"2024-03-28T16:49:37.591228+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:49:37.591228+0000","last_undegraded":"2024-03-28T16:49:37.591228+0000","last_fullsized":"2024-03-28T16:49:37.591228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585880+0000","last_change":"2024-03-28T16:48:58.689758+0000","last_active":"2024-03-28T16:49:37.585880+0000","last_peered":"2024-03-28T16:49:37.585880+0000","last_clean":"2024-03-28T16:49:37.585880+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:49:37.585880+0000","last_undegraded":"2024-03-28T16:49:37.585880+0000","last_fullsized":"2024-03-28T16:49:37.585880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:08.600334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"29'7","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.605037+0000","last_change":"2024-03-28T16:49:00.717460+0000","last_active":"2024-03-28T16:49:37.605037+0000","last_peered":"2024-03-28T16:49:37.605037+0000","last_clean":"2024-03-28T16:49:37.605037+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:49:37.605037+0000","last_undegraded":"2024-03-28T16:49:37.605037+0000","last_fullsized":"2024-03-28T16:49:37.605037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:26.416340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586231+0000","last_change":"2024-03-28T16:49:02.730400+0000","last_active":"2024-03-28T16:49:37.586231+0000","last_peered":"2024-03-28T16:49:37.586231+0000","last_clean":"2024-03-28T16:49:37.586231+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:49:37.586231+0000","last_undegraded":"2024-03-28T16:49:37.586231+0000","last_fullsized":"2024-03-28T16:49:37.586231+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584337+0000","last_change":"2024-03-28T16:49:04.743161+0000","last_active":"2024-03-28T16:49:37.584337+0000","last_peered":"2024-03-28T16:49:37.584337+0000","last_clean":"2024-03-28T16:49:37.584337+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:49:37.584337+0000","last_undegraded":"2024-03-28T16:49:37.584337+0000","last_fullsized":"2024-03-28T16:49:37.584337+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189874+0000","last_change":"2024-03-28T16:48:56.676331+0000","last_active":"2024-03-28T16:50:23.189874+0000","last_peered":"2024-03-28T16:50:23.189874+0000","last_clean":"2024-03-28T16:50:23.189874+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:50:23.189874+0000","last_undegraded":"2024-03-28T16:50:23.189874+0000","last_fullsized":"2024-03-28T16:50:23.189874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:33.697214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592955+0000","last_change":"2024-03-28T16:48:54.659330+0000","last_active":"2024-03-28T16:49:37.592955+0000","last_peered":"2024-03-28T16:49:37.592955+0000","last_clean":"2024-03-28T16:49:37.592955+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:49:37.592955+0000","last_undegraded":"2024-03-28T16:49:37.592955+0000","last_fullsized":"2024-03-28T16:49:37.592955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:30:54.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'5","reported_seq":203,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.364471+0000","last_change":"2024-03-28T16:48:58.687156+0000","last_active":"2024-03-28T16:50:30.364471+0000","last_peered":"2024-03-28T16:50:30.364471+0000","last_clean":"2024-03-28T16:50:30.364471+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:50:30.364471+0000","last_undegraded":"2024-03-28T16:50:30.364471+0000","last_fullsized":"2024-03-28T16:50:30.364471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:25:52.416012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":267,"num_read_kb":266,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'11","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595817+0000","last_change":"2024-03-28T16:49:00.704257+0000","last_active":"2024-03-28T16:49:37.595817+0000","last_peered":"2024-03-28T16:49:37.595817+0000","last_clean":"2024-03-28T16:49:37.595817+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:49:37.595817+0000","last_undegraded":"2024-03-28T16:49:37.595817+0000","last_fullsized":"2024-03-28T16:49:37.595817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:59.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'246","reported_seq":329,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854124+0000","last_change":"2024-03-28T16:49:02.737451+0000","last_active":"2024-03-28T16:50:33.854124+0000","last_peered":"2024-03-28T16:50:33.854124+0000","last_clean":"2024-03-28T16:50:33.854124+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:50:33.854124+0000","last_undegraded":"2024-03-28T16:50:33.854124+0000","last_fullsized":"2024-03-28T16:50:33.854124+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:34.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591631+0000","last_change":"2024-03-28T16:49:04.741486+0000","last_active":"2024-03-28T16:49:37.591631+0000","last_peered":"2024-03-28T16:49:37.591631+0000","last_clean":"2024-03-28T16:49:37.591631+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:49:37.591631+0000","last_undegraded":"2024-03-28T16:49:37.591631+0000","last_fullsized":"2024-03-28T16:49:37.591631+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:51.610298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591048+0000","last_change":"2024-03-28T16:48:56.697774+0000","last_active":"2024-03-28T16:49:37.591048+0000","last_peered":"2024-03-28T16:49:37.591048+0000","last_clean":"2024-03-28T16:49:37.591048+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:49:37.591048+0000","last_undegraded":"2024-03-28T16:49:37.591048+0000","last_fullsized":"2024-03-28T16:49:37.591048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583578+0000","last_change":"2024-03-28T16:48:54.659035+0000","last_active":"2024-03-28T16:49:37.583578+0000","last_peered":"2024-03-28T16:49:37.583578+0000","last_clean":"2024-03-28T16:49:37.583578+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:49:37.583578+0000","last_undegraded":"2024-03-28T16:49:37.583578+0000","last_fullsized":"2024-03-28T16:49:37.583578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:59.516240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594556+0000","last_change":"2024-03-28T16:48:58.706776+0000","last_active":"2024-03-28T16:49:37.594556+0000","last_peered":"2024-03-28T16:49:37.594556+0000","last_clean":"2024-03-28T16:49:37.594556+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:49:37.594556+0000","last_undegraded":"2024-03-28T16:49:37.594556+0000","last_fullsized":"2024-03-28T16:49:37.594556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:18.066504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.142398+0000","last_change":"2024-03-28T16:49:00.710996+0000","last_active":"2024-03-28T16:50:23.142398+0000","last_peered":"2024-03-28T16:50:23.142398+0000","last_clean":"2024-03-28T16:50:23.142398+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:23.142398+0000","last_undegraded":"2024-03-28T16:50:23.142398+0000","last_fullsized":"2024-03-28T16:50:23.142398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:05.450361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'246","reported_seq":344,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.481011+0000","last_change":"2024-03-28T16:49:02.734376+0000","last_active":"2024-03-28T16:50:35.481011+0000","last_peered":"2024-03-28T16:50:35.481011+0000","last_clean":"2024-03-28T16:50:35.481011+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:50:35.481011+0000","last_undegraded":"2024-03-28T16:50:35.481011+0000","last_fullsized":"2024-03-28T16:50:35.481011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581627+0000","last_change":"2024-03-28T16:48:56.693376+0000","last_active":"2024-03-28T16:49:37.581627+0000","last_peered":"2024-03-28T16:49:37.581627+0000","last_clean":"2024-03-28T16:49:37.581627+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:49:37.581627+0000","last_undegraded":"2024-03-28T16:49:37.581627+0000","last_fullsized":"2024-03-28T16:49:37.581627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:27.504691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600235+0000","last_change":"2024-03-28T16:48:54.671982+0000","last_active":"2024-03-28T16:49:37.600235+0000","last_peered":"2024-03-28T16:49:37.600235+0000","last_clean":"2024-03-28T16:49:37.600235+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:49:37.600235+0000","last_undegraded":"2024-03-28T16:49:37.600235+0000","last_fullsized":"2024-03-28T16:49:37.600235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586619+0000","last_change":"2024-03-28T16:48:58.687997+0000","last_active":"2024-03-28T16:49:37.586619+0000","last_peered":"2024-03-28T16:49:37.586619+0000","last_clean":"2024-03-28T16:49:37.586619+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:49:37.586619+0000","last_undegraded":"2024-03-28T16:49:37.586619+0000","last_fullsized":"2024-03-28T16:49:37.586619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:29.519686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'38","reported_seq":84,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.155887+0000","last_change":"2024-03-28T16:49:00.713077+0000","last_active":"2024-03-28T16:50:23.155887+0000","last_peered":"2024-03-28T16:50:23.155887+0000","last_clean":"2024-03-28T16:50:23.155887+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:23.155887+0000","last_undegraded":"2024-03-28T16:50:23.155887+0000","last_fullsized":"2024-03-28T16:50:23.155887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:46.727245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587114+0000","last_change":"2024-03-28T16:49:02.730507+0000","last_active":"2024-03-28T16:49:37.587114+0000","last_peered":"2024-03-28T16:49:37.587114+0000","last_clean":"2024-03-28T16:49:37.587114+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:49:37.587114+0000","last_undegraded":"2024-03-28T16:49:37.587114+0000","last_fullsized":"2024-03-28T16:49:37.587114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.935069+0000","last_change":"2024-03-28T16:49:04.728316+0000","last_active":"2024-03-28T16:50:22.935069+0000","last_peered":"2024-03-28T16:50:22.935069+0000","last_clean":"2024-03-28T16:50:22.935069+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:22.935069+0000","last_undegraded":"2024-03-28T16:50:22.935069+0000","last_fullsized":"2024-03-28T16:50:22.935069+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:28.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'22","reported_seq":100,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729800+0000","last_change":"2024-03-28T16:48:56.678278+0000","last_active":"2024-03-28T16:50:22.729800+0000","last_peered":"2024-03-28T16:50:22.729800+0000","last_clean":"2024-03-28T16:50:22.729800+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:50:22.729800+0000","last_undegraded":"2024-03-28T16:50:22.729800+0000","last_fullsized":"2024-03-28T16:50:22.729800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.051186+0000","last_change":"2024-03-28T16:48:54.672277+0000","last_active":"2024-03-28T16:49:39.051186+0000","last_peered":"2024-03-28T16:49:39.051186+0000","last_clean":"2024-03-28T16:49:39.051186+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:49:39.051186+0000","last_undegraded":"2024-03-28T16:49:39.051186+0000","last_fullsized":"2024-03-28T16:49:39.051186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585905+0000","last_change":"2024-03-28T16:48:58.690263+0000","last_active":"2024-03-28T16:49:37.585905+0000","last_peered":"2024-03-28T16:49:37.585905+0000","last_clean":"2024-03-28T16:49:37.585905+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:49:37.585905+0000","last_undegraded":"2024-03-28T16:49:37.585905+0000","last_fullsized":"2024-03-28T16:49:37.585905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:25.224033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.129187+0000","last_change":"2024-03-28T16:49:00.727195+0000","last_active":"2024-03-28T16:50:23.129187+0000","last_peered":"2024-03-28T16:50:23.129187+0000","last_clean":"2024-03-28T16:50:23.129187+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:50:23.129187+0000","last_undegraded":"2024-03-28T16:50:23.129187+0000","last_fullsized":"2024-03-28T16:50:23.129187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:48.531527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585871+0000","last_change":"2024-03-28T16:49:02.714319+0000","last_active":"2024-03-28T16:49:37.585871+0000","last_peered":"2024-03-28T16:49:37.585871+0000","last_clean":"2024-03-28T16:49:37.585871+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:49:37.585871+0000","last_undegraded":"2024-03-28T16:49:37.585871+0000","last_fullsized":"2024-03-28T16:49:37.585871+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:49.522623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'3","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.157047+0000","last_change":"2024-03-28T16:49:04.738375+0000","last_active":"2024-03-28T16:50:17.157047+0000","last_peered":"2024-03-28T16:50:17.157047+0000","last_clean":"2024-03-28T16:50:17.157047+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:50:17.157047+0000","last_undegraded":"2024-03-28T16:50:17.157047+0000","last_fullsized":"2024-03-28T16:50:17.157047+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:31.198874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'3","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.537250+0000","last_change":"2024-03-28T16:48:56.681600+0000","last_active":"2024-03-28T16:49:39.537250+0000","last_peered":"2024-03-28T16:49:39.537250+0000","last_clean":"2024-03-28T16:49:39.537250+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:49:39.537250+0000","last_undegraded":"2024-03-28T16:49:39.537250+0000","last_fullsized":"2024-03-28T16:49:39.537250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588203+0000","last_change":"2024-03-28T16:48:54.676553+0000","last_active":"2024-03-28T16:49:37.588203+0000","last_peered":"2024-03-28T16:49:37.588203+0000","last_clean":"2024-03-28T16:49:37.588203+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:49:37.588203+0000","last_undegraded":"2024-03-28T16:49:37.588203+0000","last_fullsized":"2024-03-28T16:49:37.588203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:11.709923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592044+0000","last_change":"2024-03-28T16:48:58.682276+0000","last_active":"2024-03-28T16:49:37.592044+0000","last_peered":"2024-03-28T16:49:37.592044+0000","last_clean":"2024-03-28T16:49:37.592044+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:49:37.592044+0000","last_undegraded":"2024-03-28T16:49:37.592044+0000","last_fullsized":"2024-03-28T16:49:37.592044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:44.713820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.158232+0000","last_change":"2024-03-28T16:49:00.720840+0000","last_active":"2024-03-28T16:50:23.158232+0000","last_peered":"2024-03-28T16:50:23.158232+0000","last_clean":"2024-03-28T16:50:23.158232+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:50:23.158232+0000","last_undegraded":"2024-03-28T16:50:23.158232+0000","last_fullsized":"2024-03-28T16:50:23.158232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:28.746453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'246","reported_seq":339,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854481+0000","last_change":"2024-03-28T16:49:02.713412+0000","last_active":"2024-03-28T16:50:33.854481+0000","last_peered":"2024-03-28T16:50:33.854481+0000","last_clean":"2024-03-28T16:50:33.854481+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:33.854481+0000","last_undegraded":"2024-03-28T16:50:33.854481+0000","last_fullsized":"2024-03-28T16:50:33.854481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.828873+0000","last_change":"2024-03-28T16:49:04.743207+0000","last_active":"2024-03-28T16:50:28.828873+0000","last_peered":"2024-03-28T16:50:28.828873+0000","last_clean":"2024-03-28T16:50:28.828873+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:50:28.828873+0000","last_undegraded":"2024-03-28T16:50:28.828873+0000","last_fullsized":"2024-03-28T16:50:28.828873+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:12.429622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.864959+0000","last_change":"2024-03-28T16:48:56.685083+0000","last_active":"2024-03-28T16:49:39.864959+0000","last_peered":"2024-03-28T16:49:39.864959+0000","last_clean":"2024-03-28T16:49:39.864959+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:49:39.864959+0000","last_undegraded":"2024-03-28T16:49:39.864959+0000","last_fullsized":"2024-03-28T16:49:39.864959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587111+0000","last_change":"2024-03-28T16:48:54.658093+0000","last_active":"2024-03-28T16:49:37.587111+0000","last_peered":"2024-03-28T16:49:37.587111+0000","last_clean":"2024-03-28T16:49:37.587111+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:49:37.587111+0000","last_undegraded":"2024-03-28T16:49:37.587111+0000","last_fullsized":"2024-03-28T16:49:37.587111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:20.809568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586227+0000","last_change":"2024-03-28T16:48:58.682237+0000","last_active":"2024-03-28T16:49:37.586227+0000","last_peered":"2024-03-28T16:49:37.586227+0000","last_clean":"2024-03-28T16:49:37.586227+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:49:37.586227+0000","last_undegraded":"2024-03-28T16:49:37.586227+0000","last_fullsized":"2024-03-28T16:49:37.586227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:37.069394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'50","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.781121+0000","last_change":"2024-03-28T16:49:00.725827+0000","last_active":"2024-03-28T16:50:28.781121+0000","last_peered":"2024-03-28T16:50:28.781121+0000","last_clean":"2024-03-28T16:50:28.781121+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:50:28.781121+0000","last_undegraded":"2024-03-28T16:50:28.781121+0000","last_fullsized":"2024-03-28T16:50:28.781121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:56.316763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585610+0000","last_change":"2024-03-28T16:49:02.712054+0000","last_active":"2024-03-28T16:49:37.585610+0000","last_peered":"2024-03-28T16:49:37.585610+0000","last_clean":"2024-03-28T16:49:37.585610+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:49:37.585610+0000","last_undegraded":"2024-03-28T16:49:37.585610+0000","last_fullsized":"2024-03-28T16:49:37.585610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587403+0000","last_change":"2024-03-28T16:49:04.723184+0000","last_active":"2024-03-28T16:49:37.587403+0000","last_peered":"2024-03-28T16:49:37.587403+0000","last_clean":"2024-03-28T16:49:37.587403+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:49:37.587403+0000","last_undegraded":"2024-03-28T16:49:37.587403+0000","last_fullsized":"2024-03-28T16:49:37.587403+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:32.776436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877381+0000","last_change":"2024-03-28T16:48:56.683578+0000","last_active":"2024-03-28T16:49:39.877381+0000","last_peered":"2024-03-28T16:49:39.877381+0000","last_clean":"2024-03-28T16:49:39.877381+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:49:39.877381+0000","last_undegraded":"2024-03-28T16:49:39.877381+0000","last_fullsized":"2024-03-28T16:49:39.877381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:25.885438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588449+0000","last_change":"2024-03-28T16:48:54.664924+0000","last_active":"2024-03-28T16:49:37.588449+0000","last_peered":"2024-03-28T16:49:37.588449+0000","last_clean":"2024-03-28T16:49:37.588449+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:49:37.588449+0000","last_undegraded":"2024-03-28T16:49:37.588449+0000","last_fullsized":"2024-03-28T16:49:37.588449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:27:34.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595270+0000","last_change":"2024-03-28T16:48:58.692108+0000","last_active":"2024-03-28T16:49:37.595270+0000","last_peered":"2024-03-28T16:49:37.595270+0000","last_clean":"2024-03-28T16:49:37.595270+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:49:37.595270+0000","last_undegraded":"2024-03-28T16:49:37.595270+0000","last_fullsized":"2024-03-28T16:49:37.595270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:05.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'30","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.134000+0000","last_change":"2024-03-28T16:49:00.721579+0000","last_active":"2024-03-28T16:50:23.134000+0000","last_peered":"2024-03-28T16:50:23.134000+0000","last_clean":"2024-03-28T16:50:23.134000+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:23.134000+0000","last_undegraded":"2024-03-28T16:50:23.134000+0000","last_fullsized":"2024-03-28T16:50:23.134000+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:13.394379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587827+0000","last_change":"2024-03-28T16:49:02.739236+0000","last_active":"2024-03-28T16:49:37.587827+0000","last_peered":"2024-03-28T16:49:37.587827+0000","last_clean":"2024-03-28T16:49:37.587827+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:49:37.587827+0000","last_undegraded":"2024-03-28T16:49:37.587827+0000","last_fullsized":"2024-03-28T16:49:37.587827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:34:20.723680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580108+0000","last_change":"2024-03-28T16:49:04.748645+0000","last_active":"2024-03-28T16:49:37.580108+0000","last_peered":"2024-03-28T16:49:37.580108+0000","last_clean":"2024-03-28T16:49:37.580108+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:49:37.580108+0000","last_undegraded":"2024-03-28T16:49:37.580108+0000","last_fullsized":"2024-03-28T16:49:37.580108+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:06:34.291471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593462+0000","last_change":"2024-03-28T16:48:56.695870+0000","last_active":"2024-03-28T16:49:37.593462+0000","last_peered":"2024-03-28T16:49:37.593462+0000","last_clean":"2024-03-28T16:49:37.593462+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:49:37.593462+0000","last_undegraded":"2024-03-28T16:49:37.593462+0000","last_fullsized":"2024-03-28T16:49:37.593462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:02.971815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589696+0000","last_change":"2024-03-28T16:48:54.673299+0000","last_active":"2024-03-28T16:49:37.589696+0000","last_peered":"2024-03-28T16:49:37.589696+0000","last_clean":"2024-03-28T16:49:37.589696+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:49:37.589696+0000","last_undegraded":"2024-03-28T16:49:37.589696+0000","last_fullsized":"2024-03-28T16:49:37.589696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:37.511744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588920+0000","last_change":"2024-03-28T16:48:58.683855+0000","last_active":"2024-03-28T16:49:37.588920+0000","last_peered":"2024-03-28T16:49:37.588920+0000","last_clean":"2024-03-28T16:49:37.588920+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:49:37.588920+0000","last_undegraded":"2024-03-28T16:49:37.588920+0000","last_fullsized":"2024-03-28T16:49:37.588920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:01.947384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'35","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.825556+0000","last_change":"2024-03-28T16:49:00.697730+0000","last_active":"2024-03-28T16:50:28.825556+0000","last_peered":"2024-03-28T16:50:28.825556+0000","last_clean":"2024-03-28T16:50:28.825556+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:50:28.825556+0000","last_undegraded":"2024-03-28T16:50:28.825556+0000","last_fullsized":"2024-03-28T16:50:28.825556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:27.038167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'246","reported_seq":322,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854703+0000","last_change":"2024-03-28T16:49:02.726890+0000","last_active":"2024-03-28T16:50:33.854703+0000","last_peered":"2024-03-28T16:50:33.854703+0000","last_clean":"2024-03-28T16:50:33.854703+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:33.854703+0000","last_undegraded":"2024-03-28T16:50:33.854703+0000","last_fullsized":"2024-03-28T16:50:33.854703+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:42.002449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'13","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.217530+0000","last_change":"2024-03-28T16:49:04.743257+0000","last_active":"2024-03-28T16:50:23.217530+0000","last_peered":"2024-03-28T16:50:23.217530+0000","last_clean":"2024-03-28T16:50:23.217530+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:50:23.217530+0000","last_undegraded":"2024-03-28T16:50:23.217530+0000","last_fullsized":"2024-03-28T16:50:23.217530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839667+0000","last_change":"2024-03-28T16:48:56.690053+0000","last_active":"2024-03-28T16:50:28.839667+0000","last_peered":"2024-03-28T16:50:28.839667+0000","last_clean":"2024-03-28T16:50:28.839667+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:50:28.839667+0000","last_undegraded":"2024-03-28T16:50:28.839667+0000","last_fullsized":"2024-03-28T16:50:28.839667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581989+0000","last_change":"2024-03-28T16:48:54.664571+0000","last_active":"2024-03-28T16:49:37.581989+0000","last_peered":"2024-03-28T16:49:37.581989+0000","last_clean":"2024-03-28T16:49:37.581989+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:49:37.581989+0000","last_undegraded":"2024-03-28T16:49:37.581989+0000","last_fullsized":"2024-03-28T16:49:37.581989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:56:22.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'3","reported_seq":378,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.377726+0000","last_change":"2024-03-28T16:49:37.622013+0000","last_active":"2024-03-28T16:50:30.377726+0000","last_peered":"2024-03-28T16:50:30.377726+0000","last_clean":"2024-03-28T16:50:30.377726+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:50:30.377726+0000","last_undegraded":"2024-03-28T16:50:30.377726+0000","last_fullsized":"2024-03-28T16:50:30.377726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:46.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":336,"num_read_kb":336,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587255+0000","last_change":"2024-03-28T16:49:00.725908+0000","last_active":"2024-03-28T16:49:37.587255+0000","last_peered":"2024-03-28T16:49:37.587255+0000","last_clean":"2024-03-28T16:49:37.587255+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:49:37.587255+0000","last_undegraded":"2024-03-28T16:49:37.587255+0000","last_fullsized":"2024-03-28T16:49:37.587255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:40.250784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'246","reported_seq":345,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854050+0000","last_change":"2024-03-28T16:49:02.715172+0000","last_active":"2024-03-28T16:50:33.854050+0000","last_peered":"2024-03-28T16:50:33.854050+0000","last_clean":"2024-03-28T16:50:33.854050+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:33.854050+0000","last_undegraded":"2024-03-28T16:50:33.854050+0000","last_fullsized":"2024-03-28T16:50:33.854050+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589680+0000","last_change":"2024-03-28T16:49:04.746465+0000","last_active":"2024-03-28T16:49:37.589680+0000","last_peered":"2024-03-28T16:49:37.589680+0000","last_clean":"2024-03-28T16:49:37.589680+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:49:37.589680+0000","last_undegraded":"2024-03-28T16:49:37.589680+0000","last_fullsized":"2024-03-28T16:49:37.589680+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:41.396912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189625+0000","last_change":"2024-03-28T16:48:56.681161+0000","last_active":"2024-03-28T16:50:23.189625+0000","last_peered":"2024-03-28T16:50:23.189625+0000","last_clean":"2024-03-28T16:50:23.189625+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:50:23.189625+0000","last_undegraded":"2024-03-28T16:50:23.189625+0000","last_fullsized":"2024-03-28T16:50:23.189625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:23.209038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580546+0000","last_change":"2024-03-28T16:48:54.676487+0000","last_active":"2024-03-28T16:49:37.580546+0000","last_peered":"2024-03-28T16:49:37.580546+0000","last_clean":"2024-03-28T16:49:37.580546+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:49:37.580546+0000","last_undegraded":"2024-03-28T16:49:37.580546+0000","last_fullsized":"2024-03-28T16:49:37.580546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:15.844441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588709+0000","last_change":"2024-03-28T16:48:58.703765+0000","last_active":"2024-03-28T16:49:37.588709+0000","last_peered":"2024-03-28T16:49:37.588709+0000","last_clean":"2024-03-28T16:49:37.588709+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:49:37.588709+0000","last_undegraded":"2024-03-28T16:49:37.588709+0000","last_fullsized":"2024-03-28T16:49:37.588709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:05:16.548217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'36","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.151478+0000","last_change":"2024-03-28T16:49:00.711767+0000","last_active":"2024-03-28T16:50:23.151478+0000","last_peered":"2024-03-28T16:50:23.151478+0000","last_clean":"2024-03-28T16:50:23.151478+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:23.151478+0000","last_undegraded":"2024-03-28T16:50:23.151478+0000","last_fullsized":"2024-03-28T16:50:23.151478+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:07.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590333+0000","last_change":"2024-03-28T16:49:02.720427+0000","last_active":"2024-03-28T16:49:37.590333+0000","last_peered":"2024-03-28T16:49:37.590333+0000","last_clean":"2024-03-28T16:49:37.590333+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:49:37.590333+0000","last_undegraded":"2024-03-28T16:49:37.590333+0000","last_fullsized":"2024-03-28T16:49:37.590333+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:02.281547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596396+0000","last_change":"2024-03-28T16:49:04.730860+0000","last_active":"2024-03-28T16:49:37.596396+0000","last_peered":"2024-03-28T16:49:37.596396+0000","last_clean":"2024-03-28T16:49:37.596396+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:49:37.596396+0000","last_undegraded":"2024-03-28T16:49:37.596396+0000","last_fullsized":"2024-03-28T16:49:37.596396+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189979+0000","last_change":"2024-03-28T16:48:56.693994+0000","last_active":"2024-03-28T16:50:23.189979+0000","last_peered":"2024-03-28T16:50:23.189979+0000","last_clean":"2024-03-28T16:50:23.189979+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:50:23.189979+0000","last_undegraded":"2024-03-28T16:50:23.189979+0000","last_fullsized":"2024-03-28T16:50:23.189979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:09.568996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":198,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583310+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:49:37.583310+0000","last_peered":"2024-03-28T16:49:37.583310+0000","last_clean":"2024-03-28T16:49:37.583310+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:49:37.583310+0000","last_undegraded":"2024-03-28T16:49:37.583310+0000","last_fullsized":"2024-03-28T16:49:37.583310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:35.744620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593784+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:49:37.593784+0000","last_peered":"2024-03-28T16:49:37.593784+0000","last_clean":"2024-03-28T16:49:37.593784+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:49:37.593784+0000","last_undegraded":"2024-03-28T16:49:37.593784+0000","last_fullsized":"2024-03-28T16:49:37.593784+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.194802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581636+0000","last_change":"2024-03-28T16:48:54.666099+0000","last_active":"2024-03-28T16:49:37.581636+0000","last_peered":"2024-03-28T16:49:37.581636+0000","last_clean":"2024-03-28T16:49:37.581636+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:49:37.581636+0000","last_undegraded":"2024-03-28T16:49:37.581636+0000","last_fullsized":"2024-03-28T16:49:37.581636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:03.153093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585495+0000","last_change":"2024-03-28T16:48:58.683594+0000","last_active":"2024-03-28T16:49:37.585495+0000","last_peered":"2024-03-28T16:49:37.585495+0000","last_clean":"2024-03-28T16:49:37.585495+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:49:37.585495+0000","last_undegraded":"2024-03-28T16:49:37.585495+0000","last_fullsized":"2024-03-28T16:49:37.585495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:14.457437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'70","reported_seq":138,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.807275+0000","last_change":"2024-03-28T16:49:00.715887+0000","last_active":"2024-03-28T16:50:28.807275+0000","last_peered":"2024-03-28T16:50:28.807275+0000","last_clean":"2024-03-28T16:50:28.807275+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:28.807275+0000","last_undegraded":"2024-03-28T16:50:28.807275+0000","last_fullsized":"2024-03-28T16:50:28.807275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:37:58.662402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591287+0000","last_change":"2024-03-28T16:49:02.731593+0000","last_active":"2024-03-28T16:49:37.591287+0000","last_peered":"2024-03-28T16:49:37.591287+0000","last_clean":"2024-03-28T16:49:37.591287+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:49:37.591287+0000","last_undegraded":"2024-03-28T16:49:37.591287+0000","last_fullsized":"2024-03-28T16:49:37.591287+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:36.899516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'52","reported_seq":114,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.870448+0000","last_change":"2024-03-28T16:49:04.752277+0000","last_active":"2024-03-28T16:50:28.870448+0000","last_peered":"2024-03-28T16:50:28.870448+0000","last_clean":"2024-03-28T16:50:28.870448+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:50:28.870448+0000","last_undegraded":"2024-03-28T16:50:28.870448+0000","last_fullsized":"2024-03-28T16:50:28.870448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:10.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'11","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.202723+0000","last_change":"2024-03-28T16:49:37.614693+0000","last_active":"2024-03-28T16:50:23.202723+0000","last_peered":"2024-03-28T16:50:23.202723+0000","last_clean":"2024-03-28T16:50:23.202723+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:50:23.202723+0000","last_undegraded":"2024-03-28T16:50:23.202723+0000","last_fullsized":"2024-03-28T16:50:23.202723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:53.196986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580404+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:49:37.580404+0000","last_peered":"2024-03-28T16:49:37.580404+0000","last_clean":"2024-03-28T16:49:37.580404+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:49:37.580404+0000","last_undegraded":"2024-03-28T16:49:37.580404+0000","last_fullsized":"2024-03-28T16:49:37.580404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:11.830948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585095+0000","last_change":"2024-03-28T16:48:54.662603+0000","last_active":"2024-03-28T16:49:37.585095+0000","last_peered":"2024-03-28T16:49:37.585095+0000","last_clean":"2024-03-28T16:49:37.585095+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:49:37.585095+0000","last_undegraded":"2024-03-28T16:49:37.585095+0000","last_fullsized":"2024-03-28T16:49:37.585095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594800+0000","last_change":"2024-03-28T16:48:58.706305+0000","last_active":"2024-03-28T16:49:37.594800+0000","last_peered":"2024-03-28T16:49:37.594800+0000","last_clean":"2024-03-28T16:49:37.594800+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:49:37.594800+0000","last_undegraded":"2024-03-28T16:49:37.594800+0000","last_fullsized":"2024-03-28T16:49:37.594800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:32.869035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'243","reported_seq":415,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854148+0000","last_change":"2024-03-28T16:49:00.718247+0000","last_active":"2024-03-28T16:50:33.854148+0000","last_peered":"2024-03-28T16:50:33.854148+0000","last_clean":"2024-03-28T16:50:33.854148+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:33.854148+0000","last_undegraded":"2024-03-28T16:50:33.854148+0000","last_fullsized":"2024-03-28T16:50:33.854148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:43.848104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":321,"num_read_kb":306,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582450+0000","last_change":"2024-03-28T16:49:02.740444+0000","last_active":"2024-03-28T16:49:37.582450+0000","last_peered":"2024-03-28T16:49:37.582450+0000","last_clean":"2024-03-28T16:49:37.582450+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:49:37.582450+0000","last_undegraded":"2024-03-28T16:49:37.582450+0000","last_fullsized":"2024-03-28T16:49:37.582450+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:41.589208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'4","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.938249+0000","last_change":"2024-03-28T16:49:04.744802+0000","last_active":"2024-03-28T16:50:22.938249+0000","last_peered":"2024-03-28T16:50:22.938249+0000","last_clean":"2024-03-28T16:50:22.938249+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:50:22.938249+0000","last_undegraded":"2024-03-28T16:50:22.938249+0000","last_fullsized":"2024-03-28T16:50:22.938249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.556077+0000","last_change":"2024-03-28T16:48:56.689836+0000","last_active":"2024-03-28T16:49:40.556077+0000","last_peered":"2024-03-28T16:49:40.556077+0000","last_clean":"2024-03-28T16:49:40.556077+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:49:40.556077+0000","last_undegraded":"2024-03-28T16:49:40.556077+0000","last_fullsized":"2024-03-28T16:49:40.556077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589208+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:49:37.589208+0000","last_peered":"2024-03-28T16:49:37.589208+0000","last_clean":"2024-03-28T16:49:37.589208+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:49:37.589208+0000","last_undegraded":"2024-03-28T16:49:37.589208+0000","last_fullsized":"2024-03-28T16:49:37.589208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584655+0000","last_change":"2024-03-28T16:48:54.663058+0000","last_active":"2024-03-28T16:49:37.584655+0000","last_peered":"2024-03-28T16:49:37.584655+0000","last_clean":"2024-03-28T16:49:37.584655+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:49:37.584655+0000","last_undegraded":"2024-03-28T16:49:37.584655+0000","last_fullsized":"2024-03-28T16:49:37.584655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"22'1","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.599332+0000","last_change":"2024-03-28T16:48:58.687188+0000","last_active":"2024-03-28T16:49:37.599332+0000","last_peered":"2024-03-28T16:49:37.599332+0000","last_clean":"2024-03-28T16:49:37.599332+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:49:37.599332+0000","last_undegraded":"2024-03-28T16:49:37.599332+0000","last_fullsized":"2024-03-28T16:49:37.599332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:27.900434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'16","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854777+0000","last_change":"2024-03-28T16:49:00.713728+0000","last_active":"2024-03-28T16:50:33.854777+0000","last_peered":"2024-03-28T16:50:33.854777+0000","last_clean":"2024-03-28T16:50:33.854777+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:33.854777+0000","last_undegraded":"2024-03-28T16:50:33.854777+0000","last_fullsized":"2024-03-28T16:50:33.854777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:03.960810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586565+0000","last_change":"2024-03-28T16:49:02.732230+0000","last_active":"2024-03-28T16:49:37.586565+0000","last_peered":"2024-03-28T16:49:37.586565+0000","last_clean":"2024-03-28T16:49:37.586565+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:49:37.586565+0000","last_undegraded":"2024-03-28T16:49:37.586565+0000","last_fullsized":"2024-03-28T16:49:37.586565+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:20.091845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581243+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:49:37.581243+0000","last_peered":"2024-03-28T16:49:37.581243+0000","last_clean":"2024-03-28T16:49:37.581243+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:49:37.581243+0000","last_undegraded":"2024-03-28T16:49:37.581243+0000","last_fullsized":"2024-03-28T16:49:37.581243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:51.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587517+0000","last_change":"2024-03-28T16:48:56.683769+0000","last_active":"2024-03-28T16:49:37.587517+0000","last_peered":"2024-03-28T16:49:37.587517+0000","last_clean":"2024-03-28T16:49:37.587517+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:49:37.587517+0000","last_undegraded":"2024-03-28T16:49:37.587517+0000","last_fullsized":"2024-03-28T16:49:37.587517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581400+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:49:37.581400+0000","last_peered":"2024-03-28T16:49:37.581400+0000","last_clean":"2024-03-28T16:49:37.581400+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:49:37.581400+0000","last_undegraded":"2024-03-28T16:49:37.581400+0000","last_fullsized":"2024-03-28T16:49:37.581400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:55:50.278254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591305+0000","last_change":"2024-03-28T16:48:54.666472+0000","last_active":"2024-03-28T16:49:37.591305+0000","last_peered":"2024-03-28T16:49:37.591305+0000","last_clean":"2024-03-28T16:49:37.591305+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:49:37.591305+0000","last_undegraded":"2024-03-28T16:49:37.591305+0000","last_fullsized":"2024-03-28T16:49:37.591305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587375+0000","last_change":"2024-03-28T16:48:58.688005+0000","last_active":"2024-03-28T16:49:37.587375+0000","last_peered":"2024-03-28T16:49:37.587375+0000","last_clean":"2024-03-28T16:49:37.587375+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:49:37.587375+0000","last_undegraded":"2024-03-28T16:49:37.587375+0000","last_fullsized":"2024-03-28T16:49:37.587375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:06.936540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'13","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595894+0000","last_change":"2024-03-28T16:49:00.713939+0000","last_active":"2024-03-28T16:49:37.595894+0000","last_peered":"2024-03-28T16:49:37.595894+0000","last_clean":"2024-03-28T16:49:37.595894+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:49:37.595894+0000","last_undegraded":"2024-03-28T16:49:37.595894+0000","last_fullsized":"2024-03-28T16:49:37.595894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:00:16.840810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588068+0000","last_change":"2024-03-28T16:49:02.741194+0000","last_active":"2024-03-28T16:49:37.588068+0000","last_peered":"2024-03-28T16:49:37.588068+0000","last_clean":"2024-03-28T16:49:37.588068+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:49:37.588068+0000","last_undegraded":"2024-03-28T16:49:37.588068+0000","last_fullsized":"2024-03-28T16:49:37.588068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:56.754215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585813+0000","last_change":"2024-03-28T16:49:04.741719+0000","last_active":"2024-03-28T16:49:37.585813+0000","last_peered":"2024-03-28T16:49:37.585813+0000","last_clean":"2024-03-28T16:49:37.585813+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:49:37.585813+0000","last_undegraded":"2024-03-28T16:49:37.585813+0000","last_fullsized":"2024-03-28T16:49:37.585813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:44.134615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555275+0000","last_change":"2024-03-28T16:48:56.687243+0000","last_active":"2024-03-28T16:49:40.555275+0000","last_peered":"2024-03-28T16:49:40.555275+0000","last_clean":"2024-03-28T16:49:40.555275+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:49:40.555275+0000","last_undegraded":"2024-03-28T16:49:40.555275+0000","last_fullsized":"2024-03-28T16:49:40.555275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596299+0000","last_change":"2024-03-28T16:48:54.657105+0000","last_active":"2024-03-28T16:49:37.596299+0000","last_peered":"2024-03-28T16:49:37.596299+0000","last_clean":"2024-03-28T16:49:37.596299+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:49:37.596299+0000","last_undegraded":"2024-03-28T16:49:37.596299+0000","last_fullsized":"2024-03-28T16:49:37.596299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586158+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:49:37.586158+0000","last_peered":"2024-03-28T16:49:37.586158+0000","last_clean":"2024-03-28T16:49:37.586158+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:49:37.586158+0000","last_undegraded":"2024-03-28T16:49:37.586158+0000","last_fullsized":"2024-03-28T16:49:37.586158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588917+0000","last_change":"2024-03-28T16:48:58.690031+0000","last_active":"2024-03-28T16:49:37.588917+0000","last_peered":"2024-03-28T16:49:37.588917+0000","last_clean":"2024-03-28T16:49:37.588917+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:49:37.588917+0000","last_undegraded":"2024-03-28T16:49:37.588917+0000","last_fullsized":"2024-03-28T16:49:37.588917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:29.762391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'57","reported_seq":115,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.796013+0000","last_change":"2024-03-28T16:49:00.706797+0000","last_active":"2024-03-28T16:50:28.796013+0000","last_peered":"2024-03-28T16:50:28.796013+0000","last_clean":"2024-03-28T16:50:28.796013+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:28.796013+0000","last_undegraded":"2024-03-28T16:50:28.796013+0000","last_fullsized":"2024-03-28T16:50:28.796013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:30.719266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590150+0000","last_change":"2024-03-28T16:49:02.720800+0000","last_active":"2024-03-28T16:49:37.590150+0000","last_peered":"2024-03-28T16:49:37.590150+0000","last_clean":"2024-03-28T16:49:37.590150+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:49:37.590150+0000","last_undegraded":"2024-03-28T16:49:37.590150+0000","last_fullsized":"2024-03-28T16:49:37.590150+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:07.770684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'9","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.211945+0000","last_change":"2024-03-28T16:49:04.741551+0000","last_active":"2024-03-28T16:50:30.211945+0000","last_peered":"2024-03-28T16:50:30.211945+0000","last_clean":"2024-03-28T16:50:30.211945+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:50:30.211945+0000","last_undegraded":"2024-03-28T16:50:30.211945+0000","last_fullsized":"2024-03-28T16:50:30.211945+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:55.072698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190216+0000","last_change":"2024-03-28T16:48:56.683849+0000","last_active":"2024-03-28T16:50:23.190216+0000","last_peered":"2024-03-28T16:50:23.190216+0000","last_clean":"2024-03-28T16:50:23.190216+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:50:23.190216+0000","last_undegraded":"2024-03-28T16:50:23.190216+0000","last_fullsized":"2024-03-28T16:50:23.190216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:57.646185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586877+0000","last_change":"2024-03-28T16:48:54.663076+0000","last_active":"2024-03-28T16:49:37.586877+0000","last_peered":"2024-03-28T16:49:37.586877+0000","last_clean":"2024-03-28T16:49:37.586877+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:49:37.586877+0000","last_undegraded":"2024-03-28T16:49:37.586877+0000","last_fullsized":"2024-03-28T16:49:37.586877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592775+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:49:37.592775+0000","last_peered":"2024-03-28T16:49:37.592775+0000","last_clean":"2024-03-28T16:49:37.592775+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:49:37.592775+0000","last_undegraded":"2024-03-28T16:49:37.592775+0000","last_fullsized":"2024-03-28T16:49:37.592775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:37.509224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582103+0000","last_change":"2024-03-28T16:48:58.683037+0000","last_active":"2024-03-28T16:49:37.582103+0000","last_peered":"2024-03-28T16:49:37.582103+0000","last_clean":"2024-03-28T16:49:37.582103+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:49:37.582103+0000","last_undegraded":"2024-03-28T16:49:37.582103+0000","last_fullsized":"2024-03-28T16:49:37.582103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:16.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'14","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595604+0000","last_change":"2024-03-28T16:49:00.720928+0000","last_active":"2024-03-28T16:49:37.595604+0000","last_peered":"2024-03-28T16:49:37.595604+0000","last_clean":"2024-03-28T16:49:37.595604+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:49:37.595604+0000","last_undegraded":"2024-03-28T16:49:37.595604+0000","last_fullsized":"2024-03-28T16:49:37.595604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:03.955318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'246","reported_seq":291,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.853994+0000","last_change":"2024-03-28T16:49:02.715814+0000","last_active":"2024-03-28T16:50:33.853994+0000","last_peered":"2024-03-28T16:50:33.853994+0000","last_clean":"2024-03-28T16:50:33.853994+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:33.853994+0000","last_undegraded":"2024-03-28T16:50:33.853994+0000","last_fullsized":"2024-03-28T16:50:33.853994+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.177166+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:23.177166+0000","last_peered":"2024-03-28T16:50:23.177166+0000","last_clean":"2024-03-28T16:50:23.177166+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:23.177166+0000","last_undegraded":"2024-03-28T16:50:23.177166+0000","last_fullsized":"2024-03-28T16:50:23.177166+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:14:04.371488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189696+0000","last_change":"2024-03-28T16:48:56.693531+0000","last_active":"2024-03-28T16:50:23.189696+0000","last_peered":"2024-03-28T16:50:23.189696+0000","last_clean":"2024-03-28T16:50:23.189696+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:50:23.189696+0000","last_undegraded":"2024-03-28T16:50:23.189696+0000","last_fullsized":"2024-03-28T16:50:23.189696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:22.327091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.850113+0000","last_change":"2024-03-28T16:48:54.662217+0000","last_active":"2024-03-28T16:49:39.850113+0000","last_peered":"2024-03-28T16:49:39.850113+0000","last_clean":"2024-03-28T16:49:39.850113+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:49:39.850113+0000","last_undegraded":"2024-03-28T16:49:39.850113+0000","last_fullsized":"2024-03-28T16:49:39.850113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:59.868215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591024+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:49:37.591024+0000","last_peered":"2024-03-28T16:49:37.591024+0000","last_clean":"2024-03-28T16:49:37.591024+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:49:37.591024+0000","last_undegraded":"2024-03-28T16:49:37.591024+0000","last_fullsized":"2024-03-28T16:49:37.591024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593505+0000","last_change":"2024-03-28T16:48:58.684369+0000","last_active":"2024-03-28T16:49:37.593505+0000","last_peered":"2024-03-28T16:49:37.593505+0000","last_clean":"2024-03-28T16:49:37.593505+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:49:37.593505+0000","last_undegraded":"2024-03-28T16:49:37.593505+0000","last_fullsized":"2024-03-28T16:49:37.593505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:58.333522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589611+0000","last_change":"2024-03-28T16:49:00.721989+0000","last_active":"2024-03-28T16:49:37.589611+0000","last_peered":"2024-03-28T16:49:37.589611+0000","last_clean":"2024-03-28T16:49:37.589611+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:49:37.589611+0000","last_undegraded":"2024-03-28T16:49:37.589611+0000","last_fullsized":"2024-03-28T16:49:37.589611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:27.568851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'246","reported_seq":324,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854449+0000","last_change":"2024-03-28T16:49:02.730370+0000","last_active":"2024-03-28T16:50:33.854449+0000","last_peered":"2024-03-28T16:50:33.854449+0000","last_clean":"2024-03-28T16:50:33.854449+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:33.854449+0000","last_undegraded":"2024-03-28T16:50:33.854449+0000","last_fullsized":"2024-03-28T16:50:33.854449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585106+0000","last_change":"2024-03-28T16:49:04.739259+0000","last_active":"2024-03-28T16:49:37.585106+0000","last_peered":"2024-03-28T16:49:37.585106+0000","last_clean":"2024-03-28T16:49:37.585106+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:49:37.585106+0000","last_undegraded":"2024-03-28T16:49:37.585106+0000","last_fullsized":"2024-03-28T16:49:37.585106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:50:01.255390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555184+0000","last_change":"2024-03-28T16:48:56.682608+0000","last_active":"2024-03-28T16:49:40.555184+0000","last_peered":"2024-03-28T16:49:40.555184+0000","last_clean":"2024-03-28T16:49:40.555184+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:49:40.555184+0000","last_undegraded":"2024-03-28T16:49:40.555184+0000","last_fullsized":"2024-03-28T16:49:40.555184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.855204+0000","last_change":"2024-03-28T16:48:54.675316+0000","last_active":"2024-03-28T16:49:39.855204+0000","last_peered":"2024-03-28T16:49:39.855204+0000","last_clean":"2024-03-28T16:49:39.855204+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:49:39.855204+0000","last_undegraded":"2024-03-28T16:49:39.855204+0000","last_fullsized":"2024-03-28T16:49:39.855204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588286+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:49:37.588286+0000","last_peered":"2024-03-28T16:49:37.588286+0000","last_clean":"2024-03-28T16:49:37.588286+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:49:37.588286+0000","last_undegraded":"2024-03-28T16:49:37.588286+0000","last_fullsized":"2024-03-28T16:49:37.588286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:45:04.316499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601449+0000","last_change":"2024-03-28T16:48:58.681176+0000","last_active":"2024-03-28T16:49:37.601449+0000","last_peered":"2024-03-28T16:49:37.601449+0000","last_clean":"2024-03-28T16:49:37.601449+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:49:37.601449+0000","last_undegraded":"2024-03-28T16:49:37.601449+0000","last_fullsized":"2024-03-28T16:49:37.601449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:49.369023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'76","reported_seq":147,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.790186+0000","last_change":"2024-03-28T16:49:00.709967+0000","last_active":"2024-03-28T16:50:28.790186+0000","last_peered":"2024-03-28T16:50:28.790186+0000","last_clean":"2024-03-28T16:50:28.790186+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:28.790186+0000","last_undegraded":"2024-03-28T16:50:28.790186+0000","last_fullsized":"2024-03-28T16:50:28.790186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:08.949965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585774+0000","last_change":"2024-03-28T16:49:02.715279+0000","last_active":"2024-03-28T16:49:37.585774+0000","last_peered":"2024-03-28T16:49:37.585774+0000","last_clean":"2024-03-28T16:49:37.585774+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:49:37.585774+0000","last_undegraded":"2024-03-28T16:49:37.585774+0000","last_fullsized":"2024-03-28T16:49:37.585774+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:02.364003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594229+0000","last_change":"2024-03-28T16:49:04.754190+0000","last_active":"2024-03-28T16:49:37.594229+0000","last_peered":"2024-03-28T16:49:37.594229+0000","last_clean":"2024-03-28T16:49:37.594229+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:49:37.594229+0000","last_undegraded":"2024-03-28T16:49:37.594229+0000","last_fullsized":"2024-03-28T16:49:37.594229+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:26.719819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594252+0000","last_change":"2024-03-28T16:49:04.740956+0000","last_active":"2024-03-28T16:49:37.594252+0000","last_peered":"2024-03-28T16:49:37.594252+0000","last_clean":"2024-03-28T16:49:37.594252+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:49:37.594252+0000","last_undegraded":"2024-03-28T16:49:37.594252+0000","last_fullsized":"2024-03-28T16:49:37.594252+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:55.907917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190179+0000","last_change":"2024-03-28T16:48:56.680906+0000","last_active":"2024-03-28T16:50:23.190179+0000","last_peered":"2024-03-28T16:50:23.190179+0000","last_clean":"2024-03-28T16:50:23.190179+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:50:23.190179+0000","last_undegraded":"2024-03-28T16:50:23.190179+0000","last_fullsized":"2024-03-28T16:50:23.190179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:05.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587082+0000","last_change":"2024-03-28T16:48:54.663088+0000","last_active":"2024-03-28T16:49:37.587082+0000","last_peered":"2024-03-28T16:49:37.587082+0000","last_clean":"2024-03-28T16:49:37.587082+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:49:37.587082+0000","last_undegraded":"2024-03-28T16:49:37.587082+0000","last_fullsized":"2024-03-28T16:49:37.587082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.659538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.606876+0000","last_change":"2024-03-28T16:49:37.606876+0000","last_active":"2024-03-28T16:49:37.606876+0000","last_peered":"2024-03-28T16:49:37.606876+0000","last_clean":"2024-03-28T16:49:37.606876+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:49:37.606876+0000","last_undegraded":"2024-03-28T16:49:37.606876+0000","last_fullsized":"2024-03-28T16:49:37.606876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:25.555426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'31","reported_seq":73,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.157002+0000","last_change":"2024-03-28T16:49:00.725615+0000","last_active":"2024-03-28T16:50:23.157002+0000","last_peered":"2024-03-28T16:50:23.157002+0000","last_clean":"2024-03-28T16:50:23.157002+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:50:23.157002+0000","last_undegraded":"2024-03-28T16:50:23.157002+0000","last_fullsized":"2024-03-28T16:50:23.157002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:56.763507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582514+0000","last_change":"2024-03-28T16:49:02.739259+0000","last_active":"2024-03-28T16:49:37.582514+0000","last_peered":"2024-03-28T16:49:37.582514+0000","last_clean":"2024-03-28T16:49:37.582514+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:49:37.582514+0000","last_undegraded":"2024-03-28T16:49:37.582514+0000","last_fullsized":"2024-03-28T16:49:37.582514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:33.761245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'1","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586987+0000","last_change":"2024-03-28T16:49:04.723410+0000","last_active":"2024-03-28T16:49:37.586987+0000","last_peered":"2024-03-28T16:49:37.586987+0000","last_clean":"2024-03-28T16:49:37.586987+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:49:37.586987+0000","last_undegraded":"2024-03-28T16:49:37.586987+0000","last_fullsized":"2024-03-28T16:49:37.586987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:56:19.278765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.603811+0000","last_change":"2024-03-28T16:49:37.603811+0000","last_active":"2024-03-28T16:49:37.603811+0000","last_peered":"2024-03-28T16:49:37.603811+0000","last_clean":"2024-03-28T16:49:37.603811+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:49:37.603811+0000","last_undegraded":"2024-03-28T16:49:37.603811+0000","last_fullsized":"2024-03-28T16:49:37.603811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:44.503106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.833936+0000","last_change":"2024-03-28T16:48:54.657586+0000","last_active":"2024-03-28T16:50:28.833936+0000","last_peered":"2024-03-28T16:50:28.833936+0000","last_clean":"2024-03-28T16:50:28.833936+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:50:28.833936+0000","last_undegraded":"2024-03-28T16:50:28.833936+0000","last_fullsized":"2024-03-28T16:50:28.833936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594338+0000","last_change":"2024-03-28T16:48:58.696407+0000","last_active":"2024-03-28T16:49:37.594338+0000","last_peered":"2024-03-28T16:49:37.594338+0000","last_clean":"2024-03-28T16:49:37.594338+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:49:37.594338+0000","last_undegraded":"2024-03-28T16:49:37.594338+0000","last_fullsized":"2024-03-28T16:49:37.594338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:45.492142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:15.707942+0000","last_change":"2024-03-28T16:49:00.701020+0000","last_active":"2024-03-28T16:50:15.707942+0000","last_peered":"2024-03-28T16:50:15.707942+0000","last_clean":"2024-03-28T16:50:15.707942+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:50:15.707942+0000","last_undegraded":"2024-03-28T16:50:15.707942+0000","last_fullsized":"2024-03-28T16:50:15.707942+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:42.652898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589597+0000","last_change":"2024-03-28T16:49:02.726748+0000","last_active":"2024-03-28T16:49:37.589597+0000","last_peered":"2024-03-28T16:49:37.589597+0000","last_clean":"2024-03-28T16:49:37.589597+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:49:37.589597+0000","last_undegraded":"2024-03-28T16:49:37.589597+0000","last_fullsized":"2024-03-28T16:49:37.589597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:39.742691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586140+0000","last_change":"2024-03-28T16:49:04.744040+0000","last_active":"2024-03-28T16:49:37.586140+0000","last_peered":"2024-03-28T16:49:37.586140+0000","last_clean":"2024-03-28T16:49:37.586140+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:49:37.586140+0000","last_undegraded":"2024-03-28T16:49:37.586140+0000","last_fullsized":"2024-03-28T16:49:37.586140+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:51:53.769493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.730425+0000","last_change":"2024-03-28T16:48:56.676841+0000","last_active":"2024-03-28T16:50:22.730425+0000","last_peered":"2024-03-28T16:50:22.730425+0000","last_clean":"2024-03-28T16:50:22.730425+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:50:22.730425+0000","last_undegraded":"2024-03-28T16:50:22.730425+0000","last_fullsized":"2024-03-28T16:50:22.730425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.183347+0000","last_change":"2024-03-28T16:48:54.659093+0000","last_active":"2024-03-28T16:50:23.183347+0000","last_peered":"2024-03-28T16:50:23.183347+0000","last_clean":"2024-03-28T16:50:23.183347+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:50:23.183347+0000","last_undegraded":"2024-03-28T16:50:23.183347+0000","last_fullsized":"2024-03-28T16:50:23.183347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'2","reported_seq":196,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.365615+0000","last_change":"2024-03-28T16:48:58.690200+0000","last_active":"2024-03-28T16:50:30.365615+0000","last_peered":"2024-03-28T16:50:30.365615+0000","last_clean":"2024-03-28T16:50:30.365615+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:50:30.365615+0000","last_undegraded":"2024-03-28T16:50:30.365615+0000","last_fullsized":"2024-03-28T16:50:30.365615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:59.661181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":253,"num_read_kb":252,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'35","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.577197+0000","last_change":"2024-03-28T16:49:00.718425+0000","last_active":"2024-03-28T16:50:28.577197+0000","last_peered":"2024-03-28T16:50:28.577197+0000","last_clean":"2024-03-28T16:50:28.577197+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:28.577197+0000","last_undegraded":"2024-03-28T16:50:28.577197+0000","last_fullsized":"2024-03-28T16:50:28.577197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:37.725635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":50,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585222+0000","last_change":"2024-03-28T16:49:02.715466+0000","last_active":"2024-03-28T16:49:37.585222+0000","last_peered":"2024-03-28T16:49:37.585222+0000","last_clean":"2024-03-28T16:49:37.585222+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:49:37.585222+0000","last_undegraded":"2024-03-28T16:49:37.585222+0000","last_fullsized":"2024-03-28T16:49:37.585222+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581973+0000","last_change":"2024-03-28T16:49:04.752570+0000","last_active":"2024-03-28T16:49:37.581973+0000","last_peered":"2024-03-28T16:49:37.581973+0000","last_clean":"2024-03-28T16:49:37.581973+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:49:37.581973+0000","last_undegraded":"2024-03-28T16:49:37.581973+0000","last_fullsized":"2024-03-28T16:49:37.581973+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:04:11.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189947+0000","last_change":"2024-03-28T16:48:56.683249+0000","last_active":"2024-03-28T16:50:23.189947+0000","last_peered":"2024-03-28T16:50:23.189947+0000","last_clean":"2024-03-28T16:50:23.189947+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:50:23.189947+0000","last_undegraded":"2024-03-28T16:50:23.189947+0000","last_fullsized":"2024-03-28T16:50:23.189947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:23.009747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602071+0000","last_change":"2024-03-28T16:48:54.674928+0000","last_active":"2024-03-28T16:49:37.602071+0000","last_peered":"2024-03-28T16:49:37.602071+0000","last_clean":"2024-03-28T16:49:37.602071+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:49:37.602071+0000","last_undegraded":"2024-03-28T16:49:37.602071+0000","last_fullsized":"2024-03-28T16:49:37.602071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593497+0000","last_change":"2024-03-28T16:48:58.680113+0000","last_active":"2024-03-28T16:49:37.593497+0000","last_peered":"2024-03-28T16:49:37.593497+0000","last_clean":"2024-03-28T16:49:37.593497+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:49:37.593497+0000","last_undegraded":"2024-03-28T16:49:37.593497+0000","last_fullsized":"2024-03-28T16:49:37.593497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:59.079328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.137620+0000","last_change":"2024-03-28T16:49:00.696647+0000","last_active":"2024-03-28T16:50:23.137620+0000","last_peered":"2024-03-28T16:50:23.137620+0000","last_clean":"2024-03-28T16:50:23.137620+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:50:23.137620+0000","last_undegraded":"2024-03-28T16:50:23.137620+0000","last_fullsized":"2024-03-28T16:50:23.137620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:15.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594972+0000","last_change":"2024-03-28T16:49:02.726334+0000","last_active":"2024-03-28T16:49:37.594972+0000","last_peered":"2024-03-28T16:49:37.594972+0000","last_clean":"2024-03-28T16:49:37.594972+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:49:37.594972+0000","last_undegraded":"2024-03-28T16:49:37.594972+0000","last_fullsized":"2024-03-28T16:49:37.594972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:58.829170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600011+0000","last_change":"2024-03-28T16:48:56.675359+0000","last_active":"2024-03-28T16:49:37.600011+0000","last_peered":"2024-03-28T16:49:37.600011+0000","last_clean":"2024-03-28T16:49:37.600011+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:49:37.600011+0000","last_undegraded":"2024-03-28T16:49:37.600011+0000","last_fullsized":"2024-03-28T16:49:37.600011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:35.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601915+0000","last_change":"2024-03-28T16:49:37.600567+0000","last_active":"2024-03-28T16:49:37.601915+0000","last_peered":"2024-03-28T16:49:37.601915+0000","last_clean":"2024-03-28T16:49:37.601915+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:49:37.601915+0000","last_undegraded":"2024-03-28T16:49:37.601915+0000","last_fullsized":"2024-03-28T16:49:37.601915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:40:41.090124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591166+0000","last_change":"2024-03-28T16:48:58.687213+0000","last_active":"2024-03-28T16:49:37.591166+0000","last_peered":"2024-03-28T16:49:37.591166+0000","last_clean":"2024-03-28T16:49:37.591166+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:49:37.591166+0000","last_undegraded":"2024-03-28T16:49:37.591166+0000","last_fullsized":"2024-03-28T16:49:37.591166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:13.652414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'51","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.785801+0000","last_change":"2024-03-28T16:49:00.716007+0000","last_active":"2024-03-28T16:50:28.785801+0000","last_peered":"2024-03-28T16:50:28.785801+0000","last_clean":"2024-03-28T16:50:28.785801+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:28.785801+0000","last_undegraded":"2024-03-28T16:50:28.785801+0000","last_fullsized":"2024-03-28T16:50:28.785801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:56.812114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595955+0000","last_change":"2024-03-28T16:49:02.730156+0000","last_active":"2024-03-28T16:49:37.595955+0000","last_peered":"2024-03-28T16:49:37.595955+0000","last_clean":"2024-03-28T16:49:37.595955+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:49:37.595955+0000","last_undegraded":"2024-03-28T16:49:37.595955+0000","last_fullsized":"2024-03-28T16:49:37.595955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:15.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'11","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.213440+0000","last_change":"2024-03-28T16:49:04.752525+0000","last_active":"2024-03-28T16:50:23.213440+0000","last_peered":"2024-03-28T16:50:23.213440+0000","last_clean":"2024-03-28T16:50:23.213440+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:50:23.213440+0000","last_undegraded":"2024-03-28T16:50:23.213440+0000","last_fullsized":"2024-03-28T16:50:23.213440+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:18.858161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588098+0000","last_change":"2024-03-28T16:48:56.697482+0000","last_active":"2024-03-28T16:49:37.588098+0000","last_peered":"2024-03-28T16:49:37.588098+0000","last_clean":"2024-03-28T16:49:37.588098+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:49:37.588098+0000","last_undegraded":"2024-03-28T16:49:37.588098+0000","last_fullsized":"2024-03-28T16:49:37.588098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:02.641119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584038+0000","last_change":"2024-03-28T16:48:54.668040+0000","last_active":"2024-03-28T16:49:37.584038+0000","last_peered":"2024-03-28T16:49:37.584038+0000","last_clean":"2024-03-28T16:49:37.584038+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:49:37.584038+0000","last_undegraded":"2024-03-28T16:49:37.584038+0000","last_fullsized":"2024-03-28T16:49:37.584038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596417+0000","last_change":"2024-03-28T16:48:58.686377+0000","last_active":"2024-03-28T16:49:37.596417+0000","last_peered":"2024-03-28T16:49:37.596417+0000","last_clean":"2024-03-28T16:49:37.596417+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:49:37.596417+0000","last_undegraded":"2024-03-28T16:49:37.596417+0000","last_fullsized":"2024-03-28T16:49:37.596417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:01.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.138592+0000","last_change":"2024-03-28T16:49:00.716668+0000","last_active":"2024-03-28T16:50:23.138592+0000","last_peered":"2024-03-28T16:50:23.138592+0000","last_clean":"2024-03-28T16:50:23.138592+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:23.138592+0000","last_undegraded":"2024-03-28T16:50:23.138592+0000","last_fullsized":"2024-03-28T16:50:23.138592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:48.604008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587828+0000","last_change":"2024-03-28T16:49:02.731621+0000","last_active":"2024-03-28T16:49:37.587828+0000","last_peered":"2024-03-28T16:49:37.587828+0000","last_clean":"2024-03-28T16:49:37.587828+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:49:37.587828+0000","last_undegraded":"2024-03-28T16:49:37.587828+0000","last_fullsized":"2024-03-28T16:49:37.587828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.557447+0000","last_change":"2024-03-28T16:49:04.754981+0000","last_active":"2024-03-28T16:49:40.557447+0000","last_peered":"2024-03-28T16:49:40.557447+0000","last_clean":"2024-03-28T16:49:40.557447+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:49:40.557447+0000","last_undegraded":"2024-03-28T16:49:40.557447+0000","last_fullsized":"2024-03-28T16:49:40.557447+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:33:25.577689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'3","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189773+0000","last_change":"2024-03-28T16:48:56.694984+0000","last_active":"2024-03-28T16:50:23.189773+0000","last_peered":"2024-03-28T16:50:23.189773+0000","last_clean":"2024-03-28T16:50:23.189773+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:50:23.189773+0000","last_undegraded":"2024-03-28T16:50:23.189773+0000","last_fullsized":"2024-03-28T16:50:23.189773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:09.365959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585942+0000","last_change":"2024-03-28T16:48:54.654527+0000","last_active":"2024-03-28T16:49:37.585942+0000","last_peered":"2024-03-28T16:49:37.585942+0000","last_clean":"2024-03-28T16:49:37.585942+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:49:37.585942+0000","last_undegraded":"2024-03-28T16:49:37.585942+0000","last_fullsized":"2024-03-28T16:49:37.585942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610755+0000","last_change":"2024-03-28T16:49:37.610755+0000","last_active":"2024-03-28T16:49:37.610755+0000","last_peered":"2024-03-28T16:49:37.610755+0000","last_clean":"2024-03-28T16:49:37.610755+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:49:37.610755+0000","last_undegraded":"2024-03-28T16:49:37.610755+0000","last_fullsized":"2024-03-28T16:49:37.610755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:24.156961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'32","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.163251+0000","last_change":"2024-03-28T16:49:00.706941+0000","last_active":"2024-03-28T16:50:23.163251+0000","last_peered":"2024-03-28T16:50:23.163251+0000","last_clean":"2024-03-28T16:50:23.163251+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:23.163251+0000","last_undegraded":"2024-03-28T16:50:23.163251+0000","last_fullsized":"2024-03-28T16:50:23.163251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:05.753628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":46,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587449+0000","last_change":"2024-03-28T16:49:02.734090+0000","last_active":"2024-03-28T16:49:37.587449+0000","last_peered":"2024-03-28T16:49:37.587449+0000","last_clean":"2024-03-28T16:49:37.587449+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:49:37.587449+0000","last_undegraded":"2024-03-28T16:49:37.587449+0000","last_fullsized":"2024-03-28T16:49:37.587449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:39.107257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.835263+0000","last_change":"2024-03-28T16:49:04.748086+0000","last_active":"2024-03-28T16:49:39.835263+0000","last_peered":"2024-03-28T16:49:39.835263+0000","last_clean":"2024-03-28T16:49:39.835263+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:49:39.835263+0000","last_undegraded":"2024-03-28T16:49:39.835263+0000","last_fullsized":"2024-03-28T16:49:39.835263+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:44.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588233+0000","last_change":"2024-03-28T16:48:56.682850+0000","last_active":"2024-03-28T16:49:37.588233+0000","last_peered":"2024-03-28T16:49:37.588233+0000","last_clean":"2024-03-28T16:49:37.588233+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:49:37.588233+0000","last_undegraded":"2024-03-28T16:49:37.588233+0000","last_fullsized":"2024-03-28T16:49:37.588233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592598+0000","last_change":"2024-03-28T16:48:54.674069+0000","last_active":"2024-03-28T16:49:37.592598+0000","last_peered":"2024-03-28T16:49:37.592598+0000","last_clean":"2024-03-28T16:49:37.592598+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:49:37.592598+0000","last_undegraded":"2024-03-28T16:49:37.592598+0000","last_fullsized":"2024-03-28T16:49:37.592598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.406243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587070+0000","last_change":"2024-03-28T16:48:58.681774+0000","last_active":"2024-03-28T16:49:37.587070+0000","last_peered":"2024-03-28T16:49:37.587070+0000","last_clean":"2024-03-28T16:49:37.587070+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:49:37.587070+0000","last_undegraded":"2024-03-28T16:49:37.587070+0000","last_fullsized":"2024-03-28T16:49:37.587070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:24.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589169+0000","last_change":"2024-03-28T16:49:00.720072+0000","last_active":"2024-03-28T16:49:37.589169+0000","last_peered":"2024-03-28T16:49:37.589169+0000","last_clean":"2024-03-28T16:49:37.589169+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:49:37.589169+0000","last_undegraded":"2024-03-28T16:49:37.589169+0000","last_fullsized":"2024-03-28T16:49:37.589169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:22:49.287785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'246","reported_seq":297,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854122+0000","last_change":"2024-03-28T16:49:02.734898+0000","last_active":"2024-03-28T16:50:33.854122+0000","last_peered":"2024-03-28T16:50:33.854122+0000","last_clean":"2024-03-28T16:50:33.854122+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:33.854122+0000","last_undegraded":"2024-03-28T16:50:33.854122+0000","last_fullsized":"2024-03-28T16:50:33.854122+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:05.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.579919+0000","last_change":"2024-03-28T16:49:04.748224+0000","last_active":"2024-03-28T16:49:37.579919+0000","last_peered":"2024-03-28T16:49:37.579919+0000","last_clean":"2024-03-28T16:49:37.579919+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:49:37.579919+0000","last_undegraded":"2024-03-28T16:49:37.579919+0000","last_fullsized":"2024-03-28T16:49:37.579919+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:26.578458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'5","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370813+0000","last_change":"2024-03-28T16:48:56.685308+0000","last_active":"2024-03-28T16:50:22.370813+0000","last_peered":"2024-03-28T16:50:22.370813+0000","last_clean":"2024-03-28T16:50:22.370813+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:50:22.370813+0000","last_undegraded":"2024-03-28T16:50:22.370813+0000","last_fullsized":"2024-03-28T16:50:22.370813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:39.524773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586373+0000","last_change":"2024-03-28T16:48:54.667648+0000","last_active":"2024-03-28T16:49:37.586373+0000","last_peered":"2024-03-28T16:49:37.586373+0000","last_clean":"2024-03-28T16:49:37.586373+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:49:37.586373+0000","last_undegraded":"2024-03-28T16:49:37.586373+0000","last_fullsized":"2024-03-28T16:49:37.586373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598430+0000","last_change":"2024-03-28T16:48:58.684892+0000","last_active":"2024-03-28T16:49:37.598430+0000","last_peered":"2024-03-28T16:49:37.598430+0000","last_clean":"2024-03-28T16:49:37.598430+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:49:37.598430+0000","last_undegraded":"2024-03-28T16:49:37.598430+0000","last_fullsized":"2024-03-28T16:49:37.598430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:34.392979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.126243+0000","last_change":"2024-03-28T16:49:00.722124+0000","last_active":"2024-03-28T16:50:23.126243+0000","last_peered":"2024-03-28T16:50:23.126243+0000","last_clean":"2024-03-28T16:50:23.126243+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:23.126243+0000","last_undegraded":"2024-03-28T16:50:23.126243+0000","last_fullsized":"2024-03-28T16:50:23.126243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:39.985393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584276+0000","last_change":"2024-03-28T16:49:02.715093+0000","last_active":"2024-03-28T16:49:37.584276+0000","last_peered":"2024-03-28T16:49:37.584276+0000","last_clean":"2024-03-28T16:49:37.584276+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:49:37.584276+0000","last_undegraded":"2024-03-28T16:49:37.584276+0000","last_fullsized":"2024-03-28T16:49:37.584276+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:26.448039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601171+0000","last_change":"2024-03-28T16:49:04.745332+0000","last_active":"2024-03-28T16:49:37.601171+0000","last_peered":"2024-03-28T16:49:37.601171+0000","last_clean":"2024-03-28T16:49:37.601171+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:49:37.601171+0000","last_undegraded":"2024-03-28T16:49:37.601171+0000","last_fullsized":"2024-03-28T16:49:37.601171+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:11.521117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369950+0000","last_change":"2024-03-28T16:48:56.690217+0000","last_active":"2024-03-28T16:50:22.369950+0000","last_peered":"2024-03-28T16:50:22.369950+0000","last_clean":"2024-03-28T16:50:22.369950+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:50:22.369950+0000","last_undegraded":"2024-03-28T16:50:22.369950+0000","last_fullsized":"2024-03-28T16:50:22.369950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588654+0000","last_change":"2024-03-28T16:48:54.669871+0000","last_active":"2024-03-28T16:49:37.588654+0000","last_peered":"2024-03-28T16:49:37.588654+0000","last_clean":"2024-03-28T16:49:37.588654+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:49:37.588654+0000","last_undegraded":"2024-03-28T16:49:37.588654+0000","last_fullsized":"2024-03-28T16:49:37.588654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:58.495034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594708+0000","last_change":"2024-03-28T16:48:58.692783+0000","last_active":"2024-03-28T16:49:37.594708+0000","last_peered":"2024-03-28T16:49:37.594708+0000","last_clean":"2024-03-28T16:49:37.594708+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:49:37.594708+0000","last_undegraded":"2024-03-28T16:49:37.594708+0000","last_fullsized":"2024-03-28T16:49:37.594708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:20.188622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.128390+0000","last_change":"2024-03-28T16:49:00.706559+0000","last_active":"2024-03-28T16:50:23.128390+0000","last_peered":"2024-03-28T16:50:23.128390+0000","last_clean":"2024-03-28T16:50:23.128390+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:23.128390+0000","last_undegraded":"2024-03-28T16:50:23.128390+0000","last_fullsized":"2024-03-28T16:50:23.128390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:54.800679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581059+0000","last_change":"2024-03-28T16:49:02.739372+0000","last_active":"2024-03-28T16:49:37.581059+0000","last_peered":"2024-03-28T16:49:37.581059+0000","last_clean":"2024-03-28T16:49:37.581059+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:49:37.581059+0000","last_undegraded":"2024-03-28T16:49:37.581059+0000","last_fullsized":"2024-03-28T16:49:37.581059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:22:56.097286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582527+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:49:37.582527+0000","last_peered":"2024-03-28T16:49:37.582527+0000","last_clean":"2024-03-28T16:49:37.582527+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:49:37.582527+0000","last_undegraded":"2024-03-28T16:49:37.582527+0000","last_fullsized":"2024-03-28T16:49:37.582527+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:45:27.545220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593066+0000","last_change":"2024-03-28T16:48:56.666104+0000","last_active":"2024-03-28T16:49:37.593066+0000","last_peered":"2024-03-28T16:49:37.593066+0000","last_clean":"2024-03-28T16:49:37.593066+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:49:37.593066+0000","last_undegraded":"2024-03-28T16:49:37.593066+0000","last_fullsized":"2024-03-28T16:49:37.593066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:35:42.047564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588335+0000","last_change":"2024-03-28T16:48:54.665975+0000","last_active":"2024-03-28T16:49:37.588335+0000","last_peered":"2024-03-28T16:49:37.588335+0000","last_clean":"2024-03-28T16:49:37.588335+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:49:37.588335+0000","last_undegraded":"2024-03-28T16:49:37.588335+0000","last_fullsized":"2024-03-28T16:49:37.588335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585938+0000","last_change":"2024-03-28T16:48:58.688417+0000","last_active":"2024-03-28T16:49:37.585938+0000","last_peered":"2024-03-28T16:49:37.585938+0000","last_clean":"2024-03-28T16:49:37.585938+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:49:37.585938+0000","last_undegraded":"2024-03-28T16:49:37.585938+0000","last_fullsized":"2024-03-28T16:49:37.585938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:57.154239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'24","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.159574+0000","last_change":"2024-03-28T16:49:00.717188+0000","last_active":"2024-03-28T16:50:23.159574+0000","last_peered":"2024-03-28T16:50:23.159574+0000","last_clean":"2024-03-28T16:50:23.159574+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:23.159574+0000","last_undegraded":"2024-03-28T16:50:23.159574+0000","last_fullsized":"2024-03-28T16:50:23.159574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:40:19.790152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581817+0000","last_change":"2024-03-28T16:49:02.739568+0000","last_active":"2024-03-28T16:49:37.581817+0000","last_peered":"2024-03-28T16:49:37.581817+0000","last_clean":"2024-03-28T16:49:37.581817+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:49:37.581817+0000","last_undegraded":"2024-03-28T16:49:37.581817+0000","last_fullsized":"2024-03-28T16:49:37.581817+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:32.279507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587161+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:49:37.587161+0000","last_peered":"2024-03-28T16:49:37.587161+0000","last_clean":"2024-03-28T16:49:37.587161+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:49:37.587161+0000","last_undegraded":"2024-03-28T16:49:37.587161+0000","last_fullsized":"2024-03-28T16:49:37.587161+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:59.156256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'11","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854243+0000","last_change":"2024-03-28T16:48:56.678271+0000","last_active":"2024-03-28T16:50:28.854243+0000","last_peered":"2024-03-28T16:50:28.854243+0000","last_clean":"2024-03-28T16:50:28.854243+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:50:28.854243+0000","last_undegraded":"2024-03-28T16:50:28.854243+0000","last_fullsized":"2024-03-28T16:50:28.854243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:31.748236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602333+0000","last_change":"2024-03-28T16:48:54.655009+0000","last_active":"2024-03-28T16:49:37.602333+0000","last_peered":"2024-03-28T16:49:37.602333+0000","last_clean":"2024-03-28T16:49:37.602333+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:49:37.602333+0000","last_undegraded":"2024-03-28T16:49:37.602333+0000","last_fullsized":"2024-03-28T16:49:37.602333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'2","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595281+0000","last_change":"2024-03-28T16:48:58.697773+0000","last_active":"2024-03-28T16:49:37.595281+0000","last_peered":"2024-03-28T16:49:37.595281+0000","last_clean":"2024-03-28T16:49:37.595281+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:49:37.595281+0000","last_undegraded":"2024-03-28T16:49:37.595281+0000","last_fullsized":"2024-03-28T16:49:37.595281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:25.024742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'29","reported_seq":70,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.816906+0000","last_change":"2024-03-28T16:49:00.720335+0000","last_active":"2024-03-28T16:50:28.816906+0000","last_peered":"2024-03-28T16:50:28.816906+0000","last_clean":"2024-03-28T16:50:28.816906+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:50:28.816906+0000","last_undegraded":"2024-03-28T16:50:28.816906+0000","last_fullsized":"2024-03-28T16:50:28.816906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:53.514285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":26,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586838+0000","last_change":"2024-03-28T16:49:02.714786+0000","last_active":"2024-03-28T16:49:37.586838+0000","last_peered":"2024-03-28T16:49:37.586838+0000","last_clean":"2024-03-28T16:49:37.586838+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:49:37.586838+0000","last_undegraded":"2024-03-28T16:49:37.586838+0000","last_fullsized":"2024-03-28T16:49:37.586838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582310+0000","last_change":"2024-03-28T16:49:04.753745+0000","last_active":"2024-03-28T16:49:37.582310+0000","last_peered":"2024-03-28T16:49:37.582310+0000","last_clean":"2024-03-28T16:49:37.582310+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:49:37.582310+0000","last_undegraded":"2024-03-28T16:49:37.582310+0000","last_fullsized":"2024-03-28T16:49:37.582310+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:59.796190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.619610+0000","last_change":"2024-03-28T16:49:37.618664+0000","last_active":"2024-03-28T16:49:37.619610+0000","last_peered":"2024-03-28T16:49:37.619610+0000","last_clean":"2024-03-28T16:49:37.619610+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:49:37.619610+0000","last_undegraded":"2024-03-28T16:49:37.619610+0000","last_fullsized":"2024-03-28T16:49:37.619610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:36.076211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588848+0000","last_change":"2024-03-28T16:48:54.658051+0000","last_active":"2024-03-28T16:49:37.588848+0000","last_peered":"2024-03-28T16:49:37.588848+0000","last_clean":"2024-03-28T16:49:37.588848+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:49:37.588848+0000","last_undegraded":"2024-03-28T16:49:37.588848+0000","last_fullsized":"2024-03-28T16:49:37.588848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190542+0000","last_change":"2024-03-28T16:48:56.682367+0000","last_active":"2024-03-28T16:50:23.190542+0000","last_peered":"2024-03-28T16:50:23.190542+0000","last_clean":"2024-03-28T16:50:23.190542+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:50:23.190542+0000","last_undegraded":"2024-03-28T16:50:23.190542+0000","last_fullsized":"2024-03-28T16:50:23.190542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584176+0000","last_change":"2024-03-28T16:48:54.655298+0000","last_active":"2024-03-28T16:49:37.584176+0000","last_peered":"2024-03-28T16:49:37.584176+0000","last_clean":"2024-03-28T16:49:37.584176+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:49:37.584176+0000","last_undegraded":"2024-03-28T16:49:37.584176+0000","last_fullsized":"2024-03-28T16:49:37.584176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'3","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370059+0000","last_change":"2024-03-28T16:48:56.671798+0000","last_active":"2024-03-28T16:50:22.370059+0000","last_peered":"2024-03-28T16:50:22.370059+0000","last_clean":"2024-03-28T16:50:22.370059+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:50:22.370059+0000","last_undegraded":"2024-03-28T16:50:22.370059+0000","last_fullsized":"2024-03-28T16:50:22.370059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:59.926897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589042+0000","last_change":"2024-03-28T16:48:54.657491+0000","last_active":"2024-03-28T16:49:37.589042+0000","last_peered":"2024-03-28T16:49:37.589042+0000","last_clean":"2024-03-28T16:49:37.589042+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:49:37.589042+0000","last_undegraded":"2024-03-28T16:49:37.589042+0000","last_fullsized":"2024-03-28T16:49:37.589042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:40:58.160099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586819+0000","last_change":"2024-03-28T16:48:56.683483+0000","last_active":"2024-03-28T16:49:37.586819+0000","last_peered":"2024-03-28T16:49:37.586819+0000","last_clean":"2024-03-28T16:49:37.586819+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:49:37.586819+0000","last_undegraded":"2024-03-28T16:49:37.586819+0000","last_fullsized":"2024-03-28T16:49:37.586819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588791+0000","last_change":"2024-03-28T16:48:54.665811+0000","last_active":"2024-03-28T16:49:37.588791+0000","last_peered":"2024-03-28T16:49:37.588791+0000","last_clean":"2024-03-28T16:49:37.588791+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:49:37.588791+0000","last_undegraded":"2024-03-28T16:49:37.588791+0000","last_fullsized":"2024-03-28T16:49:37.588791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:38.364800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591495+0000","last_change":"2024-03-28T16:48:56.677917+0000","last_active":"2024-03-28T16:49:37.591495+0000","last_peered":"2024-03-28T16:49:37.591495+0000","last_clean":"2024-03-28T16:49:37.591495+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:49:37.591495+0000","last_undegraded":"2024-03-28T16:49:37.591495+0000","last_fullsized":"2024-03-28T16:49:37.591495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587233+0000","last_change":"2024-03-28T16:48:54.655391+0000","last_active":"2024-03-28T16:49:37.587233+0000","last_peered":"2024-03-28T16:49:37.587233+0000","last_clean":"2024-03-28T16:49:37.587233+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:49:37.587233+0000","last_undegraded":"2024-03-28T16:49:37.587233+0000","last_fullsized":"2024-03-28T16:49:37.587233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840006+0000","last_change":"2024-03-28T16:48:56.678419+0000","last_active":"2024-03-28T16:50:28.840006+0000","last_peered":"2024-03-28T16:50:28.840006+0000","last_clean":"2024-03-28T16:50:28.840006+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:50:28.840006+0000","last_undegraded":"2024-03-28T16:50:28.840006+0000","last_fullsized":"2024-03-28T16:50:28.840006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:03.478538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595229+0000","last_change":"2024-03-28T16:48:54.669640+0000","last_active":"2024-03-28T16:49:37.595229+0000","last_peered":"2024-03-28T16:49:37.595229+0000","last_clean":"2024-03-28T16:49:37.595229+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:49:37.595229+0000","last_undegraded":"2024-03-28T16:49:37.595229+0000","last_fullsized":"2024-03-28T16:49:37.595229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527969+0000","last_change":"2024-03-28T16:48:56.689880+0000","last_active":"2024-03-28T16:49:39.527969+0000","last_peered":"2024-03-28T16:49:39.527969+0000","last_clean":"2024-03-28T16:49:39.527969+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:49:39.527969+0000","last_undegraded":"2024-03-28T16:49:39.527969+0000","last_fullsized":"2024-03-28T16:49:39.527969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598088+0000","last_change":"2024-03-28T16:48:54.661594+0000","last_active":"2024-03-28T16:49:37.598088+0000","last_peered":"2024-03-28T16:49:37.598088+0000","last_clean":"2024-03-28T16:49:37.598088+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:49:37.598088+0000","last_undegraded":"2024-03-28T16:49:37.598088+0000","last_fullsized":"2024-03-28T16:49:37.598088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'5","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839320+0000","last_change":"2024-03-28T16:48:56.679523+0000","last_active":"2024-03-28T16:50:28.839320+0000","last_peered":"2024-03-28T16:50:28.839320+0000","last_clean":"2024-03-28T16:50:28.839320+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:50:28.839320+0000","last_undegraded":"2024-03-28T16:50:28.839320+0000","last_fullsized":"2024-03-28T16:50:28.839320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588478+0000","last_change":"2024-03-28T16:48:54.659870+0000","last_active":"2024-03-28T16:49:37.588478+0000","last_peered":"2024-03-28T16:49:37.588478+0000","last_clean":"2024-03-28T16:49:37.588478+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:49:37.588478+0000","last_undegraded":"2024-03-28T16:49:37.588478+0000","last_fullsized":"2024-03-28T16:49:37.588478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.526775+0000","last_change":"2024-03-28T16:48:56.687788+0000","last_active":"2024-03-28T16:49:39.526775+0000","last_peered":"2024-03-28T16:49:39.526775+0000","last_clean":"2024-03-28T16:49:39.526775+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:49:39.526775+0000","last_undegraded":"2024-03-28T16:49:39.526775+0000","last_fullsized":"2024-03-28T16:49:39.526775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:05.073812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581474+0000","last_change":"2024-03-28T16:48:54.657662+0000","last_active":"2024-03-28T16:49:37.581474+0000","last_peered":"2024-03-28T16:49:37.581474+0000","last_clean":"2024-03-28T16:49:37.581474+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:49:37.581474+0000","last_undegraded":"2024-03-28T16:49:37.581474+0000","last_fullsized":"2024-03-28T16:49:37.581474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:59.369957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'1","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369831+0000","last_change":"2024-03-28T16:48:56.682872+0000","last_active":"2024-03-28T16:50:22.369831+0000","last_peered":"2024-03-28T16:50:22.369831+0000","last_clean":"2024-03-28T16:50:22.369831+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:50:22.369831+0000","last_undegraded":"2024-03-28T16:50:22.369831+0000","last_fullsized":"2024-03-28T16:50:22.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:50.772258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584620+0000","last_change":"2024-03-28T16:48:54.663496+0000","last_active":"2024-03-28T16:49:37.584620+0000","last_peered":"2024-03-28T16:49:37.584620+0000","last_clean":"2024-03-28T16:49:37.584620+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:49:37.584620+0000","last_undegraded":"2024-03-28T16:49:37.584620+0000","last_fullsized":"2024-03-28T16:49:37.584620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587192+0000","last_change":"2024-03-28T16:48:56.678342+0000","last_active":"2024-03-28T16:49:37.587192+0000","last_peered":"2024-03-28T16:49:37.587192+0000","last_clean":"2024-03-28T16:49:37.587192+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:49:37.587192+0000","last_undegraded":"2024-03-28T16:49:37.587192+0000","last_fullsized":"2024-03-28T16:49:37.587192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585680+0000","last_change":"2024-03-28T16:48:54.675150+0000","last_active":"2024-03-28T16:49:37.585680+0000","last_peered":"2024-03-28T16:49:37.585680+0000","last_clean":"2024-03-28T16:49:37.585680+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:49:37.585680+0000","last_undegraded":"2024-03-28T16:49:37.585680+0000","last_fullsized":"2024-03-28T16:49:37.585680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190164+0000","last_change":"2024-03-28T16:48:56.671157+0000","last_active":"2024-03-28T16:50:23.190164+0000","last_peered":"2024-03-28T16:50:23.190164+0000","last_clean":"2024-03-28T16:50:23.190164+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:50:23.190164+0000","last_undegraded":"2024-03-28T16:50:23.190164+0000","last_fullsized":"2024-03-28T16:50:23.190164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:59.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585793+0000","last_change":"2024-03-28T16:48:54.674973+0000","last_active":"2024-03-28T16:49:37.585793+0000","last_peered":"2024-03-28T16:49:37.585793+0000","last_clean":"2024-03-28T16:49:37.585793+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:49:37.585793+0000","last_undegraded":"2024-03-28T16:49:37.585793+0000","last_fullsized":"2024-03-28T16:49:37.585793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840005+0000","last_change":"2024-03-28T16:48:56.683829+0000","last_active":"2024-03-28T16:50:28.840005+0000","last_peered":"2024-03-28T16:50:28.840005+0000","last_clean":"2024-03-28T16:50:28.840005+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:50:28.840005+0000","last_undegraded":"2024-03-28T16:50:28.840005+0000","last_fullsized":"2024-03-28T16:50:28.840005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587774+0000","last_change":"2024-03-28T16:48:54.676815+0000","last_active":"2024-03-28T16:49:37.587774+0000","last_peered":"2024-03-28T16:49:37.587774+0000","last_clean":"2024-03-28T16:49:37.587774+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:49:37.587774+0000","last_undegraded":"2024-03-28T16:49:37.587774+0000","last_fullsized":"2024-03-28T16:49:37.587774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:19.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839390+0000","last_change":"2024-03-28T16:48:56.693060+0000","last_active":"2024-03-28T16:50:28.839390+0000","last_peered":"2024-03-28T16:50:28.839390+0000","last_clean":"2024-03-28T16:50:28.839390+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:50:28.839390+0000","last_undegraded":"2024-03-28T16:50:28.839390+0000","last_fullsized":"2024-03-28T16:50:28.839390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589164+0000","last_change":"2024-03-28T16:48:54.663003+0000","last_active":"2024-03-28T16:49:37.589164+0000","last_peered":"2024-03-28T16:49:37.589164+0000","last_clean":"2024-03-28T16:49:37.589164+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:49:37.589164+0000","last_undegraded":"2024-03-28T16:49:37.589164+0000","last_fullsized":"2024-03-28T16:49:37.589164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:43.737071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.078594+0000","last_change":"2024-03-28T16:48:56.685753+0000","last_active":"2024-03-28T16:49:39.078594+0000","last_peered":"2024-03-28T16:49:39.078594+0000","last_clean":"2024-03-28T16:49:39.078594+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:49:39.078594+0000","last_undegraded":"2024-03-28T16:49:39.078594+0000","last_fullsized":"2024-03-28T16:49:39.078594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:12.442441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598216+0000","last_change":"2024-03-28T16:48:54.661896+0000","last_active":"2024-03-28T16:49:37.598216+0000","last_peered":"2024-03-28T16:49:37.598216+0000","last_clean":"2024-03-28T16:49:37.598216+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:49:37.598216+0000","last_undegraded":"2024-03-28T16:49:37.598216+0000","last_fullsized":"2024-03-28T16:49:37.598216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'9","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877960+0000","last_change":"2024-03-28T16:48:56.680119+0000","last_active":"2024-03-28T16:49:39.877960+0000","last_peered":"2024-03-28T16:49:39.877960+0000","last_clean":"2024-03-28T16:49:39.877960+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:49:39.877960+0000","last_undegraded":"2024-03-28T16:49:39.877960+0000","last_fullsized":"2024-03-28T16:49:39.877960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588667+0000","last_change":"2024-03-28T16:48:54.665812+0000","last_active":"2024-03-28T16:49:37.588667+0000","last_peered":"2024-03-28T16:49:37.588667+0000","last_clean":"2024-03-28T16:49:37.588667+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:49:37.588667+0000","last_undegraded":"2024-03-28T16:49:37.588667+0000","last_fullsized":"2024-03-28T16:49:37.588667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:31.951558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581570+0000","last_change":"2024-03-28T16:48:54.665260+0000","last_active":"2024-03-28T16:49:37.581570+0000","last_peered":"2024-03-28T16:49:37.581570+0000","last_clean":"2024-03-28T16:49:37.581570+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:49:37.581570+0000","last_undegraded":"2024-03-28T16:49:37.581570+0000","last_fullsized":"2024-03-28T16:49:37.581570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:37.463264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591429+0000","last_change":"2024-03-28T16:48:56.690793+0000","last_active":"2024-03-28T16:49:37.591429+0000","last_peered":"2024-03-28T16:49:37.591429+0000","last_clean":"2024-03-28T16:49:37.591429+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:49:37.591429+0000","last_undegraded":"2024-03-28T16:49:37.591429+0000","last_fullsized":"2024-03-28T16:49:37.591429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584402+0000","last_change":"2024-03-28T16:48:54.663735+0000","last_active":"2024-03-28T16:49:37.584402+0000","last_peered":"2024-03-28T16:49:37.584402+0000","last_clean":"2024-03-28T16:49:37.584402+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:49:37.584402+0000","last_undegraded":"2024-03-28T16:49:37.584402+0000","last_fullsized":"2024-03-28T16:49:37.584402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587762+0000","last_change":"2024-03-28T16:48:56.683832+0000","last_active":"2024-03-28T16:49:37.587762+0000","last_peered":"2024-03-28T16:49:37.587762+0000","last_clean":"2024-03-28T16:49:37.587762+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:49:37.587762+0000","last_undegraded":"2024-03-28T16:49:37.587762+0000","last_fullsized":"2024-03-28T16:49:37.587762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:38.808030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1968,"ondisk_log_size":1968,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1344,"num_read_kb":1127,"num_write":517,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1175,"ondisk_log_size":1175,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":854,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39916,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331924,"statfs":{"total":96636764160,"available":96595890176,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.752}]},{"osd":2,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.51900000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705691,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.75},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56699999999999995}]},{"osd":3,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.64200000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705691,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78500000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705691,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:50:43.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T16:50:43.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.890+0000 7fa605e84640 1 Processor -- start 2024-03-28T16:50:43.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.890+0000 7fa605e84640 1 -- start start 2024-03-28T16:50:43.896 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa600099020 0x7fa600099400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa600099940 con 0x7fa600097160 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60009a1c0 con 0x7fa600099020 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5feffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa600099020 0x7fa600099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5feffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa600099020 0x7fa600099400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34142/0 (socket says 172.21.15.119:34142) 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5feffd640 1 -- 172.21.15.119:0/1751520115 learned_addr learned my addr 172.21.15.119:0/1751520115 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5ff7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa600097160 0x7fa600098910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43816/0 (socket says 172.21.15.119:43816) 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5feffd640 1 -- 172.21.15.119:0/1751520115 >> v1:172.21.15.119:6789/0 conn(0x7fa600097160 legacy=0x7fa600098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5feffd640 1 -- 172.21.15.119:0/1751520115 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60009a4d0 con 0x7fa600099020 2024-03-28T16:50:43.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5feffd640 1 --2- 172.21.15.119:0/1751520115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600099020 0x7fa600099400 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40097f0 tx=0x7fa5f401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b3514aa8721b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5fe7fc640 1 -- 172.21.15.119:0/1751520115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5f40aa570 con 0x7fa600099020 2024-03-28T16:50:43.898 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5fe7fc640 1 -- 172.21.15.119:0/1751520115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f40aa6d0 con 0x7fa600099020 2024-03-28T16:50:43.899 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa5fe7fc640 1 -- 172.21.15.119:0/1751520115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5f40aa990 con 0x7fa600099020 2024-03-28T16:50:43.900 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 -- 172.21.15.119:0/1751520115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600099020 msgr2=0x7fa600099400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 --2- 172.21.15.119:0/1751520115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600099020 0x7fa600099400 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40097f0 tx=0x7fa5f401c4c0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 -- 172.21.15.119:0/1751520115 shutdown_connections 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 --2- 172.21.15.119:0/1751520115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600099020 0x7fa600099400 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 -- 172.21.15.119:0/1751520115 >> 172.21.15.119:0/1751520115 conn(0x7fa6000a0a00 msgr2=0x7fa60009cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 -- 172.21.15.119:0/1751520115 shutdown_connections 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.894+0000 7fa605e84640 1 -- 172.21.15.119:0/1751520115 wait complete. 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa605e84640 1 Processor -- start 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa605e84640 1 -- start start 2024-03-28T16:50:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa605e84640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600097160 0x7fa600132200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa605e84640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa600095ba0 con 0x7fa600097160 2024-03-28T16:50:43.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600097160 0x7fa600132200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600097160 0x7fa600132200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34150/0 (socket says 172.21.15.119:34150) 2024-03-28T16:50:43.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5ff7fe640 1 -- 172.21.15.119:0/1151951893 learned_addr learned my addr 172.21.15.119:0/1151951893 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:43.902 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5ff7fe640 1 -- 172.21.15.119:0/1151951893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40094d0 con 0x7fa600097160 2024-03-28T16:50:43.903 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5ff7fe640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600097160 0x7fa600132200 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c480 tx=0x7fa5f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5e7fff640 1 -- 172.21.15.119:0/1151951893 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5f00073b0 con 0x7fa600097160 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5e7fff640 1 -- 172.21.15.119:0/1151951893 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0007510 con 0x7fa600097160 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa600095770 con 0x7fa600097160 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5e7fff640 1 -- 172.21.15.119:0/1151951893 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5f00047f0 con 0x7fa600097160 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa600094a70 con 0x7fa600097160 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa600071bd0 con 0x7fa600097160 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5e7fff640 1 -- 172.21.15.119:0/1151951893 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa5f0004950 con 0x7fa600097160 2024-03-28T16:50:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5e7fff640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5ec035a10 0x7fa5ec037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:43.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5e7fff640 1 -- 172.21.15.119:0/1151951893 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa5f0047a90 con 0x7fa600097160 2024-03-28T16:50:43.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.898+0000 7fa5feffd640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5ec035a10 0x7fa5ec037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:43.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.902+0000 7fa5feffd640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5ec035a10 0x7fa5ec037ed0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ae000 tx=0x7fa5f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:43.902+0000 7fa5e7fff640 1 -- 172.21.15.119:0/1151951893 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5f001a8b0 con 0x7fa600097160 2024-03-28T16:50:44.062 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.058+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa600099940 con 0x7fa600097160 2024-03-28T16:50:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa5e7fff640 1 -- 172.21.15.119:0/1151951893 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+21125 (secure 0 0 0) 0x7fa600099940 con 0x7fa600097160 2024-03-28T16:50:44.064 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:50:44.064 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":31,"fsid":"5a56263c-53d8-4d15-86cb-f8a4a2a85230","created":"2024-03-28T16:48:31.676831+0000","modified":"2024-03-28T16:49:37.571420+0000","last_up_change":"2024-03-28T16:48:40.552150+0000","last_in_change":"2024-03-28T16:48:32.834854+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T16:48:42.560893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-03-28T16:48:47.418826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-03-28T16:48:52.791746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-03-28T16:48:55.014059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.190000057220459,"score_stable":1.190000057220459,"optimal_score":1,"raw_score_acting":1.190000057220459,"raw_score_stable":1.190000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-03-28T16:48:56.844473+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-03-28T16:48:58.845409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-03-28T16:49:00.767701+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-03-28T16:49:02.892857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75d33f3f-95d7-438e-8617-4161e6766019","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6801","nonce":2028815515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6803","nonce":2028815515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6807","nonce":2028815515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":2028815515},{"type":"v1","addr":"172.21.15.119:6805","nonce":2028815515}]},"public_addr":"172.21.15.119:6801/2028815515","cluster_addr":"172.21.15.119:6803/2028815515","heartbeat_back_addr":"172.21.15.119:6807/2028815515","heartbeat_front_addr":"172.21.15.119:6805/2028815515","state":["exists","up"]},{"osd":1,"uuid":"29d55d30-92be-4309-a398-2412a4b9674f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6807","nonce":11349688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6811","nonce":11349688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6815","nonce":11349688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":11349688},{"type":"v1","addr":"172.21.15.170:6813","nonce":11349688}]},"public_addr":"172.21.15.170:6807/11349688","cluster_addr":"172.21.15.170:6811/11349688","heartbeat_back_addr":"172.21.15.170:6815/11349688","heartbeat_front_addr":"172.21.15.170:6813/11349688","state":["exists","up"]},{"osd":2,"uuid":"60322449-dbf6-4117-9bc4-10ac42982aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6801","nonce":447751618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6803","nonce":447751618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6810","nonce":447751618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":447751618},{"type":"v1","addr":"172.21.15.170:6806","nonce":447751618}]},"public_addr":"172.21.15.170:6801/447751618","cluster_addr":"172.21.15.170:6803/447751618","heartbeat_back_addr":"172.21.15.170:6810/447751618","heartbeat_front_addr":"172.21.15.170:6806/447751618","state":["exists","up"]},{"osd":3,"uuid":"df1546de-4221-40e6-b2b6-cdc2d7f86a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6817","nonce":2102412426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6819","nonce":2102412426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6823","nonce":2102412426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":2102412426},{"type":"v1","addr":"172.21.15.170:6821","nonce":2102412426}]},"public_addr":"172.21.15.170:6817/2102412426","cluster_addr":"172.21.15.170:6819/2102412426","heartbeat_back_addr":"172.21.15.170:6823/2102412426","heartbeat_front_addr":"172.21.15.170:6821/2102412426","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.161553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.340062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.387074+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T16:48:38.214018+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":3,"to":0}]},{"pgid":"3.36","mappings":[{"from":3,"to":1}]},{"pgid":"4.6","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":3,"to":0}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]},{"pgid":"5.f","mappings":[{"from":0,"to":2}]},{"pgid":"5.12","mappings":[{"from":0,"to":2}]},{"pgid":"5.1c","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T16:50:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5ec035a10 msgr2=0x7fa5ec037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5ec035a10 0x7fa5ec037ed0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ae000 tx=0x7fa5f4008040 comp rx=0 tx=0).stop 2024-03-28T16:50:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600097160 msgr2=0x7fa600132200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600097160 0x7fa600132200 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c480 tx=0x7fa5f000c940 comp rx=0 tx=0).stop 2024-03-28T16:50:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 shutdown_connections 2024-03-28T16:50:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa5ec035a10 0x7fa5ec037ed0 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40ae000 tx=0x7fa5f4008040 comp rx=0 tx=0).stop 2024-03-28T16:50:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 --2- 172.21.15.119:0/1151951893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa600097160 0x7fa600132200 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 >> 172.21.15.119:0/1151951893 conn(0x7fa6000a0a00 msgr2=0x7fa60006f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 shutdown_connections 2024-03-28T16:50:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:44.062+0000 7fa605e84640 1 -- 172.21.15.119:0/1151951893 wait complete. 2024-03-28T16:50:45.082 INFO:tasks.ceph:Scrubbing osd.0 2024-03-28T16:50:45.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-28T16:50:45.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c08eeb640 1 Processor -- start 2024-03-28T16:50:45.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c08eeb640 1 -- start start 2024-03-28T16:50:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c08eeb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c041353f0 0x7f1c041377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c08eeb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c04096430 con 0x7f1c0412ed80 2024-03-28T16:50:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c08eeb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c04094e90 con 0x7f1c041353f0 2024-03-28T16:50:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c01d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c041353f0 0x7f1c041377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c0259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1c0412ed80 0x7f1c0412f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43830/0 (socket says 172.21.15.119:43830) 2024-03-28T16:50:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c0259c640 1 -- 172.21.15.119:0/3313938337 learned_addr learned my addr 172.21.15.119:0/3313938337 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:45.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c0159a640 1 -- 172.21.15.119:0/3313938337 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511019680 0 0) 0x7f1c04096430 con 0x7f1c0412ed80 2024-03-28T16:50:45.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c0159a640 1 -- 172.21.15.119:0/3313938337 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bec003530 con 0x7f1c0412ed80 2024-03-28T16:50:45.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c01d9b640 1 -- 172.21.15.119:0/3313938337 >> v1:172.21.15.119:6789/0 conn(0x7f1c0412ed80 legacy=0x7f1c0412f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.220 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.214+0000 7f1c01d9b640 1 -- 172.21.15.119:0/3313938337 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0412f870 con 0x7f1c041353f0 2024-03-28T16:50:45.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c01d9b640 1 --2- 172.21.15.119:0/3313938337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c041353f0 0x7f1c041377e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009740 tx=0x7f1bf801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ca320e7011a2b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c0159a640 1 -- 172.21.15.119:0/3313938337 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf80ab530 con 0x7f1c041353f0 2024-03-28T16:50:45.221 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c0159a640 1 -- 172.21.15.119:0/3313938337 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80a7050 con 0x7f1c041353f0 2024-03-28T16:50:45.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c0159a640 1 -- 172.21.15.119:0/3313938337 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf80ab7e0 con 0x7f1c041353f0 2024-03-28T16:50:45.222 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3313938337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c041353f0 msgr2=0x7f1c041377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3313938337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c041353f0 0x7f1c041377e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009740 tx=0x7f1bf801c370 comp rx=0 tx=0).stop 2024-03-28T16:50:45.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3313938337 shutdown_connections 2024-03-28T16:50:45.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3313938337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c041353f0 0x7f1c041377e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3313938337 >> 172.21.15.119:0/3313938337 conn(0x7f1c0409a050 msgr2=0x7f1c0409a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3313938337 shutdown_connections 2024-03-28T16:50:45.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3313938337 wait complete. 2024-03-28T16:50:45.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.218+0000 7f1c08eeb640 1 Processor -- start 2024-03-28T16:50:45.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c08eeb640 1 -- start start 2024-03-28T16:50:45.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c08eeb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0412ed80 0x7f1c041c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c08eeb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0412c800 con 0x7f1c0412ed80 2024-03-28T16:50:45.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c0259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0412ed80 0x7f1c041c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.225 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c0259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0412ed80 0x7f1c041c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34168/0 (socket says 172.21.15.119:34168) 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c0259c640 1 -- 172.21.15.119:0/3428547069 learned_addr learned my addr 172.21.15.119:0/3428547069 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c0259c640 1 -- 172.21.15.119:0/3428547069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf8009420 con 0x7f1c0412ed80 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c0259c640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0412ed80 0x7f1c041c9be0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400c450 tx=0x7f1bf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf4005c20 con 0x7f1c0412ed80 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf4005d80 con 0x7f1c0412ed80 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bf40071a0 con 0x7f1c0412ed80 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c04137880 con 0x7f1c0412ed80 2024-03-28T16:50:45.226 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0406d450 con 0x7f1c0412ed80 2024-03-28T16:50:45.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c04062860 con 0x7f1c0412ed80 2024-03-28T16:50:45.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f1bf4007300 con 0x7f1c0412ed80 2024-03-28T16:50:45.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1be4035a40 0x7f1be4037f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f1bf4047c10 con 0x7f1c0412ed80 2024-03-28T16:50:45.228 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f1be40418a0 0x7f1be4043d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1be4044410 con 0x7f1be40418a0 2024-03-28T16:50:45.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bf404c2a0 con 0x7f1c0412ed80 2024-03-28T16:50:45.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c02d9d640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f1be40418a0 0x7f1be4043d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c01d9b640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1be4035a40 0x7f1be4037f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.222+0000 7f1c02d9d640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f1be40418a0 0x7f1be4043d40 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.226+0000 7f1c01d9b640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1be4035a40 0x7f1be4037f00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80ad000 tx=0x7f1bf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.229 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.226+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== osd.0 v2:172.21.15.119:6800/2028815515 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1be4044410 con 0x7f1be40418a0 2024-03-28T16:50:45.240 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 --> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1c0412be80 con 0x7f1be40418a0 2024-03-28T16:50:45.241 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1bf2ffd640 1 -- 172.21.15.119:0/3428547069 <== osd.0 v2:172.21.15.119:6800/2028815515 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1c0412be80 con 0x7f1be40418a0 2024-03-28T16:50:45.241 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T16:50:45.241 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T16:50:45.241 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T16:50:45.242 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f1be40418a0 msgr2=0x7f1be4043d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f1be40418a0 0x7f1be4043d40 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1be4035a40 msgr2=0x7f1be4037f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1be4035a40 0x7f1be4037f00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80ad000 tx=0x7f1bf8008040 comp rx=0 tx=0).stop 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0412ed80 msgr2=0x7f1c041c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0412ed80 0x7f1c041c9be0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400c450 tx=0x7f1bf400c910 comp rx=0 tx=0).stop 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 shutdown_connections 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:6800/2028815515,v1:172.21.15.119:6801/2028815515] conn(0x7f1be40418a0 0x7f1be4043d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.243 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1be4035a40 0x7f1be4037f00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 --2- 172.21.15.119:0/3428547069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c0412ed80 0x7f1c041c9be0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 >> 172.21.15.119:0/3428547069 conn(0x7f1c0409a050 msgr2=0x7f1c0412d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 shutdown_connections 2024-03-28T16:50:45.244 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.238+0000 7f1c08eeb640 1 -- 172.21.15.119:0/3428547069 wait complete. 2024-03-28T16:50:45.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-28T16:50:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69f359640 1 Processor -- start 2024-03-28T16:50:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69f359640 1 -- start start 2024-03-28T16:50:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69f359640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd69812aca0 0x7fd69812f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69f359640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69806c700 con 0x7fd69812a1b0 2024-03-28T16:50:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69f359640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980954e0 con 0x7fd69812aca0 2024-03-28T16:50:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69d0f5640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd69812a1b0 0x7fd69812a590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43846/0 (socket says 172.21.15.119:43846) 2024-03-28T16:50:45.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69d0f5640 1 -- 172.21.15.119:0/2771809070 learned_addr learned my addr 172.21.15.119:0/2771809070 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:45.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69c8f4640 1 --2- 172.21.15.119:0/2771809070 >> v2:172.21.15.119:3300/0 conn(0x7fd69812aca0 0x7fd69812f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd68ffff640 1 -- 172.21.15.119:0/2771809070 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657002980 0 0) 0x7fd69806c700 con 0x7fd69812a1b0 2024-03-28T16:50:45.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd68ffff640 1 -- 172.21.15.119:0/2771809070 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd684003530 con 0x7fd69812a1b0 2024-03-28T16:50:45.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69c8f4640 1 -- 172.21.15.119:0/2771809070 >> v1:172.21.15.119:6789/0 conn(0x7fd69812a1b0 legacy=0x7fd69812a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.371 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.366+0000 7fd69c8f4640 1 -- 172.21.15.119:0/2771809070 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698095f90 con 0x7fd69812aca0 2024-03-28T16:50:45.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69c8f4640 1 --2- 172.21.15.119:0/2771809070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812aca0 0x7fd69812f6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd68800a8e0 tx=0x7fd68801d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d9b29765549d6c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.372 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd68ffff640 1 -- 172.21.15.119:0/2771809070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6880b1530 con 0x7fd69812aca0 2024-03-28T16:50:45.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd68ffff640 1 -- 172.21.15.119:0/2771809070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6880b1690 con 0x7fd69812aca0 2024-03-28T16:50:45.373 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd68ffff640 1 -- 172.21.15.119:0/2771809070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6880b1950 con 0x7fd69812aca0 2024-03-28T16:50:45.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 -- 172.21.15.119:0/2771809070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812aca0 msgr2=0x7fd69812f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 --2- 172.21.15.119:0/2771809070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812aca0 0x7fd69812f6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd68800a8e0 tx=0x7fd68801d740 comp rx=0 tx=0).stop 2024-03-28T16:50:45.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 -- 172.21.15.119:0/2771809070 shutdown_connections 2024-03-28T16:50:45.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 --2- 172.21.15.119:0/2771809070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812aca0 0x7fd69812f6e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 -- 172.21.15.119:0/2771809070 >> 172.21.15.119:0/2771809070 conn(0x7fd69809a0c0 msgr2=0x7fd69809a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 -- 172.21.15.119:0/2771809070 shutdown_connections 2024-03-28T16:50:45.374 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 -- 172.21.15.119:0/2771809070 wait complete. 2024-03-28T16:50:45.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 Processor -- start 2024-03-28T16:50:45.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 -- start start 2024-03-28T16:50:45.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812a1b0 0x7fd6981c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69f359640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd698127560 con 0x7fd69812a1b0 2024-03-28T16:50:45.376 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69d0f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812a1b0 0x7fd6981c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69d0f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812a1b0 0x7fd6981c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34180/0 (socket says 172.21.15.119:34180) 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69d0f5640 1 -- 172.21.15.119:0/190711298 learned_addr learned my addr 172.21.15.119:0/190711298 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69d0f5640 1 -- 172.21.15.119:0/190711298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68800a5c0 con 0x7fd69812a1b0 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd69d0f5640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812a1b0 0x7fd6981c3290 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd69400c450 tx=0x7fd69400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd68dffb640 1 -- 172.21.15.119:0/190711298 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd694005c20 con 0x7fd69812a1b0 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd68dffb640 1 -- 172.21.15.119:0/190711298 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd694005d80 con 0x7fd69812a1b0 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.370+0000 7fd68dffb640 1 -- 172.21.15.119:0/190711298 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6940071a0 con 0x7fd69812a1b0 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69806c3f0 con 0x7fd69812a1b0 2024-03-28T16:50:45.377 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd698125780 con 0x7fd69812a1b0 2024-03-28T16:50:45.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd698126440 con 0x7fd69812a1b0 2024-03-28T16:50:45.378 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd68dffb640 1 -- 172.21.15.119:0/190711298 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fd694007300 con 0x7fd69812a1b0 2024-03-28T16:50:45.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd68dffb640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd67c035a10 0x7fd67c037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd68dffb640 1 -- 172.21.15.119:0/190711298 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd694047cd0 con 0x7fd69812a1b0 2024-03-28T16:50:45.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd69c8f4640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd67c035a10 0x7fd67c037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.379 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.374+0000 7fd69c8f4640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd67c035a10 0x7fd67c037ed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd6880b0000 tx=0x7fd6880a8000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.381 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.378+0000 7fd68dffb640 1 -- 172.21.15.119:0/190711298 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd694012400 con 0x7fd69812a1b0 2024-03-28T16:50:45.528 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.526+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6980959c0 con 0x7fd67c035a10 2024-03-28T16:50:45.530 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.526+0000 7fd68dffb640 1 -- 172.21.15.119:0/190711298 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd6880b1400 con 0x7fd67c035a10 2024-03-28T16:50:45.530 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 0 to deep-scrub 2024-03-28T16:50:45.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd67c035a10 msgr2=0x7fd67c037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.532 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd67c035a10 0x7fd67c037ed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd6880b0000 tx=0x7fd6880a8000 comp rx=0 tx=0).stop 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812a1b0 msgr2=0x7fd6981c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812a1b0 0x7fd6981c3290 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd69400c450 tx=0x7fd69400c910 comp rx=0 tx=0).stop 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 shutdown_connections 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fd67c035a10 0x7fd67c037ed0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 --2- 172.21.15.119:0/190711298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd69812a1b0 0x7fd6981c3290 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 >> 172.21.15.119:0/190711298 conn(0x7fd69809a0c0 msgr2=0x7fd6980969f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 shutdown_connections 2024-03-28T16:50:45.533 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.530+0000 7fd69f359640 1 -- 172.21.15.119:0/190711298 wait complete. 2024-03-28T16:50:45.547 INFO:tasks.ceph:Scrubbing osd.1 2024-03-28T16:50:45.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-28T16:50:45.663 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a56be2640 1 Processor -- start 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a56be2640 1 -- start start 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a56be2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a500941a0 0x7f4a50094580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a56be2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a50095830 con 0x7f4a50094ac0 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a56be2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5006c370 con 0x7f4a500941a0 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a5497e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a500941a0 0x7f4a50094580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a4ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4a50094ac0 0x7f4a5012a140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43854/0 (socket says 172.21.15.119:43854) 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a4ffff640 1 -- 172.21.15.119:0/598451866 learned_addr learned my addr 172.21.15.119:0/598451866 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a5497e640 1 -- 172.21.15.119:0/598451866 >> v1:172.21.15.119:6789/0 conn(0x7f4a50094ac0 legacy=0x7f4a5012a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a5497e640 1 -- 172.21.15.119:0/598451866 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50071e40 con 0x7f4a500941a0 2024-03-28T16:50:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.658+0000 7f4a4f7fe640 1 -- 172.21.15.119:0/598451866 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988712661 0 0) 0x7f4a50095830 con 0x7f4a50094ac0 2024-03-28T16:50:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a5497e640 1 --2- 172.21.15.119:0/598451866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a50094580 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009740 tx=0x7f4a4001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa34678e0460ccb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a4f7fe640 1 -- 172.21.15.119:0/598451866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a400ab530 con 0x7f4a500941a0 2024-03-28T16:50:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a4f7fe640 1 -- 172.21.15.119:0/598451866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a400a7050 con 0x7f4a500941a0 2024-03-28T16:50:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a4f7fe640 1 -- 172.21.15.119:0/598451866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a400ab7e0 con 0x7f4a500941a0 2024-03-28T16:50:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 -- 172.21.15.119:0/598451866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 msgr2=0x7f4a50094580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 --2- 172.21.15.119:0/598451866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a50094580 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009740 tx=0x7f4a4001c310 comp rx=0 tx=0).stop 2024-03-28T16:50:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 -- 172.21.15.119:0/598451866 shutdown_connections 2024-03-28T16:50:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 --2- 172.21.15.119:0/598451866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a50094580 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 -- 172.21.15.119:0/598451866 >> 172.21.15.119:0/598451866 conn(0x7f4a5009a050 msgr2=0x7f4a5009a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 -- 172.21.15.119:0/598451866 shutdown_connections 2024-03-28T16:50:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 -- 172.21.15.119:0/598451866 wait complete. 2024-03-28T16:50:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 Processor -- start 2024-03-28T16:50:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.662+0000 7f4a56be2640 1 -- start start 2024-03-28T16:50:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a56be2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a5008c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a56be2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5008deb0 con 0x7f4a500941a0 2024-03-28T16:50:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a5497e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a5008c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a5497e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a5008c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34188/0 (socket says 172.21.15.119:34188) 2024-03-28T16:50:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a5497e640 1 -- 172.21.15.119:0/583135297 learned_addr learned my addr 172.21.15.119:0/583135297 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a5497e640 1 -- 172.21.15.119:0/583135297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009420 con 0x7f4a500941a0 2024-03-28T16:50:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a5497e640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a5008c170 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4a40018f80 tx=0x7f4a4001abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a400b7450 con 0x7f4a500941a0 2024-03-28T16:50:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a400b75b0 con 0x7f4a500941a0 2024-03-28T16:50:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5008da10 con 0x7f4a500941a0 2024-03-28T16:50:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a400b7890 con 0x7f4a500941a0 2024-03-28T16:50:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5008c6b0 con 0x7f4a500941a0 2024-03-28T16:50:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a50062860 con 0x7f4a500941a0 2024-03-28T16:50:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f4a400b79f0 con 0x7f4a500941a0 2024-03-28T16:50:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a3c035a10 0x7f4a3c037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4a400e2cc0 con 0x7f4a500941a0 2024-03-28T16:50:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f4a3c041870 0x7f4a3c043d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a3c0443e0 con 0x7f4a3c041870 2024-03-28T16:50:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a400e5330 con 0x7f4a500941a0 2024-03-28T16:50:45.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4ffff640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a3c035a10 0x7f4a3c037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a5517f640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f4a3c041870 0x7f4a3c043d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a5517f640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f4a3c041870 0x7f4a3c043d10 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.675 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.666+0000 7f4a4ffff640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a3c035a10 0x7f4a3c037ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a44007fd0 tx=0x7f4a44007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.670+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== osd.1 v2:172.21.15.170:6805/11349688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4a3c0443e0 con 0x7f4a3c041870 2024-03-28T16:50:45.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 --> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4a500717f0 con 0x7f4a3c041870 2024-03-28T16:50:45.686 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a4d7fa640 1 -- 172.21.15.119:0/583135297 <== osd.1 v2:172.21.15.170:6805/11349688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4a500717f0 con 0x7f4a3c041870 2024-03-28T16:50:45.686 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T16:50:45.686 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T16:50:45.686 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T16:50:45.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f4a3c041870 msgr2=0x7f4a3c043d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.687 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f4a3c041870 0x7f4a3c043d10 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a3c035a10 msgr2=0x7f4a3c037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a3c035a10 0x7f4a3c037ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a44007fd0 tx=0x7f4a44007980 comp rx=0 tx=0).stop 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 msgr2=0x7f4a5008c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a5008c170 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4a40018f80 tx=0x7f4a4001abb0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 shutdown_connections 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6805/11349688,v1:172.21.15.170:6807/11349688] conn(0x7f4a3c041870 0x7f4a3c043d10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f4a3c035a10 0x7f4a3c037ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.688 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 --2- 172.21.15.119:0/583135297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a500941a0 0x7f4a5008c170 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 >> 172.21.15.119:0/583135297 conn(0x7f4a5009a050 msgr2=0x7f4a5012b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 shutdown_connections 2024-03-28T16:50:45.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.682+0000 7f4a56be2640 1 -- 172.21.15.119:0/583135297 wait complete. 2024-03-28T16:50:45.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-28T16:50:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f129b73b640 1 Processor -- start 2024-03-28T16:50:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f129b73b640 1 -- start start 2024-03-28T16:50:45.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f129b73b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1294098040 0x7f1294098420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.813 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f129b73b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129409a6d0 con 0x7f1294098960 2024-03-28T16:50:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f129b73b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129409a3c0 con 0x7f1294098040 2024-03-28T16:50:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f12994d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1294098040 0x7f1294098420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f12994d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1294098040 0x7f1294098420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34192/0 (socket says 172.21.15.119:34192) 2024-03-28T16:50:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f12994d7640 1 -- 172.21.15.119:0/2651736930 learned_addr learned my addr 172.21.15.119:0/2651736930 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f12994d7640 1 -- 172.21.15.119:0/2651736930 >> v1:172.21.15.119:6789/0 conn(0x7f1294098960 legacy=0x7f12941326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.810+0000 7f12994d7640 1 -- 172.21.15.119:0/2651736930 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294099810 con 0x7f1294098040 2024-03-28T16:50:45.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f12994d7640 1 --2- 172.21.15.119:0/2651736930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294098420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f12880097f0 tx=0x7f128801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1fb2ff85c54876 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f1287fff640 1 -- 172.21.15.119:0/2651736930 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12880ab530 con 0x7f1294098040 2024-03-28T16:50:45.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f1287fff640 1 -- 172.21.15.119:0/2651736930 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12880a7050 con 0x7f1294098040 2024-03-28T16:50:45.816 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f1287fff640 1 -- 172.21.15.119:0/2651736930 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12880ab7e0 con 0x7f1294098040 2024-03-28T16:50:45.817 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- 172.21.15.119:0/2651736930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 msgr2=0x7f1294098420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 --2- 172.21.15.119:0/2651736930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294098420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f12880097f0 tx=0x7f128801c430 comp rx=0 tx=0).stop 2024-03-28T16:50:45.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- 172.21.15.119:0/2651736930 shutdown_connections 2024-03-28T16:50:45.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 --2- 172.21.15.119:0/2651736930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294098420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- 172.21.15.119:0/2651736930 >> 172.21.15.119:0/2651736930 conn(0x7f129409e870 msgr2=0x7f129409ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- 172.21.15.119:0/2651736930 shutdown_connections 2024-03-28T16:50:45.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- 172.21.15.119:0/2651736930 wait complete. 2024-03-28T16:50:45.818 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 Processor -- start 2024-03-28T16:50:45.819 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- start start 2024-03-28T16:50:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294132360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1294093330 con 0x7f1294098040 2024-03-28T16:50:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f12994d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294132360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f12994d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294132360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34194/0 (socket says 172.21.15.119:34194) 2024-03-28T16:50:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f12994d7640 1 -- 172.21.15.119:0/2222728130 learned_addr learned my addr 172.21.15.119:0/2222728130 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f12994d7640 1 -- 172.21.15.119:0/2222728130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12880094d0 con 0x7f1294098040 2024-03-28T16:50:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f12994d7640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294132360 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f12880097f0 tx=0x7f12880191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f1285ffb640 1 -- 172.21.15.119:0/2222728130 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12880b4830 con 0x7f1294098040 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f1285ffb640 1 -- 172.21.15.119:0/2222728130 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12880b49b0 con 0x7f1294098040 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1294125d30 con 0x7f1294098040 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f1285ffb640 1 -- 172.21.15.119:0/2222728130 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12880b4c90 con 0x7f1294098040 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.814+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1294092e90 con 0x7f1294098040 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.818+0000 7f1285ffb640 1 -- 172.21.15.119:0/2222728130 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f12880b2040 con 0x7f1294098040 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.818+0000 7f1285ffb640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1278035a10 0x7f1278037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.818+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1294125ba0 con 0x7f1294098040 2024-03-28T16:50:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.818+0000 7f1285ffb640 1 -- 172.21.15.119:0/2222728130 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f12880e2b10 con 0x7f1294098040 2024-03-28T16:50:45.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.818+0000 7f1298cd6640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1278035a10 0x7f1278037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:45.823 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.818+0000 7f1298cd6640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1278035a10 0x7f1278037ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1290001610 tx=0x7f12900079e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:45.824 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.822+0000 7f1285ffb640 1 -- 172.21.15.119:0/2222728130 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12880bedd0 con 0x7f1294098040 2024-03-28T16:50:45.982 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.978+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1294094200 con 0x7f1278035a10 2024-03-28T16:50:45.983 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.978+0000 7f1285ffb640 1 -- 172.21.15.119:0/2222728130 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1294094200 con 0x7f1278035a10 2024-03-28T16:50:45.983 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 1 to deep-scrub 2024-03-28T16:50:45.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1278035a10 msgr2=0x7f1278037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1278035a10 0x7f1278037ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1290001610 tx=0x7f12900079e0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 msgr2=0x7f1294132360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:45.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294132360 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f12880097f0 tx=0x7f12880191d0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.986 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 shutdown_connections 2024-03-28T16:50:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f1278035a10 0x7f1278037ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 --2- 172.21.15.119:0/2222728130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1294098040 0x7f1294132360 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 >> 172.21.15.119:0/2222728130 conn(0x7f129409e870 msgr2=0x7f129409b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 shutdown_connections 2024-03-28T16:50:45.987 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:45.982+0000 7f129b73b640 1 -- 172.21.15.119:0/2222728130 wait complete. 2024-03-28T16:50:46.001 INFO:tasks.ceph:Scrubbing osd.2 2024-03-28T16:50:46.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-28T16:50:46.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1f1cc640 1 Processor -- start 2024-03-28T16:50:46.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1f1cc640 1 -- start start 2024-03-28T16:50:46.112 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1f1cc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc1806c0d0 0x7fcc1806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1f1cc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc180721d0 con 0x7fcc1806c9f0 2024-03-28T16:50:46.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1f1cc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc180727a0 con 0x7fcc1806c0d0 2024-03-28T16:50:46.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1cf68640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc1806c0d0 0x7fcc1806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc17fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcc1806c9f0 0x7fcc18132210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43866/0 (socket says 172.21.15.119:43866) 2024-03-28T16:50:46.113 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc17fff640 1 -- 172.21.15.119:0/3349880912 learned_addr learned my addr 172.21.15.119:0/3349880912 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:46.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc177fe640 1 -- 172.21.15.119:0/3349880912 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469997906 0 0) 0x7fcc180721d0 con 0x7fcc1806c9f0 2024-03-28T16:50:46.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc177fe640 1 -- 172.21.15.119:0/3349880912 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc08003530 con 0x7fcc1806c9f0 2024-03-28T16:50:46.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1cf68640 1 -- 172.21.15.119:0/3349880912 >> v1:172.21.15.119:6789/0 conn(0x7fcc1806c9f0 legacy=0x7fcc18132210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.114 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1cf68640 1 -- 172.21.15.119:0/3349880912 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc18071c30 con 0x7fcc1806c0d0 2024-03-28T16:50:46.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.110+0000 7fcc1cf68640 1 --2- 172.21.15.119:0/3349880912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806c4b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009770 tx=0x7fcc0c01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd37eab26f469217 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.115 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc177fe640 1 -- 172.21.15.119:0/3349880912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c0ab530 con 0x7fcc1806c0d0 2024-03-28T16:50:46.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc177fe640 1 -- 172.21.15.119:0/3349880912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c0a7050 con 0x7fcc1806c0d0 2024-03-28T16:50:46.116 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc177fe640 1 -- 172.21.15.119:0/3349880912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c0ab7e0 con 0x7fcc1806c0d0 2024-03-28T16:50:46.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/3349880912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 msgr2=0x7fcc1806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/3349880912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806c4b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009770 tx=0x7fcc0c01c370 comp rx=0 tx=0).stop 2024-03-28T16:50:46.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/3349880912 shutdown_connections 2024-03-28T16:50:46.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/3349880912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806c4b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.118 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/3349880912 >> 172.21.15.119:0/3349880912 conn(0x7fcc1809d470 msgr2=0x7fcc1809d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/3349880912 shutdown_connections 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/3349880912 wait complete. 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 Processor -- start 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- start start 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc18093f00 con 0x7fcc1806c0d0 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1cf68640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.119 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1cf68640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34212/0 (socket says 172.21.15.119:34212) 2024-03-28T16:50:46.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1cf68640 1 -- 172.21.15.119:0/1058004894 learned_addr learned my addr 172.21.15.119:0/1058004894 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:46.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1cf68640 1 -- 172.21.15.119:0/1058004894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c009420 con 0x7fcc1806c0d0 2024-03-28T16:50:46.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1cf68640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806d640 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0ae040 tx=0x7fcc0c019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c0abe10 con 0x7fcc1806c0d0 2024-03-28T16:50:46.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c0b4410 con 0x7fcc1806c0d0 2024-03-28T16:50:46.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc18071110 con 0x7fcc1806c0d0 2024-03-28T16:50:46.120 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc0c0b46d0 con 0x7fcc1806c0d0 2024-03-28T16:50:46.121 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc18133770 con 0x7fcc1806c0d0 2024-03-28T16:50:46.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc18093b90 con 0x7fcc1806c0d0 2024-03-28T16:50:46.122 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fcc0c0b2040 con 0x7fcc1806c0d0 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbfc035a10 0x7fcbfc037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fcc0c0e2760 con 0x7fcc1806c0d0 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fcbfc041810 0x7fcbfc043cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbfc044380 con 0x7fcbfc041810 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.114+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc0c0e4d70 con 0x7fcc1806c0d0 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.118+0000 7fcc17fff640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbfc035a10 0x7fcbfc037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.118+0000 7fcc1d769640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fcbfc041810 0x7fcbfc043cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.118+0000 7fcc17fff640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbfc035a10 0x7fcbfc037ed0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcc04007fd0 tx=0x7fcc04007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.123 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.118+0000 7fcc1d769640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fcbfc041810 0x7fcbfc043cb0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.124 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.118+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== osd.2 v2:172.21.15.170:6800/447751618 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcbfc044380 con 0x7fcbfc041810 2024-03-28T16:50:46.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 --> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcc18070d60 con 0x7fcbfc041810 2024-03-28T16:50:46.134 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc157fa640 1 -- 172.21.15.119:0/1058004894 <== osd.2 v2:172.21.15.170:6800/447751618 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcc10011040 con 0x7fcbfc041810 2024-03-28T16:50:46.135 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T16:50:46.135 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T16:50:46.135 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T16:50:46.136 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fcbfc041810 msgr2=0x7fcbfc043cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fcbfc041810 0x7fcbfc043cb0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbfc035a10 msgr2=0x7fcbfc037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbfc035a10 0x7fcbfc037ed0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcc04007fd0 tx=0x7fcc04007980 comp rx=0 tx=0).stop 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 msgr2=0x7fcc1806d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806d640 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0ae040 tx=0x7fcc0c019110 comp rx=0 tx=0).stop 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 shutdown_connections 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6800/447751618,v1:172.21.15.170:6801/447751618] conn(0x7fcbfc041810 0x7fcbfc043cb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.137 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fcbfc035a10 0x7fcbfc037ed0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 --2- 172.21.15.119:0/1058004894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc1806c0d0 0x7fcc1806d640 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 >> 172.21.15.119:0/1058004894 conn(0x7fcc1809d470 msgr2=0x7fcc1808c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 shutdown_connections 2024-03-28T16:50:46.138 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.130+0000 7fcc1f1cc640 1 -- 172.21.15.119:0/1058004894 wait complete. 2024-03-28T16:50:46.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-28T16:50:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f344f640 1 Processor -- start 2024-03-28T16:50:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f344f640 1 -- start start 2024-03-28T16:50:46.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f344f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f96ec0936c0 0x7f96ec093aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f344f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec095f80 con 0x7f96ec093fe0 2024-03-28T16:50:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f344f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec0724e0 con 0x7f96ec0936c0 2024-03-28T16:50:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f09ea640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f96ec093fe0 0x7f96ec12e180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43870/0 (socket says 172.21.15.119:43870) 2024-03-28T16:50:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f09ea640 1 -- 172.21.15.119:0/674116145 learned_addr learned my addr 172.21.15.119:0/674116145 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f11eb640 1 --2- 172.21.15.119:0/674116145 >> v2:172.21.15.119:3300/0 conn(0x7f96ec0936c0 0x7f96ec093aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f11eb640 1 -- 172.21.15.119:0/674116145 >> v1:172.21.15.119:6789/0 conn(0x7f96ec093fe0 legacy=0x7f96ec12e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f11eb640 1 -- 172.21.15.119:0/674116145 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec094b70 con 0x7f96ec0936c0 2024-03-28T16:50:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96e3fff640 1 -- 172.21.15.119:0/674116145 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619295627 0 0) 0x7f96ec095f80 con 0x7f96ec093fe0 2024-03-28T16:50:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96f11eb640 1 --2- 172.21.15.119:0/674116145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec093aa0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009740 tx=0x7f96dc01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2779f63952410344 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96e3fff640 1 -- 172.21.15.119:0/674116145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96dc0ab530 con 0x7f96ec0936c0 2024-03-28T16:50:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96e3fff640 1 -- 172.21.15.119:0/674116145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc0a7050 con 0x7f96ec0936c0 2024-03-28T16:50:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.250+0000 7f96e3fff640 1 -- 172.21.15.119:0/674116145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96dc0ab7e0 con 0x7f96ec0936c0 2024-03-28T16:50:46.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/674116145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 msgr2=0x7f96ec093aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 --2- 172.21.15.119:0/674116145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec093aa0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009740 tx=0x7f96dc01c370 comp rx=0 tx=0).stop 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/674116145 shutdown_connections 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 --2- 172.21.15.119:0/674116145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec093aa0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/674116145 >> 172.21.15.119:0/674116145 conn(0x7f96ec09a0c0 msgr2=0x7f96ec09a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/674116145 shutdown_connections 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/674116145 wait complete. 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 Processor -- start 2024-03-28T16:50:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- start start 2024-03-28T16:50:46.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec1c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec06fec0 con 0x7f96ec0936c0 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f11eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec1c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f11eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec1c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34234/0 (socket says 172.21.15.119:34234) 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f11eb640 1 -- 172.21.15.119:0/2922992030 learned_addr learned my addr 172.21.15.119:0/2922992030 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f11eb640 1 -- 172.21.15.119:0/2922992030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc009420 con 0x7f96ec0936c0 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f11eb640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec1c56f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f96dc019fa0 tx=0x7f96dc01abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96e1ffb640 1 -- 172.21.15.119:0/2922992030 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96dc0b7450 con 0x7f96ec0936c0 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96e1ffb640 1 -- 172.21.15.119:0/2922992030 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc0b75b0 con 0x7f96ec0936c0 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec092130 con 0x7f96ec0936c0 2024-03-28T16:50:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96e1ffb640 1 -- 172.21.15.119:0/2922992030 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96dc0b7850 con 0x7f96ec0936c0 2024-03-28T16:50:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec071930 con 0x7f96ec0936c0 2024-03-28T16:50:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec071b00 con 0x7f96ec0936c0 2024-03-28T16:50:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96e1ffb640 1 -- 172.21.15.119:0/2922992030 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f96dc0b79b0 con 0x7f96ec0936c0 2024-03-28T16:50:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96e1ffb640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f96d0035a10 0x7f96d0037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96e1ffb640 1 -- 172.21.15.119:0/2922992030 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f96dc0e32d0 con 0x7f96ec0936c0 2024-03-28T16:50:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.254+0000 7f96f09ea640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f96d0035a10 0x7f96d0037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.258+0000 7f96f09ea640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f96d0035a10 0x7f96d0037ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f96e8007fd0 tx=0x7f96e8007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.263 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.258+0000 7f96e1ffb640 1 -- 172.21.15.119:0/2922992030 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96dc0c74e0 con 0x7f96ec0936c0 2024-03-28T16:50:46.409 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.406+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f96ec0943f0 con 0x7f96d0035a10 2024-03-28T16:50:46.410 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.406+0000 7f96e1ffb640 1 -- 172.21.15.119:0/2922992030 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f96e80070d0 con 0x7f96d0035a10 2024-03-28T16:50:46.410 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 2 to deep-scrub 2024-03-28T16:50:46.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f96d0035a10 msgr2=0x7f96d0037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.412 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f96d0035a10 0x7f96d0037ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f96e8007fd0 tx=0x7f96e8007940 comp rx=0 tx=0).stop 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 msgr2=0x7f96ec1c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec1c56f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f96dc019fa0 tx=0x7f96dc01abb0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 shutdown_connections 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f96d0035a10 0x7f96d0037ed0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 --2- 172.21.15.119:0/2922992030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96ec0936c0 0x7f96ec1c56f0 secure :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f96dc019fa0 tx=0x7f96dc01abb0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 >> 172.21.15.119:0/2922992030 conn(0x7f96ec09a0c0 msgr2=0x7f96ec096ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 shutdown_connections 2024-03-28T16:50:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.410+0000 7f96f344f640 1 -- 172.21.15.119:0/2922992030 wait complete. 2024-03-28T16:50:46.427 INFO:tasks.ceph:Scrubbing osd.3 2024-03-28T16:50:46.427 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-28T16:50:46.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a4e2c8640 1 Processor -- start 2024-03-28T16:50:46.550 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a4e2c8640 1 -- start start 2024-03-28T16:50:46.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a4e2c8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a4812bbe0 0x7f9a4812bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a4e2c8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4809c070 con 0x7f9a4812c500 2024-03-28T16:50:46.551 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a4e2c8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a48099e90 con 0x7f9a4812bbe0 2024-03-28T16:50:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a47fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a4812bbe0 0x7f9a4812bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a477fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9a4812c500 0x7f9a48128a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43882/0 (socket says 172.21.15.119:43882) 2024-03-28T16:50:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a47fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a4812bbe0 0x7f9a4812bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34236/0 (socket says 172.21.15.119:34236) 2024-03-28T16:50:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a477fe640 1 -- 172.21.15.119:0/597530343 learned_addr learned my addr 172.21.15.119:0/597530343 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a47fff640 1 -- 172.21.15.119:0/597530343 >> v1:172.21.15.119:6789/0 conn(0x7f9a4812c500 legacy=0x7f9a48128a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.546+0000 7f9a47fff640 1 -- 172.21.15.119:0/597530343 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4809b7f0 con 0x7f9a4812bbe0 2024-03-28T16:50:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a47fff640 1 --2- 172.21.15.119:0/597530343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4812bfc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009d90 tx=0x7f9a3801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bef68f295ea1425 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a46ffd640 1 -- 172.21.15.119:0/597530343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a380a6560 con 0x7f9a4812bbe0 2024-03-28T16:50:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a46ffd640 1 -- 172.21.15.119:0/597530343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a380a66c0 con 0x7f9a4812bbe0 2024-03-28T16:50:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a46ffd640 1 -- 172.21.15.119:0/597530343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a380a6980 con 0x7f9a4812bbe0 2024-03-28T16:50:46.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/597530343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 msgr2=0x7f9a4812bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.555 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/597530343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4812bfc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009d90 tx=0x7f9a3801c8c0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/597530343 shutdown_connections 2024-03-28T16:50:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/597530343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4812bfc0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/597530343 >> 172.21.15.119:0/597530343 conn(0x7f9a4809cfe0 msgr2=0x7f9a4809d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/597530343 shutdown_connections 2024-03-28T16:50:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/597530343 wait complete. 2024-03-28T16:50:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 Processor -- start 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 -- start start 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4806d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a4e2c8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4806c620 con 0x7f9a4812bbe0 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4806d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34248/0 (socket says 172.21.15.119:34248) 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.550+0000 7f9a47fff640 1 -- 172.21.15.119:0/1643574906 learned_addr learned my addr 172.21.15.119:0/1643574906 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a47fff640 1 -- 172.21.15.119:0/1643574906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38009a70 con 0x7f9a4812bbe0 2024-03-28T16:50:46.557 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a47fff640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4806d430 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009d90 tx=0x7f9a38019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.559 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a380a7750 con 0x7f9a4812bbe0 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a380a78b0 con 0x7f9a4812bbe0 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a380a7b70 con 0x7f9a4812bbe0 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4806ed40 con 0x7f9a4812bbe0 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4811bd70 con 0x7f9a4812bbe0 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a48134130 con 0x7f9a4812bbe0 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f9a380a7cd0 con 0x7f9a4812bbe0 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9a34035a10 0x7f9a34037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9a380b7040 con 0x7f9a4812bbe0 2024-03-28T16:50:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f9a34041810 0x7f9a34043cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a34044380 con 0x7f9a34041810 2024-03-28T16:50:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a380e6480 con 0x7f9a4812bbe0 2024-03-28T16:50:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a477fe640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9a34035a10 0x7f9a34037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a4c865640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f9a34041810 0x7f9a34043cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a4c865640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f9a34041810 0x7f9a34043cb0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.554+0000 7f9a477fe640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9a34035a10 0x7f9a34037ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005890 tx=0x7f9a3c007a90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.562 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.558+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== osd.3 v2:172.21.15.170:6816/2102412426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9a34044380 con 0x7f9a34041810 2024-03-28T16:50:46.573 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 --> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a4811b400 con 0x7f9a34041810 2024-03-28T16:50:46.574 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a44ff9640 1 -- 172.21.15.119:0/1643574906 <== osd.3 v2:172.21.15.170:6816/2102412426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9a4811b400 con 0x7f9a34041810 2024-03-28T16:50:46.574 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-03-28T16:50:46.575 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T16:50:46.575 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-03-28T16:50:46.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f9a34041810 msgr2=0x7f9a34043cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f9a34041810 0x7f9a34043cb0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9a34035a10 msgr2=0x7f9a34037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9a34035a10 0x7f9a34037ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005890 tx=0x7f9a3c007a90 comp rx=0 tx=0).stop 2024-03-28T16:50:46.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 msgr2=0x7f9a4806d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.576 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4806d430 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009d90 tx=0x7f9a38019660 comp rx=0 tx=0).stop 2024-03-28T16:50:46.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 shutdown_connections 2024-03-28T16:50:46.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6816/2102412426,v1:172.21.15.170:6817/2102412426] conn(0x7f9a34041810 0x7f9a34043cb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9a34035a10 0x7f9a34037ed0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 --2- 172.21.15.119:0/1643574906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a4812bbe0 0x7f9a4806d430 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 >> 172.21.15.119:0/1643574906 conn(0x7f9a4809cfe0 msgr2=0x7f9a480710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 shutdown_connections 2024-03-28T16:50:46.577 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.570+0000 7f9a4e2c8640 1 -- 172.21.15.119:0/1643574906 wait complete. 2024-03-28T16:50:46.590 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-28T16:50:46.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68b29ec640 1 Processor -- start 2024-03-28T16:50:46.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68b29ec640 1 -- start start 2024-03-28T16:50:46.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68b29ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68ac0966f0 0x7f68ac096ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68b29ec640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac099220 con 0x7f68ac097010 2024-03-28T16:50:46.702 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68b29ec640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac099960 con 0x7f68ac0966f0 2024-03-28T16:50:46.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68abfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68ac0966f0 0x7f68ac096ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68ab7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f68ac097010 0x7f68ac097cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43890/0 (socket says 172.21.15.119:43890) 2024-03-28T16:50:46.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68ab7fe640 1 -- 172.21.15.119:0/320542484 learned_addr learned my addr 172.21.15.119:0/320542484 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:46.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68abfff640 1 -- 172.21.15.119:0/320542484 >> v1:172.21.15.119:6789/0 conn(0x7f68ac097010 legacy=0x7f68ac097cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.703 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.698+0000 7f68abfff640 1 -- 172.21.15.119:0/320542484 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac09a4d0 con 0x7f68ac0966f0 2024-03-28T16:50:46.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68abfff640 1 --2- 172.21.15.119:0/320542484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac096ad0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f689c009740 tx=0x7f689c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84f4dcb5bbe6992 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68aaffd640 1 -- 172.21.15.119:0/320542484 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c0ac550 con 0x7f68ac0966f0 2024-03-28T16:50:46.704 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68aaffd640 1 -- 172.21.15.119:0/320542484 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c0ac6b0 con 0x7f68ac0966f0 2024-03-28T16:50:46.705 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68aaffd640 1 -- 172.21.15.119:0/320542484 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c0ac970 con 0x7f68ac0966f0 2024-03-28T16:50:46.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- 172.21.15.119:0/320542484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 msgr2=0x7f68ac096ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 --2- 172.21.15.119:0/320542484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac096ad0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f689c009740 tx=0x7f689c01c400 comp rx=0 tx=0).stop 2024-03-28T16:50:46.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- 172.21.15.119:0/320542484 shutdown_connections 2024-03-28T16:50:46.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 --2- 172.21.15.119:0/320542484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac096ad0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.707 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- 172.21.15.119:0/320542484 >> 172.21.15.119:0/320542484 conn(0x7f68ac09e540 msgr2=0x7f68ac09e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- 172.21.15.119:0/320542484 shutdown_connections 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- 172.21.15.119:0/320542484 wait complete. 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 Processor -- start 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- start start 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac1c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac133210 con 0x7f68ac0966f0 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68abfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac1c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68abfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac1c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34274/0 (socket says 172.21.15.119:34274) 2024-03-28T16:50:46.708 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68abfff640 1 -- 172.21.15.119:0/3507270987 learned_addr learned my addr 172.21.15.119:0/3507270987 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68abfff640 1 -- 172.21.15.119:0/3507270987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c009420 con 0x7f68ac0966f0 2024-03-28T16:50:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68abfff640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac1c54d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f689c0af040 tx=0x7f689c01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68a8ff9640 1 -- 172.21.15.119:0/3507270987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c0ac550 con 0x7f68ac0966f0 2024-03-28T16:50:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68a8ff9640 1 -- 172.21.15.119:0/3507270987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c0b8450 con 0x7f68ac0966f0 2024-03-28T16:50:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac133e20 con 0x7f68ac0966f0 2024-03-28T16:50:46.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68a8ff9640 1 -- 172.21.15.119:0/3507270987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c0b8710 con 0x7f68ac0966f0 2024-03-28T16:50:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.702+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac133400 con 0x7f68ac0966f0 2024-03-28T16:50:46.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.706+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac0747d0 con 0x7f68ac0966f0 2024-03-28T16:50:46.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.706+0000 7f68a8ff9640 1 -- 172.21.15.119:0/3507270987 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f689c0aa070 con 0x7f68ac0966f0 2024-03-28T16:50:46.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.706+0000 7f68a8ff9640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6898035a10 0x7f6898037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:46.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.706+0000 7f68a8ff9640 1 -- 172.21.15.119:0/3507270987 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f689c0e2cf0 con 0x7f68ac0966f0 2024-03-28T16:50:46.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.706+0000 7f68ab7fe640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6898035a10 0x7f6898037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:46.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.706+0000 7f68ab7fe640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6898035a10 0x7f6898037ed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68a0001610 tx=0x7f68a0007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:46.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.710+0000 7f68a8ff9640 1 -- 172.21.15.119:0/3507270987 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f689c0c8710 con 0x7f68ac0966f0 2024-03-28T16:50:46.868 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f68ac1322d0 con 0x7f6898035a10 2024-03-28T16:50:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68a8ff9640 1 -- 172.21.15.119:0/3507270987 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f68a0001300 con 0x7f6898035a10 2024-03-28T16:50:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 3 to deep-scrub 2024-03-28T16:50:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6898035a10 msgr2=0x7f6898037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6898035a10 0x7f6898037ed0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68a0001610 tx=0x7f68a0007ad0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 msgr2=0x7f68ac1c54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac1c54d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f689c0af040 tx=0x7f689c01ac40 comp rx=0 tx=0).stop 2024-03-28T16:50:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 shutdown_connections 2024-03-28T16:50:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f6898035a10 0x7f6898037ed0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 --2- 172.21.15.119:0/3507270987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68ac0966f0 0x7f68ac1c54d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 >> 172.21.15.119:0/3507270987 conn(0x7f68ac09e540 msgr2=0x7f68ac12dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 shutdown_connections 2024-03-28T16:50:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:46.866+0000 7f68b29ec640 1 -- 172.21.15.119:0/3507270987 wait complete. 2024-03-28T16:50:46.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:50:47.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.002+0000 7fbfb6350640 1 Processor -- start 2024-03-28T16:50:47.007 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.002+0000 7fbfb6350640 1 -- start start 2024-03-28T16:50:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbfb01290f0 0x7fbfb012c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb0095ee0 con 0x7fbfb0128d10 2024-03-28T16:50:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb0096320 con 0x7fbfb01290f0 2024-03-28T16:50:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaf7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbfb01290f0 0x7fbfb012c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:47.009 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbfb0128d10 0x7fbfb012bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43904/0 (socket says 172.21.15.119:43904) 2024-03-28T16:50:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaffff640 1 -- 172.21.15.119:0/3234697353 learned_addr learned my addr 172.21.15.119:0/3234697353 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:50:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaeffd640 1 -- 172.21.15.119:0/3234697353 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184658968 0 0) 0x7fbfb0095ee0 con 0x7fbfb0128d10 2024-03-28T16:50:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaeffd640 1 -- 172.21.15.119:0/3234697353 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf9c003530 con 0x7fbfb0128d10 2024-03-28T16:50:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaf7fe640 1 -- 172.21.15.119:0/3234697353 >> v1:172.21.15.119:6789/0 conn(0x7fbfb0128d10 legacy=0x7fbfb012bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:47.010 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaf7fe640 1 -- 172.21.15.119:0/3234697353 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb0128620 con 0x7fbfb01290f0 2024-03-28T16:50:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaf7fe640 1 --2- 172.21.15.119:0/3234697353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01290f0 0x7fbfb012c230 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009d90 tx=0x7fbfa001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b083bfe27160774 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaeffd640 1 -- 172.21.15.119:0/3234697353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfa00a66b0 con 0x7fbfb01290f0 2024-03-28T16:50:47.011 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaeffd640 1 -- 172.21.15.119:0/3234697353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa00a6810 con 0x7fbfb01290f0 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfaeffd640 1 -- 172.21.15.119:0/3234697353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfa00a6ab0 con 0x7fbfb01290f0 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 -- 172.21.15.119:0/3234697353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01290f0 msgr2=0x7fbfb012c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 --2- 172.21.15.119:0/3234697353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01290f0 0x7fbfb012c230 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009d90 tx=0x7fbfa001c9e0 comp rx=0 tx=0).stop 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 -- 172.21.15.119:0/3234697353 shutdown_connections 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 --2- 172.21.15.119:0/3234697353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb01290f0 0x7fbfb012c230 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 -- 172.21.15.119:0/3234697353 >> 172.21.15.119:0/3234697353 conn(0x7fbfb009a2f0 msgr2=0x7fbfb009a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 -- 172.21.15.119:0/3234697353 shutdown_connections 2024-03-28T16:50:47.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.006+0000 7fbfb6350640 1 -- 172.21.15.119:0/3234697353 wait complete. 2024-03-28T16:50:47.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfb6350640 1 Processor -- start 2024-03-28T16:50:47.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfb6350640 1 -- start start 2024-03-28T16:50:47.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfb6350640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb0128d10 0x7fbfb006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:47.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfb6350640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb012ed20 con 0x7fbfb0128d10 2024-03-28T16:50:47.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfaffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb0128d10 0x7fbfb006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfaffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb0128d10 0x7fbfb006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34292/0 (socket says 172.21.15.119:34292) 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfaffff640 1 -- 172.21.15.119:0/607536660 learned_addr learned my addr 172.21.15.119:0/607536660 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfaffff640 1 -- 172.21.15.119:0/607536660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0009a70 con 0x7fbfb0128d10 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfaffff640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb0128d10 0x7fbfb006c1e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c3d0 tx=0x7fbf9800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfacff9640 1 -- 172.21.15.119:0/607536660 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf98005ce0 con 0x7fbfb0128d10 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfacff9640 1 -- 172.21.15.119:0/607536660 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98005e40 con 0x7fbfb0128d10 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb012f030 con 0x7fbfb0128d10 2024-03-28T16:50:47.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfacff9640 1 -- 172.21.15.119:0/607536660 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf98007220 con 0x7fbfb0128d10 2024-03-28T16:50:47.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb006dcf0 con 0x7fbfb0128d10 2024-03-28T16:50:47.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfacff9640 1 -- 172.21.15.119:0/607536660 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fbf98007440 con 0x7fbfb0128d10 2024-03-28T16:50:47.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfacff9640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf900359c0 0x7fbf90037e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:50:47.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb012eb70 con 0x7fbfb0128d10 2024-03-28T16:50:47.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfacff9640 1 -- 172.21.15.119:0/607536660 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fbf98047d50 con 0x7fbfb0128d10 2024-03-28T16:50:47.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfaf7fe640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf900359c0 0x7fbf90037e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:50:47.017 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.010+0000 7fbfaf7fe640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf900359c0 0x7fbf90037e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00b0000 tx=0x7fbfa00a8070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:50:47.018 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.014+0000 7fbfacff9640 1 -- 172.21.15.119:0/607536660 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf98019be0 con 0x7fbfb0128d10 2024-03-28T16:50:47.173 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.170+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfb0095170 con 0x7fbf900359c0 2024-03-28T16:50:47.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.218+0000 7fbfacff9640 1 -- 172.21.15.119:0/607536660 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+673162 (secure 0 0 0) 0x7fbfa0003b20 con 0x7fbf900359c0 2024-03-28T16:50:47.222 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:50:47.232 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:50:47.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf900359c0 msgr2=0x7fbf90037e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:47.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf900359c0 0x7fbf90037e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00b0000 tx=0x7fbfa00a8070 comp rx=0 tx=0).stop 2024-03-28T16:50:47.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb0128d10 msgr2=0x7fbfb006c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:50:47.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb0128d10 0x7fbfb006c1e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800c3d0 tx=0x7fbf9800c890 comp rx=0 tx=0).stop 2024-03-28T16:50:47.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 shutdown_connections 2024-03-28T16:50:47.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fbf900359c0 0x7fbf90037e80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:47.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 --2- 172.21.15.119:0/607536660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfb0128d10 0x7fbfb006c1e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:50:47.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 >> 172.21.15.119:0/607536660 conn(0x7fbfb009a2f0 msgr2=0x7fbfb0121940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:50:47.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 shutdown_connections 2024-03-28T16:50:47.234 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:50:47.226+0000 7fbfb6350640 1 -- 172.21.15.119:0/607536660 wait complete. 2024-03-28T16:50:47.252 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-03-28T16:50:46.535866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":3912,"num_read_kb":3559,"num_write":1505,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3798,"ondisk_log_size":3798,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":160768,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":157945,"kb_avail":377326592,"statfs":{"total":386547056640,"available":386382430208,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":161736470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":521,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":52,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006186"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600005+0000","last_change":"2024-03-28T16:48:54.656021+0000","last_active":"2024-03-28T16:49:37.600005+0000","last_peered":"2024-03-28T16:49:37.600005+0000","last_clean":"2024-03-28T16:49:37.600005+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:49:37.600005+0000","last_undegraded":"2024-03-28T16:49:37.600005+0000","last_fullsized":"2024-03-28T16:49:37.600005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585246+0000","last_change":"2024-03-28T16:48:56.687475+0000","last_active":"2024-03-28T16:49:37.585246+0000","last_peered":"2024-03-28T16:49:37.585246+0000","last_clean":"2024-03-28T16:49:37.585246+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:49:37.585246+0000","last_undegraded":"2024-03-28T16:49:37.585246+0000","last_fullsized":"2024-03-28T16:49:37.585246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587869+0000","last_change":"2024-03-28T16:48:54.666372+0000","last_active":"2024-03-28T16:49:37.587869+0000","last_peered":"2024-03-28T16:49:37.587869+0000","last_clean":"2024-03-28T16:49:37.587869+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:49:37.587869+0000","last_undegraded":"2024-03-28T16:49:37.587869+0000","last_fullsized":"2024-03-28T16:49:37.587869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555560+0000","last_change":"2024-03-28T16:48:56.688050+0000","last_active":"2024-03-28T16:49:40.555560+0000","last_peered":"2024-03-28T16:49:40.555560+0000","last_clean":"2024-03-28T16:49:40.555560+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:49:40.555560+0000","last_undegraded":"2024-03-28T16:49:40.555560+0000","last_fullsized":"2024-03-28T16:49:40.555560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:32.377984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592873+0000","last_change":"2024-03-28T16:48:54.665976+0000","last_active":"2024-03-28T16:49:37.592873+0000","last_peered":"2024-03-28T16:49:37.592873+0000","last_clean":"2024-03-28T16:49:37.592873+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:49:37.592873+0000","last_undegraded":"2024-03-28T16:49:37.592873+0000","last_fullsized":"2024-03-28T16:49:37.592873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:38.265023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370127+0000","last_change":"2024-03-28T16:48:56.690734+0000","last_active":"2024-03-28T16:50:22.370127+0000","last_peered":"2024-03-28T16:50:22.370127+0000","last_clean":"2024-03-28T16:50:22.370127+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:50:22.370127+0000","last_undegraded":"2024-03-28T16:50:22.370127+0000","last_fullsized":"2024-03-28T16:50:22.370127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:26.690845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.374097+0000","last_change":"2024-03-28T16:48:54.656958+0000","last_active":"2024-03-28T16:50:22.374097+0000","last_peered":"2024-03-28T16:50:22.374097+0000","last_clean":"2024-03-28T16:50:22.374097+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:50:22.374097+0000","last_undegraded":"2024-03-28T16:50:22.374097+0000","last_fullsized":"2024-03-28T16:50:22.374097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.564230+0000","last_change":"2024-03-28T16:49:37.602262+0000","last_active":"2024-03-28T16:49:40.564230+0000","last_peered":"2024-03-28T16:49:40.564230+0000","last_clean":"2024-03-28T16:49:40.564230+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:49:40.564230+0000","last_undegraded":"2024-03-28T16:49:40.564230+0000","last_fullsized":"2024-03-28T16:49:40.564230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:12:52.595685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593536+0000","last_change":"2024-03-28T16:48:54.661971+0000","last_active":"2024-03-28T16:49:37.593536+0000","last_peered":"2024-03-28T16:49:37.593536+0000","last_clean":"2024-03-28T16:49:37.593536+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:49:37.593536+0000","last_undegraded":"2024-03-28T16:49:37.593536+0000","last_fullsized":"2024-03-28T16:49:37.593536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:29.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585408+0000","last_change":"2024-03-28T16:48:56.671464+0000","last_active":"2024-03-28T16:49:37.585408+0000","last_peered":"2024-03-28T16:49:37.585408+0000","last_clean":"2024-03-28T16:49:37.585408+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:49:37.585408+0000","last_undegraded":"2024-03-28T16:49:37.585408+0000","last_fullsized":"2024-03-28T16:49:37.585408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:18.173627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590098+0000","last_change":"2024-03-28T16:48:54.665454+0000","last_active":"2024-03-28T16:49:37.590098+0000","last_peered":"2024-03-28T16:49:37.590098+0000","last_clean":"2024-03-28T16:49:37.590098+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:49:37.590098+0000","last_undegraded":"2024-03-28T16:49:37.590098+0000","last_fullsized":"2024-03-28T16:49:37.590098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587928+0000","last_change":"2024-03-28T16:48:56.688375+0000","last_active":"2024-03-28T16:49:37.587928+0000","last_peered":"2024-03-28T16:49:37.587928+0000","last_clean":"2024-03-28T16:49:37.587928+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:49:37.587928+0000","last_undegraded":"2024-03-28T16:49:37.587928+0000","last_fullsized":"2024-03-28T16:49:37.587928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:56.236071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.859167+0000","last_change":"2024-03-28T16:48:54.672203+0000","last_active":"2024-03-28T16:49:39.859167+0000","last_peered":"2024-03-28T16:49:39.859167+0000","last_clean":"2024-03-28T16:49:39.859167+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:49:39.859167+0000","last_undegraded":"2024-03-28T16:49:39.859167+0000","last_fullsized":"2024-03-28T16:49:39.859167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:04:05.939269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555495+0000","last_change":"2024-03-28T16:48:56.695671+0000","last_active":"2024-03-28T16:49:40.555495+0000","last_peered":"2024-03-28T16:49:40.555495+0000","last_clean":"2024-03-28T16:49:40.555495+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:49:40.555495+0000","last_undegraded":"2024-03-28T16:49:40.555495+0000","last_fullsized":"2024-03-28T16:49:40.555495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585143+0000","last_change":"2024-03-28T16:48:54.658053+0000","last_active":"2024-03-28T16:49:37.585143+0000","last_peered":"2024-03-28T16:49:37.585143+0000","last_clean":"2024-03-28T16:49:37.585143+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:49:37.585143+0000","last_undegraded":"2024-03-28T16:49:37.585143+0000","last_fullsized":"2024-03-28T16:49:37.585143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839573+0000","last_change":"2024-03-28T16:48:56.681114+0000","last_active":"2024-03-28T16:50:28.839573+0000","last_peered":"2024-03-28T16:50:28.839573+0000","last_clean":"2024-03-28T16:50:28.839573+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:50:28.839573+0000","last_undegraded":"2024-03-28T16:50:28.839573+0000","last_fullsized":"2024-03-28T16:50:28.839573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527185+0000","last_change":"2024-03-28T16:48:56.677244+0000","last_active":"2024-03-28T16:49:39.527185+0000","last_peered":"2024-03-28T16:49:39.527185+0000","last_clean":"2024-03-28T16:49:39.527185+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:49:39.527185+0000","last_undegraded":"2024-03-28T16:49:39.527185+0000","last_fullsized":"2024-03-28T16:49:39.527185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:55.139244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602851+0000","last_change":"2024-03-28T16:48:54.671518+0000","last_active":"2024-03-28T16:49:37.602851+0000","last_peered":"2024-03-28T16:49:37.602851+0000","last_clean":"2024-03-28T16:49:37.602851+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:49:37.602851+0000","last_undegraded":"2024-03-28T16:49:37.602851+0000","last_fullsized":"2024-03-28T16:49:37.602851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'9","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.563949+0000","last_change":"2024-03-28T16:48:56.692955+0000","last_active":"2024-03-28T16:49:40.563949+0000","last_peered":"2024-03-28T16:49:40.563949+0000","last_clean":"2024-03-28T16:49:40.563949+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:49:40.563949+0000","last_undegraded":"2024-03-28T16:49:40.563949+0000","last_fullsized":"2024-03-28T16:49:40.563949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610918+0000","last_change":"2024-03-28T16:49:37.610918+0000","last_active":"2024-03-28T16:49:37.610918+0000","last_peered":"2024-03-28T16:49:37.610918+0000","last_clean":"2024-03-28T16:49:37.610918+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:49:37.610918+0000","last_undegraded":"2024-03-28T16:49:37.610918+0000","last_fullsized":"2024-03-28T16:49:37.610918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:21.127441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840116+0000","last_change":"2024-03-28T16:48:56.687940+0000","last_active":"2024-03-28T16:50:28.840116+0000","last_peered":"2024-03-28T16:50:28.840116+0000","last_clean":"2024-03-28T16:50:28.840116+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:50:28.840116+0000","last_undegraded":"2024-03-28T16:50:28.840116+0000","last_fullsized":"2024-03-28T16:50:28.840116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:40:18.735480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586038+0000","last_change":"2024-03-28T16:48:54.670148+0000","last_active":"2024-03-28T16:49:37.586038+0000","last_peered":"2024-03-28T16:49:37.586038+0000","last_clean":"2024-03-28T16:49:37.586038+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:49:37.586038+0000","last_undegraded":"2024-03-28T16:49:37.586038+0000","last_fullsized":"2024-03-28T16:49:37.586038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:49.487203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.865044+0000","last_change":"2024-03-28T16:48:56.678642+0000","last_active":"2024-03-28T16:49:39.865044+0000","last_peered":"2024-03-28T16:49:39.865044+0000","last_clean":"2024-03-28T16:49:39.865044+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:49:39.865044+0000","last_undegraded":"2024-03-28T16:49:39.865044+0000","last_fullsized":"2024-03-28T16:49:39.865044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586134+0000","last_change":"2024-03-28T16:48:54.655014+0000","last_active":"2024-03-28T16:49:37.586134+0000","last_peered":"2024-03-28T16:49:37.586134+0000","last_clean":"2024-03-28T16:49:37.586134+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:49:37.586134+0000","last_undegraded":"2024-03-28T16:49:37.586134+0000","last_fullsized":"2024-03-28T16:49:37.586134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.876760+0000","last_change":"2024-03-28T16:48:56.680884+0000","last_active":"2024-03-28T16:49:39.876760+0000","last_peered":"2024-03-28T16:49:39.876760+0000","last_clean":"2024-03-28T16:49:39.876760+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:49:39.876760+0000","last_undegraded":"2024-03-28T16:49:39.876760+0000","last_fullsized":"2024-03-28T16:49:39.876760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.842066+0000","last_change":"2024-03-28T16:48:54.670541+0000","last_active":"2024-03-28T16:49:39.842066+0000","last_peered":"2024-03-28T16:49:39.842066+0000","last_clean":"2024-03-28T16:49:39.842066+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:49:39.842066+0000","last_undegraded":"2024-03-28T16:49:39.842066+0000","last_fullsized":"2024-03-28T16:49:39.842066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854794+0000","last_change":"2024-03-28T16:48:56.671588+0000","last_active":"2024-03-28T16:50:28.854794+0000","last_peered":"2024-03-28T16:50:28.854794+0000","last_clean":"2024-03-28T16:50:28.854794+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:50:28.854794+0000","last_undegraded":"2024-03-28T16:50:28.854794+0000","last_fullsized":"2024-03-28T16:50:28.854794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:22.649572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582658+0000","last_change":"2024-03-28T16:48:54.673730+0000","last_active":"2024-03-28T16:49:37.582658+0000","last_peered":"2024-03-28T16:49:37.582658+0000","last_clean":"2024-03-28T16:49:37.582658+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:49:37.582658+0000","last_undegraded":"2024-03-28T16:49:37.582658+0000","last_fullsized":"2024-03-28T16:49:37.582658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:36:08.968340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'19","reported_seq":77,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.826736+0000","last_change":"2024-03-28T16:49:04.742579+0000","last_active":"2024-03-28T16:50:28.826736+0000","last_peered":"2024-03-28T16:50:28.826736+0000","last_clean":"2024-03-28T16:50:28.826736+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:50:28.826736+0000","last_undegraded":"2024-03-28T16:50:28.826736+0000","last_fullsized":"2024-03-28T16:50:28.826736+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:00.654089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839392+0000","last_change":"2024-03-28T16:48:56.683934+0000","last_active":"2024-03-28T16:50:28.839392+0000","last_peered":"2024-03-28T16:50:28.839392+0000","last_clean":"2024-03-28T16:50:28.839392+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:50:28.839392+0000","last_undegraded":"2024-03-28T16:50:28.839392+0000","last_fullsized":"2024-03-28T16:50:28.839392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:38.624478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592554+0000","last_change":"2024-03-28T16:48:54.670023+0000","last_active":"2024-03-28T16:49:37.592554+0000","last_peered":"2024-03-28T16:49:37.592554+0000","last_clean":"2024-03-28T16:49:37.592554+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:49:37.592554+0000","last_undegraded":"2024-03-28T16:49:37.592554+0000","last_fullsized":"2024-03-28T16:49:37.592554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:48.828596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587566+0000","last_change":"2024-03-28T16:48:58.699910+0000","last_active":"2024-03-28T16:49:37.587566+0000","last_peered":"2024-03-28T16:49:37.587566+0000","last_clean":"2024-03-28T16:49:37.587566+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:49:37.587566+0000","last_undegraded":"2024-03-28T16:49:37.587566+0000","last_fullsized":"2024-03-28T16:49:37.587566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:35.076776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.149453+0000","last_change":"2024-03-28T16:49:00.727669+0000","last_active":"2024-03-28T16:50:23.149453+0000","last_peered":"2024-03-28T16:50:23.149453+0000","last_clean":"2024-03-28T16:50:23.149453+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:50:23.149453+0000","last_undegraded":"2024-03-28T16:50:23.149453+0000","last_fullsized":"2024-03-28T16:50:23.149453+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:28.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601464+0000","last_change":"2024-03-28T16:49:02.739439+0000","last_active":"2024-03-28T16:49:37.601464+0000","last_peered":"2024-03-28T16:49:37.601464+0000","last_clean":"2024-03-28T16:49:37.601464+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:49:37.601464+0000","last_undegraded":"2024-03-28T16:49:37.601464+0000","last_fullsized":"2024-03-28T16:49:37.601464+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'6","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.307391+0000","last_change":"2024-03-28T16:49:04.742984+0000","last_active":"2024-03-28T16:50:17.307391+0000","last_peered":"2024-03-28T16:50:17.307391+0000","last_clean":"2024-03-28T16:50:17.307391+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:50:17.307391+0000","last_undegraded":"2024-03-28T16:50:17.307391+0000","last_fullsized":"2024-03-28T16:50:17.307391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:30.200369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839461+0000","last_change":"2024-03-28T16:48:56.671488+0000","last_active":"2024-03-28T16:50:28.839461+0000","last_peered":"2024-03-28T16:50:28.839461+0000","last_clean":"2024-03-28T16:50:28.839461+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:50:28.839461+0000","last_undegraded":"2024-03-28T16:50:28.839461+0000","last_fullsized":"2024-03-28T16:50:28.839461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:39.819684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591228+0000","last_change":"2024-03-28T16:48:54.655701+0000","last_active":"2024-03-28T16:49:37.591228+0000","last_peered":"2024-03-28T16:49:37.591228+0000","last_clean":"2024-03-28T16:49:37.591228+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:49:37.591228+0000","last_undegraded":"2024-03-28T16:49:37.591228+0000","last_fullsized":"2024-03-28T16:49:37.591228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585880+0000","last_change":"2024-03-28T16:48:58.689758+0000","last_active":"2024-03-28T16:49:37.585880+0000","last_peered":"2024-03-28T16:49:37.585880+0000","last_clean":"2024-03-28T16:49:37.585880+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:49:37.585880+0000","last_undegraded":"2024-03-28T16:49:37.585880+0000","last_fullsized":"2024-03-28T16:49:37.585880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:08.600334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"29'7","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.605037+0000","last_change":"2024-03-28T16:49:00.717460+0000","last_active":"2024-03-28T16:49:37.605037+0000","last_peered":"2024-03-28T16:49:37.605037+0000","last_clean":"2024-03-28T16:49:37.605037+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:49:37.605037+0000","last_undegraded":"2024-03-28T16:49:37.605037+0000","last_fullsized":"2024-03-28T16:49:37.605037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:26.416340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586231+0000","last_change":"2024-03-28T16:49:02.730400+0000","last_active":"2024-03-28T16:49:37.586231+0000","last_peered":"2024-03-28T16:49:37.586231+0000","last_clean":"2024-03-28T16:49:37.586231+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:49:37.586231+0000","last_undegraded":"2024-03-28T16:49:37.586231+0000","last_fullsized":"2024-03-28T16:49:37.586231+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584337+0000","last_change":"2024-03-28T16:49:04.743161+0000","last_active":"2024-03-28T16:49:37.584337+0000","last_peered":"2024-03-28T16:49:37.584337+0000","last_clean":"2024-03-28T16:49:37.584337+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:49:37.584337+0000","last_undegraded":"2024-03-28T16:49:37.584337+0000","last_fullsized":"2024-03-28T16:49:37.584337+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189874+0000","last_change":"2024-03-28T16:48:56.676331+0000","last_active":"2024-03-28T16:50:23.189874+0000","last_peered":"2024-03-28T16:50:23.189874+0000","last_clean":"2024-03-28T16:50:23.189874+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:50:23.189874+0000","last_undegraded":"2024-03-28T16:50:23.189874+0000","last_fullsized":"2024-03-28T16:50:23.189874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:33.697214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592955+0000","last_change":"2024-03-28T16:48:54.659330+0000","last_active":"2024-03-28T16:49:37.592955+0000","last_peered":"2024-03-28T16:49:37.592955+0000","last_clean":"2024-03-28T16:49:37.592955+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:49:37.592955+0000","last_undegraded":"2024-03-28T16:49:37.592955+0000","last_fullsized":"2024-03-28T16:49:37.592955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:30:54.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'5","reported_seq":203,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.364471+0000","last_change":"2024-03-28T16:48:58.687156+0000","last_active":"2024-03-28T16:50:30.364471+0000","last_peered":"2024-03-28T16:50:30.364471+0000","last_clean":"2024-03-28T16:50:30.364471+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:50:30.364471+0000","last_undegraded":"2024-03-28T16:50:30.364471+0000","last_fullsized":"2024-03-28T16:50:30.364471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:25:52.416012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":267,"num_read_kb":266,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'11","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595817+0000","last_change":"2024-03-28T16:49:00.704257+0000","last_active":"2024-03-28T16:49:37.595817+0000","last_peered":"2024-03-28T16:49:37.595817+0000","last_clean":"2024-03-28T16:49:37.595817+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:49:37.595817+0000","last_undegraded":"2024-03-28T16:49:37.595817+0000","last_fullsized":"2024-03-28T16:49:37.595817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:59.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'246","reported_seq":329,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854124+0000","last_change":"2024-03-28T16:49:02.737451+0000","last_active":"2024-03-28T16:50:33.854124+0000","last_peered":"2024-03-28T16:50:33.854124+0000","last_clean":"2024-03-28T16:50:33.854124+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:50:33.854124+0000","last_undegraded":"2024-03-28T16:50:33.854124+0000","last_fullsized":"2024-03-28T16:50:33.854124+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:34.133910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591631+0000","last_change":"2024-03-28T16:49:04.741486+0000","last_active":"2024-03-28T16:49:37.591631+0000","last_peered":"2024-03-28T16:49:37.591631+0000","last_clean":"2024-03-28T16:49:37.591631+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:49:37.591631+0000","last_undegraded":"2024-03-28T16:49:37.591631+0000","last_fullsized":"2024-03-28T16:49:37.591631+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:32:51.610298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591048+0000","last_change":"2024-03-28T16:48:56.697774+0000","last_active":"2024-03-28T16:49:37.591048+0000","last_peered":"2024-03-28T16:49:37.591048+0000","last_clean":"2024-03-28T16:49:37.591048+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:49:37.591048+0000","last_undegraded":"2024-03-28T16:49:37.591048+0000","last_fullsized":"2024-03-28T16:49:37.591048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583578+0000","last_change":"2024-03-28T16:48:54.659035+0000","last_active":"2024-03-28T16:49:37.583578+0000","last_peered":"2024-03-28T16:49:37.583578+0000","last_clean":"2024-03-28T16:49:37.583578+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:49:37.583578+0000","last_undegraded":"2024-03-28T16:49:37.583578+0000","last_fullsized":"2024-03-28T16:49:37.583578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:59.516240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594556+0000","last_change":"2024-03-28T16:48:58.706776+0000","last_active":"2024-03-28T16:49:37.594556+0000","last_peered":"2024-03-28T16:49:37.594556+0000","last_clean":"2024-03-28T16:49:37.594556+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:49:37.594556+0000","last_undegraded":"2024-03-28T16:49:37.594556+0000","last_fullsized":"2024-03-28T16:49:37.594556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:18.066504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'31","reported_seq":76,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.220247+0000","last_change":"2024-03-28T16:49:00.710996+0000","last_active":"2024-03-28T16:50:42.220247+0000","last_peered":"2024-03-28T16:50:42.220247+0000","last_clean":"2024-03-28T16:50:42.220247+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:42.220247+0000","last_undegraded":"2024-03-28T16:50:42.220247+0000","last_fullsized":"2024-03-28T16:50:42.220247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:05.450361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'246","reported_seq":344,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.481011+0000","last_change":"2024-03-28T16:49:02.734376+0000","last_active":"2024-03-28T16:50:35.481011+0000","last_peered":"2024-03-28T16:50:35.481011+0000","last_clean":"2024-03-28T16:50:35.481011+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:50:35.481011+0000","last_undegraded":"2024-03-28T16:50:35.481011+0000","last_fullsized":"2024-03-28T16:50:35.481011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:38:03.109809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581627+0000","last_change":"2024-03-28T16:48:56.693376+0000","last_active":"2024-03-28T16:49:37.581627+0000","last_peered":"2024-03-28T16:49:37.581627+0000","last_clean":"2024-03-28T16:49:37.581627+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:49:37.581627+0000","last_undegraded":"2024-03-28T16:49:37.581627+0000","last_fullsized":"2024-03-28T16:49:37.581627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:27.504691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600235+0000","last_change":"2024-03-28T16:48:54.671982+0000","last_active":"2024-03-28T16:49:37.600235+0000","last_peered":"2024-03-28T16:49:37.600235+0000","last_clean":"2024-03-28T16:49:37.600235+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:49:37.600235+0000","last_undegraded":"2024-03-28T16:49:37.600235+0000","last_fullsized":"2024-03-28T16:49:37.600235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586619+0000","last_change":"2024-03-28T16:48:58.687997+0000","last_active":"2024-03-28T16:49:37.586619+0000","last_peered":"2024-03-28T16:49:37.586619+0000","last_clean":"2024-03-28T16:49:37.586619+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:49:37.586619+0000","last_undegraded":"2024-03-28T16:49:37.586619+0000","last_fullsized":"2024-03-28T16:49:37.586619+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:29.519686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'41","reported_seq":91,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.284581+0000","last_change":"2024-03-28T16:49:00.713077+0000","last_active":"2024-03-28T16:50:42.284581+0000","last_peered":"2024-03-28T16:50:42.284581+0000","last_clean":"2024-03-28T16:50:42.284581+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:42.284581+0000","last_undegraded":"2024-03-28T16:50:42.284581+0000","last_fullsized":"2024-03-28T16:50:42.284581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:46.727245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587114+0000","last_change":"2024-03-28T16:49:02.730507+0000","last_active":"2024-03-28T16:49:37.587114+0000","last_peered":"2024-03-28T16:49:37.587114+0000","last_clean":"2024-03-28T16:49:37.587114+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:49:37.587114+0000","last_undegraded":"2024-03-28T16:49:37.587114+0000","last_fullsized":"2024-03-28T16:49:37.587114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.935069+0000","last_change":"2024-03-28T16:49:04.728316+0000","last_active":"2024-03-28T16:50:22.935069+0000","last_peered":"2024-03-28T16:50:22.935069+0000","last_clean":"2024-03-28T16:50:22.935069+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:22.935069+0000","last_undegraded":"2024-03-28T16:50:22.935069+0000","last_fullsized":"2024-03-28T16:50:22.935069+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:28.460966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'22","reported_seq":100,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729800+0000","last_change":"2024-03-28T16:48:56.678278+0000","last_active":"2024-03-28T16:50:22.729800+0000","last_peered":"2024-03-28T16:50:22.729800+0000","last_clean":"2024-03-28T16:50:22.729800+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:50:22.729800+0000","last_undegraded":"2024-03-28T16:50:22.729800+0000","last_fullsized":"2024-03-28T16:50:22.729800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:45.750100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.051186+0000","last_change":"2024-03-28T16:48:54.672277+0000","last_active":"2024-03-28T16:49:39.051186+0000","last_peered":"2024-03-28T16:49:39.051186+0000","last_clean":"2024-03-28T16:49:39.051186+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:49:39.051186+0000","last_undegraded":"2024-03-28T16:49:39.051186+0000","last_fullsized":"2024-03-28T16:49:39.051186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:17.303339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585905+0000","last_change":"2024-03-28T16:48:58.690263+0000","last_active":"2024-03-28T16:49:37.585905+0000","last_peered":"2024-03-28T16:49:37.585905+0000","last_clean":"2024-03-28T16:49:37.585905+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:49:37.585905+0000","last_undegraded":"2024-03-28T16:49:37.585905+0000","last_fullsized":"2024-03-28T16:49:37.585905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:25.224033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.129187+0000","last_change":"2024-03-28T16:49:00.727195+0000","last_active":"2024-03-28T16:50:23.129187+0000","last_peered":"2024-03-28T16:50:23.129187+0000","last_clean":"2024-03-28T16:50:23.129187+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:50:23.129187+0000","last_undegraded":"2024-03-28T16:50:23.129187+0000","last_fullsized":"2024-03-28T16:50:23.129187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:48.531527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585871+0000","last_change":"2024-03-28T16:49:02.714319+0000","last_active":"2024-03-28T16:49:37.585871+0000","last_peered":"2024-03-28T16:49:37.585871+0000","last_clean":"2024-03-28T16:49:37.585871+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:49:37.585871+0000","last_undegraded":"2024-03-28T16:49:37.585871+0000","last_fullsized":"2024-03-28T16:49:37.585871+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:49.522623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'3","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:17.157047+0000","last_change":"2024-03-28T16:49:04.738375+0000","last_active":"2024-03-28T16:50:17.157047+0000","last_peered":"2024-03-28T16:50:17.157047+0000","last_clean":"2024-03-28T16:50:17.157047+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:50:17.157047+0000","last_undegraded":"2024-03-28T16:50:17.157047+0000","last_fullsized":"2024-03-28T16:50:17.157047+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:30:31.198874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'3","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.537250+0000","last_change":"2024-03-28T16:48:56.681600+0000","last_active":"2024-03-28T16:49:39.537250+0000","last_peered":"2024-03-28T16:49:39.537250+0000","last_clean":"2024-03-28T16:49:39.537250+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:49:39.537250+0000","last_undegraded":"2024-03-28T16:49:39.537250+0000","last_fullsized":"2024-03-28T16:49:39.537250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588203+0000","last_change":"2024-03-28T16:48:54.676553+0000","last_active":"2024-03-28T16:49:37.588203+0000","last_peered":"2024-03-28T16:49:37.588203+0000","last_clean":"2024-03-28T16:49:37.588203+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:49:37.588203+0000","last_undegraded":"2024-03-28T16:49:37.588203+0000","last_fullsized":"2024-03-28T16:49:37.588203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:11.709923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592044+0000","last_change":"2024-03-28T16:48:58.682276+0000","last_active":"2024-03-28T16:49:37.592044+0000","last_peered":"2024-03-28T16:49:37.592044+0000","last_clean":"2024-03-28T16:49:37.592044+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:49:37.592044+0000","last_undegraded":"2024-03-28T16:49:37.592044+0000","last_fullsized":"2024-03-28T16:49:37.592044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:44.713820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.158232+0000","last_change":"2024-03-28T16:49:00.720840+0000","last_active":"2024-03-28T16:50:23.158232+0000","last_peered":"2024-03-28T16:50:23.158232+0000","last_clean":"2024-03-28T16:50:23.158232+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:50:23.158232+0000","last_undegraded":"2024-03-28T16:50:23.158232+0000","last_fullsized":"2024-03-28T16:50:23.158232+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:28.746453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'246","reported_seq":339,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854481+0000","last_change":"2024-03-28T16:49:02.713412+0000","last_active":"2024-03-28T16:50:33.854481+0000","last_peered":"2024-03-28T16:50:33.854481+0000","last_clean":"2024-03-28T16:50:33.854481+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:33.854481+0000","last_undegraded":"2024-03-28T16:50:33.854481+0000","last_fullsized":"2024-03-28T16:50:33.854481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.828873+0000","last_change":"2024-03-28T16:49:04.743207+0000","last_active":"2024-03-28T16:50:28.828873+0000","last_peered":"2024-03-28T16:50:28.828873+0000","last_clean":"2024-03-28T16:50:28.828873+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:50:28.828873+0000","last_undegraded":"2024-03-28T16:50:28.828873+0000","last_fullsized":"2024-03-28T16:50:28.828873+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:12.429622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.864959+0000","last_change":"2024-03-28T16:48:56.685083+0000","last_active":"2024-03-28T16:49:39.864959+0000","last_peered":"2024-03-28T16:49:39.864959+0000","last_clean":"2024-03-28T16:49:39.864959+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:49:39.864959+0000","last_undegraded":"2024-03-28T16:49:39.864959+0000","last_fullsized":"2024-03-28T16:49:39.864959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587111+0000","last_change":"2024-03-28T16:48:54.658093+0000","last_active":"2024-03-28T16:49:37.587111+0000","last_peered":"2024-03-28T16:49:37.587111+0000","last_clean":"2024-03-28T16:49:37.587111+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:49:37.587111+0000","last_undegraded":"2024-03-28T16:49:37.587111+0000","last_fullsized":"2024-03-28T16:49:37.587111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:20.809568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586227+0000","last_change":"2024-03-28T16:48:58.682237+0000","last_active":"2024-03-28T16:49:37.586227+0000","last_peered":"2024-03-28T16:49:37.586227+0000","last_clean":"2024-03-28T16:49:37.586227+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:49:37.586227+0000","last_undegraded":"2024-03-28T16:49:37.586227+0000","last_fullsized":"2024-03-28T16:49:37.586227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:37.069394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'50","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.781121+0000","last_change":"2024-03-28T16:49:00.725827+0000","last_active":"2024-03-28T16:50:28.781121+0000","last_peered":"2024-03-28T16:50:28.781121+0000","last_clean":"2024-03-28T16:50:28.781121+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:50:28.781121+0000","last_undegraded":"2024-03-28T16:50:28.781121+0000","last_fullsized":"2024-03-28T16:50:28.781121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:56.316763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585610+0000","last_change":"2024-03-28T16:49:02.712054+0000","last_active":"2024-03-28T16:49:37.585610+0000","last_peered":"2024-03-28T16:49:37.585610+0000","last_clean":"2024-03-28T16:49:37.585610+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:49:37.585610+0000","last_undegraded":"2024-03-28T16:49:37.585610+0000","last_fullsized":"2024-03-28T16:49:37.585610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587403+0000","last_change":"2024-03-28T16:49:04.723184+0000","last_active":"2024-03-28T16:49:37.587403+0000","last_peered":"2024-03-28T16:49:37.587403+0000","last_clean":"2024-03-28T16:49:37.587403+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:49:37.587403+0000","last_undegraded":"2024-03-28T16:49:37.587403+0000","last_fullsized":"2024-03-28T16:49:37.587403+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:32.776436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877381+0000","last_change":"2024-03-28T16:48:56.683578+0000","last_active":"2024-03-28T16:49:39.877381+0000","last_peered":"2024-03-28T16:49:39.877381+0000","last_clean":"2024-03-28T16:49:39.877381+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:49:39.877381+0000","last_undegraded":"2024-03-28T16:49:39.877381+0000","last_fullsized":"2024-03-28T16:49:39.877381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:25.885438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588449+0000","last_change":"2024-03-28T16:48:54.664924+0000","last_active":"2024-03-28T16:49:37.588449+0000","last_peered":"2024-03-28T16:49:37.588449+0000","last_clean":"2024-03-28T16:49:37.588449+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:49:37.588449+0000","last_undegraded":"2024-03-28T16:49:37.588449+0000","last_fullsized":"2024-03-28T16:49:37.588449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:27:34.783586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595270+0000","last_change":"2024-03-28T16:48:58.692108+0000","last_active":"2024-03-28T16:49:37.595270+0000","last_peered":"2024-03-28T16:49:37.595270+0000","last_clean":"2024-03-28T16:49:37.595270+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:49:37.595270+0000","last_undegraded":"2024-03-28T16:49:37.595270+0000","last_fullsized":"2024-03-28T16:49:37.595270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:05.493534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'33","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.173814+0000","last_change":"2024-03-28T16:49:00.721579+0000","last_active":"2024-03-28T16:50:42.173814+0000","last_peered":"2024-03-28T16:50:42.173814+0000","last_clean":"2024-03-28T16:50:42.173814+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:42.173814+0000","last_undegraded":"2024-03-28T16:50:42.173814+0000","last_fullsized":"2024-03-28T16:50:42.173814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:13.394379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587827+0000","last_change":"2024-03-28T16:49:02.739236+0000","last_active":"2024-03-28T16:49:37.587827+0000","last_peered":"2024-03-28T16:49:37.587827+0000","last_clean":"2024-03-28T16:49:37.587827+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:49:37.587827+0000","last_undegraded":"2024-03-28T16:49:37.587827+0000","last_fullsized":"2024-03-28T16:49:37.587827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:34:20.723680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580108+0000","last_change":"2024-03-28T16:49:04.748645+0000","last_active":"2024-03-28T16:49:37.580108+0000","last_peered":"2024-03-28T16:49:37.580108+0000","last_clean":"2024-03-28T16:49:37.580108+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:49:37.580108+0000","last_undegraded":"2024-03-28T16:49:37.580108+0000","last_fullsized":"2024-03-28T16:49:37.580108+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:06:34.291471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593462+0000","last_change":"2024-03-28T16:48:56.695870+0000","last_active":"2024-03-28T16:49:37.593462+0000","last_peered":"2024-03-28T16:49:37.593462+0000","last_clean":"2024-03-28T16:49:37.593462+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:49:37.593462+0000","last_undegraded":"2024-03-28T16:49:37.593462+0000","last_fullsized":"2024-03-28T16:49:37.593462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:02.971815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589696+0000","last_change":"2024-03-28T16:48:54.673299+0000","last_active":"2024-03-28T16:49:37.589696+0000","last_peered":"2024-03-28T16:49:37.589696+0000","last_clean":"2024-03-28T16:49:37.589696+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:49:37.589696+0000","last_undegraded":"2024-03-28T16:49:37.589696+0000","last_fullsized":"2024-03-28T16:49:37.589696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:37.511744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588920+0000","last_change":"2024-03-28T16:48:58.683855+0000","last_active":"2024-03-28T16:49:37.588920+0000","last_peered":"2024-03-28T16:49:37.588920+0000","last_clean":"2024-03-28T16:49:37.588920+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:49:37.588920+0000","last_undegraded":"2024-03-28T16:49:37.588920+0000","last_fullsized":"2024-03-28T16:49:37.588920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:01.947384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'38","reported_seq":86,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.367231+0000","last_change":"2024-03-28T16:49:00.697730+0000","last_active":"2024-03-28T16:50:42.367231+0000","last_peered":"2024-03-28T16:50:42.367231+0000","last_clean":"2024-03-28T16:50:42.367231+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:50:42.367231+0000","last_undegraded":"2024-03-28T16:50:42.367231+0000","last_fullsized":"2024-03-28T16:50:42.367231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:27.038167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'249","reported_seq":325,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.065536+0000","last_change":"2024-03-28T16:49:02.726890+0000","last_active":"2024-03-28T16:50:42.065536+0000","last_peered":"2024-03-28T16:50:42.065536+0000","last_clean":"2024-03-28T16:50:42.065536+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:42.065536+0000","last_undegraded":"2024-03-28T16:50:42.065536+0000","last_fullsized":"2024-03-28T16:50:42.065536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:42.002449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'13","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.217530+0000","last_change":"2024-03-28T16:49:04.743257+0000","last_active":"2024-03-28T16:50:23.217530+0000","last_peered":"2024-03-28T16:50:23.217530+0000","last_clean":"2024-03-28T16:50:23.217530+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:50:23.217530+0000","last_undegraded":"2024-03-28T16:50:23.217530+0000","last_fullsized":"2024-03-28T16:50:23.217530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839667+0000","last_change":"2024-03-28T16:48:56.690053+0000","last_active":"2024-03-28T16:50:28.839667+0000","last_peered":"2024-03-28T16:50:28.839667+0000","last_clean":"2024-03-28T16:50:28.839667+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:50:28.839667+0000","last_undegraded":"2024-03-28T16:50:28.839667+0000","last_fullsized":"2024-03-28T16:50:28.839667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:21.717501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581989+0000","last_change":"2024-03-28T16:48:54.664571+0000","last_active":"2024-03-28T16:49:37.581989+0000","last_peered":"2024-03-28T16:49:37.581989+0000","last_clean":"2024-03-28T16:49:37.581989+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:49:37.581989+0000","last_undegraded":"2024-03-28T16:49:37.581989+0000","last_fullsized":"2024-03-28T16:49:37.581989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:56:22.004074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'3","reported_seq":382,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.036588+0000","last_change":"2024-03-28T16:49:37.622013+0000","last_active":"2024-03-28T16:50:42.036588+0000","last_peered":"2024-03-28T16:50:42.036588+0000","last_clean":"2024-03-28T16:50:42.036588+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:50:42.036588+0000","last_undegraded":"2024-03-28T16:50:42.036588+0000","last_fullsized":"2024-03-28T16:50:42.036588+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:46.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":340,"num_read_kb":340,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587255+0000","last_change":"2024-03-28T16:49:00.725908+0000","last_active":"2024-03-28T16:49:37.587255+0000","last_peered":"2024-03-28T16:49:37.587255+0000","last_clean":"2024-03-28T16:49:37.587255+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:49:37.587255+0000","last_undegraded":"2024-03-28T16:49:37.587255+0000","last_fullsized":"2024-03-28T16:49:37.587255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:30:40.250784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'249","reported_seq":348,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.068323+0000","last_change":"2024-03-28T16:49:02.715172+0000","last_active":"2024-03-28T16:50:42.068323+0000","last_peered":"2024-03-28T16:50:42.068323+0000","last_clean":"2024-03-28T16:50:42.068323+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:42.068323+0000","last_undegraded":"2024-03-28T16:50:42.068323+0000","last_fullsized":"2024-03-28T16:50:42.068323+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589680+0000","last_change":"2024-03-28T16:49:04.746465+0000","last_active":"2024-03-28T16:49:37.589680+0000","last_peered":"2024-03-28T16:49:37.589680+0000","last_clean":"2024-03-28T16:49:37.589680+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:49:37.589680+0000","last_undegraded":"2024-03-28T16:49:37.589680+0000","last_fullsized":"2024-03-28T16:49:37.589680+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:41.396912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189625+0000","last_change":"2024-03-28T16:48:56.681161+0000","last_active":"2024-03-28T16:50:23.189625+0000","last_peered":"2024-03-28T16:50:23.189625+0000","last_clean":"2024-03-28T16:50:23.189625+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:50:23.189625+0000","last_undegraded":"2024-03-28T16:50:23.189625+0000","last_fullsized":"2024-03-28T16:50:23.189625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:23.209038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580546+0000","last_change":"2024-03-28T16:48:54.676487+0000","last_active":"2024-03-28T16:49:37.580546+0000","last_peered":"2024-03-28T16:49:37.580546+0000","last_clean":"2024-03-28T16:49:37.580546+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:49:37.580546+0000","last_undegraded":"2024-03-28T16:49:37.580546+0000","last_fullsized":"2024-03-28T16:49:37.580546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:15.844441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588709+0000","last_change":"2024-03-28T16:48:58.703765+0000","last_active":"2024-03-28T16:49:37.588709+0000","last_peered":"2024-03-28T16:49:37.588709+0000","last_clean":"2024-03-28T16:49:37.588709+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:49:37.588709+0000","last_undegraded":"2024-03-28T16:49:37.588709+0000","last_fullsized":"2024-03-28T16:49:37.588709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:05:16.548217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'39","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.262915+0000","last_change":"2024-03-28T16:49:00.711767+0000","last_active":"2024-03-28T16:50:42.262915+0000","last_peered":"2024-03-28T16:50:42.262915+0000","last_clean":"2024-03-28T16:50:42.262915+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:42.262915+0000","last_undegraded":"2024-03-28T16:50:42.262915+0000","last_fullsized":"2024-03-28T16:50:42.262915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:07.000235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590333+0000","last_change":"2024-03-28T16:49:02.720427+0000","last_active":"2024-03-28T16:49:37.590333+0000","last_peered":"2024-03-28T16:49:37.590333+0000","last_clean":"2024-03-28T16:49:37.590333+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:49:37.590333+0000","last_undegraded":"2024-03-28T16:49:37.590333+0000","last_fullsized":"2024-03-28T16:49:37.590333+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:26:02.281547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596396+0000","last_change":"2024-03-28T16:49:04.730860+0000","last_active":"2024-03-28T16:49:37.596396+0000","last_peered":"2024-03-28T16:49:37.596396+0000","last_clean":"2024-03-28T16:49:37.596396+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:49:37.596396+0000","last_undegraded":"2024-03-28T16:49:37.596396+0000","last_fullsized":"2024-03-28T16:49:37.596396+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:05.547556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189979+0000","last_change":"2024-03-28T16:48:56.693994+0000","last_active":"2024-03-28T16:50:23.189979+0000","last_peered":"2024-03-28T16:50:23.189979+0000","last_clean":"2024-03-28T16:50:23.189979+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:50:23.189979+0000","last_undegraded":"2024-03-28T16:50:23.189979+0000","last_fullsized":"2024-03-28T16:50:23.189979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:09.568996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":199,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583446+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:49:37.583446+0000","last_peered":"2024-03-28T16:49:37.583446+0000","last_clean":"2024-03-28T16:49:37.583446+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:49:37.583446+0000","last_undegraded":"2024-03-28T16:49:37.583446+0000","last_fullsized":"2024-03-28T16:49:37.583446+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:17:35.744620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593784+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:49:37.593784+0000","last_peered":"2024-03-28T16:49:37.593784+0000","last_clean":"2024-03-28T16:49:37.593784+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:49:37.593784+0000","last_undegraded":"2024-03-28T16:49:37.593784+0000","last_fullsized":"2024-03-28T16:49:37.593784+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.194802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581636+0000","last_change":"2024-03-28T16:48:54.666099+0000","last_active":"2024-03-28T16:49:37.581636+0000","last_peered":"2024-03-28T16:49:37.581636+0000","last_clean":"2024-03-28T16:49:37.581636+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:49:37.581636+0000","last_undegraded":"2024-03-28T16:49:37.581636+0000","last_fullsized":"2024-03-28T16:49:37.581636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:03.153093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585495+0000","last_change":"2024-03-28T16:48:58.683594+0000","last_active":"2024-03-28T16:49:37.585495+0000","last_peered":"2024-03-28T16:49:37.585495+0000","last_clean":"2024-03-28T16:49:37.585495+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:49:37.585495+0000","last_undegraded":"2024-03-28T16:49:37.585495+0000","last_fullsized":"2024-03-28T16:49:37.585495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:14.457437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'70","reported_seq":138,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.807275+0000","last_change":"2024-03-28T16:49:00.715887+0000","last_active":"2024-03-28T16:50:28.807275+0000","last_peered":"2024-03-28T16:50:28.807275+0000","last_clean":"2024-03-28T16:50:28.807275+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:28.807275+0000","last_undegraded":"2024-03-28T16:50:28.807275+0000","last_fullsized":"2024-03-28T16:50:28.807275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:37:58.662402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591287+0000","last_change":"2024-03-28T16:49:02.731593+0000","last_active":"2024-03-28T16:49:37.591287+0000","last_peered":"2024-03-28T16:49:37.591287+0000","last_clean":"2024-03-28T16:49:37.591287+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:49:37.591287+0000","last_undegraded":"2024-03-28T16:49:37.591287+0000","last_fullsized":"2024-03-28T16:49:37.591287+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:36.899516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'52","reported_seq":114,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.870448+0000","last_change":"2024-03-28T16:49:04.752277+0000","last_active":"2024-03-28T16:50:28.870448+0000","last_peered":"2024-03-28T16:50:28.870448+0000","last_clean":"2024-03-28T16:50:28.870448+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:50:28.870448+0000","last_undegraded":"2024-03-28T16:50:28.870448+0000","last_fullsized":"2024-03-28T16:50:28.870448+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:10.717151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'11","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.202723+0000","last_change":"2024-03-28T16:49:37.614693+0000","last_active":"2024-03-28T16:50:23.202723+0000","last_peered":"2024-03-28T16:50:23.202723+0000","last_clean":"2024-03-28T16:50:23.202723+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:50:23.202723+0000","last_undegraded":"2024-03-28T16:50:23.202723+0000","last_fullsized":"2024-03-28T16:50:23.202723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:53.196986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580404+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:49:37.580404+0000","last_peered":"2024-03-28T16:49:37.580404+0000","last_clean":"2024-03-28T16:49:37.580404+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:49:37.580404+0000","last_undegraded":"2024-03-28T16:49:37.580404+0000","last_fullsized":"2024-03-28T16:49:37.580404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:54:11.830948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585095+0000","last_change":"2024-03-28T16:48:54.662603+0000","last_active":"2024-03-28T16:49:37.585095+0000","last_peered":"2024-03-28T16:49:37.585095+0000","last_clean":"2024-03-28T16:49:37.585095+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:49:37.585095+0000","last_undegraded":"2024-03-28T16:49:37.585095+0000","last_fullsized":"2024-03-28T16:49:37.585095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594800+0000","last_change":"2024-03-28T16:48:58.706305+0000","last_active":"2024-03-28T16:49:37.594800+0000","last_peered":"2024-03-28T16:49:37.594800+0000","last_clean":"2024-03-28T16:49:37.594800+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:49:37.594800+0000","last_undegraded":"2024-03-28T16:49:37.594800+0000","last_fullsized":"2024-03-28T16:49:37.594800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:32.869035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'254","reported_seq":439,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.272963+0000","last_change":"2024-03-28T16:49:00.718247+0000","last_active":"2024-03-28T16:50:42.272963+0000","last_peered":"2024-03-28T16:50:42.272963+0000","last_clean":"2024-03-28T16:50:42.272963+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:42.272963+0000","last_undegraded":"2024-03-28T16:50:42.272963+0000","last_fullsized":"2024-03-28T16:50:42.272963+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:43.848104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":342,"num_read_kb":327,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582450+0000","last_change":"2024-03-28T16:49:02.740444+0000","last_active":"2024-03-28T16:49:37.582450+0000","last_peered":"2024-03-28T16:49:37.582450+0000","last_clean":"2024-03-28T16:49:37.582450+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:49:37.582450+0000","last_undegraded":"2024-03-28T16:49:37.582450+0000","last_fullsized":"2024-03-28T16:49:37.582450+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:41.589208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'4","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.938249+0000","last_change":"2024-03-28T16:49:04.744802+0000","last_active":"2024-03-28T16:50:22.938249+0000","last_peered":"2024-03-28T16:50:22.938249+0000","last_clean":"2024-03-28T16:50:22.938249+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:50:22.938249+0000","last_undegraded":"2024-03-28T16:50:22.938249+0000","last_fullsized":"2024-03-28T16:50:22.938249+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:28.319529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.556077+0000","last_change":"2024-03-28T16:48:56.689836+0000","last_active":"2024-03-28T16:49:40.556077+0000","last_peered":"2024-03-28T16:49:40.556077+0000","last_clean":"2024-03-28T16:49:40.556077+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:49:40.556077+0000","last_undegraded":"2024-03-28T16:49:40.556077+0000","last_fullsized":"2024-03-28T16:49:40.556077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589208+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:49:37.589208+0000","last_peered":"2024-03-28T16:49:37.589208+0000","last_clean":"2024-03-28T16:49:37.589208+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:49:37.589208+0000","last_undegraded":"2024-03-28T16:49:37.589208+0000","last_fullsized":"2024-03-28T16:49:37.589208+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584655+0000","last_change":"2024-03-28T16:48:54.663058+0000","last_active":"2024-03-28T16:49:37.584655+0000","last_peered":"2024-03-28T16:49:37.584655+0000","last_clean":"2024-03-28T16:49:37.584655+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:49:37.584655+0000","last_undegraded":"2024-03-28T16:49:37.584655+0000","last_fullsized":"2024-03-28T16:49:37.584655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"22'1","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.599332+0000","last_change":"2024-03-28T16:48:58.687188+0000","last_active":"2024-03-28T16:49:37.599332+0000","last_peered":"2024-03-28T16:49:37.599332+0000","last_clean":"2024-03-28T16:49:37.599332+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:49:37.599332+0000","last_undegraded":"2024-03-28T16:49:37.599332+0000","last_fullsized":"2024-03-28T16:49:37.599332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:46:27.900434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'16","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854777+0000","last_change":"2024-03-28T16:49:00.713728+0000","last_active":"2024-03-28T16:50:33.854777+0000","last_peered":"2024-03-28T16:50:33.854777+0000","last_clean":"2024-03-28T16:50:33.854777+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:33.854777+0000","last_undegraded":"2024-03-28T16:50:33.854777+0000","last_fullsized":"2024-03-28T16:50:33.854777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:19:03.960810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586565+0000","last_change":"2024-03-28T16:49:02.732230+0000","last_active":"2024-03-28T16:49:37.586565+0000","last_peered":"2024-03-28T16:49:37.586565+0000","last_clean":"2024-03-28T16:49:37.586565+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:49:37.586565+0000","last_undegraded":"2024-03-28T16:49:37.586565+0000","last_fullsized":"2024-03-28T16:49:37.586565+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:20.091845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581243+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:49:37.581243+0000","last_peered":"2024-03-28T16:49:37.581243+0000","last_clean":"2024-03-28T16:49:37.581243+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:49:37.581243+0000","last_undegraded":"2024-03-28T16:49:37.581243+0000","last_fullsized":"2024-03-28T16:49:37.581243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:17:51.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587517+0000","last_change":"2024-03-28T16:48:56.683769+0000","last_active":"2024-03-28T16:49:37.587517+0000","last_peered":"2024-03-28T16:49:37.587517+0000","last_clean":"2024-03-28T16:49:37.587517+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:49:37.587517+0000","last_undegraded":"2024-03-28T16:49:37.587517+0000","last_fullsized":"2024-03-28T16:49:37.587517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581400+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:49:37.581400+0000","last_peered":"2024-03-28T16:49:37.581400+0000","last_clean":"2024-03-28T16:49:37.581400+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:49:37.581400+0000","last_undegraded":"2024-03-28T16:49:37.581400+0000","last_fullsized":"2024-03-28T16:49:37.581400+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:55:50.278254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591305+0000","last_change":"2024-03-28T16:48:54.666472+0000","last_active":"2024-03-28T16:49:37.591305+0000","last_peered":"2024-03-28T16:49:37.591305+0000","last_clean":"2024-03-28T16:49:37.591305+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:49:37.591305+0000","last_undegraded":"2024-03-28T16:49:37.591305+0000","last_fullsized":"2024-03-28T16:49:37.591305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587375+0000","last_change":"2024-03-28T16:48:58.688005+0000","last_active":"2024-03-28T16:49:37.587375+0000","last_peered":"2024-03-28T16:49:37.587375+0000","last_clean":"2024-03-28T16:49:37.587375+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:49:37.587375+0000","last_undegraded":"2024-03-28T16:49:37.587375+0000","last_fullsized":"2024-03-28T16:49:37.587375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:06.936540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'13","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595894+0000","last_change":"2024-03-28T16:49:00.713939+0000","last_active":"2024-03-28T16:49:37.595894+0000","last_peered":"2024-03-28T16:49:37.595894+0000","last_clean":"2024-03-28T16:49:37.595894+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:49:37.595894+0000","last_undegraded":"2024-03-28T16:49:37.595894+0000","last_fullsized":"2024-03-28T16:49:37.595894+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:00:16.840810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588068+0000","last_change":"2024-03-28T16:49:02.741194+0000","last_active":"2024-03-28T16:49:37.588068+0000","last_peered":"2024-03-28T16:49:37.588068+0000","last_clean":"2024-03-28T16:49:37.588068+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:49:37.588068+0000","last_undegraded":"2024-03-28T16:49:37.588068+0000","last_fullsized":"2024-03-28T16:49:37.588068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:56.754215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585813+0000","last_change":"2024-03-28T16:49:04.741719+0000","last_active":"2024-03-28T16:49:37.585813+0000","last_peered":"2024-03-28T16:49:37.585813+0000","last_clean":"2024-03-28T16:49:37.585813+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:49:37.585813+0000","last_undegraded":"2024-03-28T16:49:37.585813+0000","last_fullsized":"2024-03-28T16:49:37.585813+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:44.134615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555275+0000","last_change":"2024-03-28T16:48:56.687243+0000","last_active":"2024-03-28T16:49:40.555275+0000","last_peered":"2024-03-28T16:49:40.555275+0000","last_clean":"2024-03-28T16:49:40.555275+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:49:40.555275+0000","last_undegraded":"2024-03-28T16:49:40.555275+0000","last_fullsized":"2024-03-28T16:49:40.555275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596299+0000","last_change":"2024-03-28T16:48:54.657105+0000","last_active":"2024-03-28T16:49:37.596299+0000","last_peered":"2024-03-28T16:49:37.596299+0000","last_clean":"2024-03-28T16:49:37.596299+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:49:37.596299+0000","last_undegraded":"2024-03-28T16:49:37.596299+0000","last_fullsized":"2024-03-28T16:49:37.596299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586158+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:49:37.586158+0000","last_peered":"2024-03-28T16:49:37.586158+0000","last_clean":"2024-03-28T16:49:37.586158+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:49:37.586158+0000","last_undegraded":"2024-03-28T16:49:37.586158+0000","last_fullsized":"2024-03-28T16:49:37.586158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:13.218038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588917+0000","last_change":"2024-03-28T16:48:58.690031+0000","last_active":"2024-03-28T16:49:37.588917+0000","last_peered":"2024-03-28T16:49:37.588917+0000","last_clean":"2024-03-28T16:49:37.588917+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:49:37.588917+0000","last_undegraded":"2024-03-28T16:49:37.588917+0000","last_fullsized":"2024-03-28T16:49:37.588917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:29.762391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'57","reported_seq":115,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.796013+0000","last_change":"2024-03-28T16:49:00.706797+0000","last_active":"2024-03-28T16:50:28.796013+0000","last_peered":"2024-03-28T16:50:28.796013+0000","last_clean":"2024-03-28T16:50:28.796013+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:28.796013+0000","last_undegraded":"2024-03-28T16:50:28.796013+0000","last_fullsized":"2024-03-28T16:50:28.796013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:30.719266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":58,"num_read_kb":46,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590150+0000","last_change":"2024-03-28T16:49:02.720800+0000","last_active":"2024-03-28T16:49:37.590150+0000","last_peered":"2024-03-28T16:49:37.590150+0000","last_clean":"2024-03-28T16:49:37.590150+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:49:37.590150+0000","last_undegraded":"2024-03-28T16:49:37.590150+0000","last_fullsized":"2024-03-28T16:49:37.590150+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:07.770684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'9","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.211945+0000","last_change":"2024-03-28T16:49:04.741551+0000","last_active":"2024-03-28T16:50:30.211945+0000","last_peered":"2024-03-28T16:50:30.211945+0000","last_clean":"2024-03-28T16:50:30.211945+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:50:30.211945+0000","last_undegraded":"2024-03-28T16:50:30.211945+0000","last_fullsized":"2024-03-28T16:50:30.211945+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:25:55.072698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190216+0000","last_change":"2024-03-28T16:48:56.683849+0000","last_active":"2024-03-28T16:50:23.190216+0000","last_peered":"2024-03-28T16:50:23.190216+0000","last_clean":"2024-03-28T16:50:23.190216+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:50:23.190216+0000","last_undegraded":"2024-03-28T16:50:23.190216+0000","last_fullsized":"2024-03-28T16:50:23.190216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:57.646185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586877+0000","last_change":"2024-03-28T16:48:54.663076+0000","last_active":"2024-03-28T16:49:37.586877+0000","last_peered":"2024-03-28T16:49:37.586877+0000","last_clean":"2024-03-28T16:49:37.586877+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:49:37.586877+0000","last_undegraded":"2024-03-28T16:49:37.586877+0000","last_fullsized":"2024-03-28T16:49:37.586877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592775+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:49:37.592775+0000","last_peered":"2024-03-28T16:49:37.592775+0000","last_clean":"2024-03-28T16:49:37.592775+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:49:37.592775+0000","last_undegraded":"2024-03-28T16:49:37.592775+0000","last_fullsized":"2024-03-28T16:49:37.592775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:37.509224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582103+0000","last_change":"2024-03-28T16:48:58.683037+0000","last_active":"2024-03-28T16:49:37.582103+0000","last_peered":"2024-03-28T16:49:37.582103+0000","last_clean":"2024-03-28T16:49:37.582103+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:49:37.582103+0000","last_undegraded":"2024-03-28T16:49:37.582103+0000","last_fullsized":"2024-03-28T16:49:37.582103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:56:16.061364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'14","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595604+0000","last_change":"2024-03-28T16:49:00.720928+0000","last_active":"2024-03-28T16:49:37.595604+0000","last_peered":"2024-03-28T16:49:37.595604+0000","last_clean":"2024-03-28T16:49:37.595604+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:49:37.595604+0000","last_undegraded":"2024-03-28T16:49:37.595604+0000","last_fullsized":"2024-03-28T16:49:37.595604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:03.955318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'249","reported_seq":294,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.062971+0000","last_change":"2024-03-28T16:49:02.715814+0000","last_active":"2024-03-28T16:50:42.062971+0000","last_peered":"2024-03-28T16:50:42.062971+0000","last_clean":"2024-03-28T16:50:42.062971+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:42.062971+0000","last_undegraded":"2024-03-28T16:50:42.062971+0000","last_fullsized":"2024-03-28T16:50:42.062971+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:24.706892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.177166+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:23.177166+0000","last_peered":"2024-03-28T16:50:23.177166+0000","last_clean":"2024-03-28T16:50:23.177166+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:23.177166+0000","last_undegraded":"2024-03-28T16:50:23.177166+0000","last_fullsized":"2024-03-28T16:50:23.177166+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:14:04.371488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189696+0000","last_change":"2024-03-28T16:48:56.693531+0000","last_active":"2024-03-28T16:50:23.189696+0000","last_peered":"2024-03-28T16:50:23.189696+0000","last_clean":"2024-03-28T16:50:23.189696+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:50:23.189696+0000","last_undegraded":"2024-03-28T16:50:23.189696+0000","last_fullsized":"2024-03-28T16:50:23.189696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:22.327091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.850113+0000","last_change":"2024-03-28T16:48:54.662217+0000","last_active":"2024-03-28T16:49:39.850113+0000","last_peered":"2024-03-28T16:49:39.850113+0000","last_clean":"2024-03-28T16:49:39.850113+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:49:39.850113+0000","last_undegraded":"2024-03-28T16:49:39.850113+0000","last_fullsized":"2024-03-28T16:49:39.850113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:59.868215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591024+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:49:37.591024+0000","last_peered":"2024-03-28T16:49:37.591024+0000","last_clean":"2024-03-28T16:49:37.591024+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:49:37.591024+0000","last_undegraded":"2024-03-28T16:49:37.591024+0000","last_fullsized":"2024-03-28T16:49:37.591024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:32:05.078823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593505+0000","last_change":"2024-03-28T16:48:58.684369+0000","last_active":"2024-03-28T16:49:37.593505+0000","last_peered":"2024-03-28T16:49:37.593505+0000","last_clean":"2024-03-28T16:49:37.593505+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:49:37.593505+0000","last_undegraded":"2024-03-28T16:49:37.593505+0000","last_fullsized":"2024-03-28T16:49:37.593505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:58.333522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589611+0000","last_change":"2024-03-28T16:49:00.721989+0000","last_active":"2024-03-28T16:49:37.589611+0000","last_peered":"2024-03-28T16:49:37.589611+0000","last_clean":"2024-03-28T16:49:37.589611+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:49:37.589611+0000","last_undegraded":"2024-03-28T16:49:37.589611+0000","last_fullsized":"2024-03-28T16:49:37.589611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:27.568851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'246","reported_seq":324,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:33.854449+0000","last_change":"2024-03-28T16:49:02.730370+0000","last_active":"2024-03-28T16:50:33.854449+0000","last_peered":"2024-03-28T16:50:33.854449+0000","last_clean":"2024-03-28T16:50:33.854449+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:33.854449+0000","last_undegraded":"2024-03-28T16:50:33.854449+0000","last_fullsized":"2024-03-28T16:50:33.854449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:53.399504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585106+0000","last_change":"2024-03-28T16:49:04.739259+0000","last_active":"2024-03-28T16:49:37.585106+0000","last_peered":"2024-03-28T16:49:37.585106+0000","last_clean":"2024-03-28T16:49:37.585106+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:49:37.585106+0000","last_undegraded":"2024-03-28T16:49:37.585106+0000","last_fullsized":"2024-03-28T16:49:37.585106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:50:01.255390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.555184+0000","last_change":"2024-03-28T16:48:56.682608+0000","last_active":"2024-03-28T16:49:40.555184+0000","last_peered":"2024-03-28T16:49:40.555184+0000","last_clean":"2024-03-28T16:49:40.555184+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:49:40.555184+0000","last_undegraded":"2024-03-28T16:49:40.555184+0000","last_fullsized":"2024-03-28T16:49:40.555184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.855204+0000","last_change":"2024-03-28T16:48:54.675316+0000","last_active":"2024-03-28T16:49:39.855204+0000","last_peered":"2024-03-28T16:49:39.855204+0000","last_clean":"2024-03-28T16:49:39.855204+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:49:39.855204+0000","last_undegraded":"2024-03-28T16:49:39.855204+0000","last_fullsized":"2024-03-28T16:49:39.855204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:54.571336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588286+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:49:37.588286+0000","last_peered":"2024-03-28T16:49:37.588286+0000","last_clean":"2024-03-28T16:49:37.588286+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:49:37.588286+0000","last_undegraded":"2024-03-28T16:49:37.588286+0000","last_fullsized":"2024-03-28T16:49:37.588286+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:45:04.316499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601449+0000","last_change":"2024-03-28T16:48:58.681176+0000","last_active":"2024-03-28T16:49:37.601449+0000","last_peered":"2024-03-28T16:49:37.601449+0000","last_clean":"2024-03-28T16:49:37.601449+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:49:37.601449+0000","last_undegraded":"2024-03-28T16:49:37.601449+0000","last_fullsized":"2024-03-28T16:49:37.601449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:49.369023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'85","reported_seq":168,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.250387+0000","last_change":"2024-03-28T16:49:00.709967+0000","last_active":"2024-03-28T16:50:42.250387+0000","last_peered":"2024-03-28T16:50:42.250387+0000","last_clean":"2024-03-28T16:50:42.250387+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:42.250387+0000","last_undegraded":"2024-03-28T16:50:42.250387+0000","last_fullsized":"2024-03-28T16:50:42.250387+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:36:08.949965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585774+0000","last_change":"2024-03-28T16:49:02.715279+0000","last_active":"2024-03-28T16:49:37.585774+0000","last_peered":"2024-03-28T16:49:37.585774+0000","last_clean":"2024-03-28T16:49:37.585774+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:49:37.585774+0000","last_undegraded":"2024-03-28T16:49:37.585774+0000","last_fullsized":"2024-03-28T16:49:37.585774+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:02.364003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594229+0000","last_change":"2024-03-28T16:49:04.754190+0000","last_active":"2024-03-28T16:49:37.594229+0000","last_peered":"2024-03-28T16:49:37.594229+0000","last_clean":"2024-03-28T16:49:37.594229+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:49:37.594229+0000","last_undegraded":"2024-03-28T16:49:37.594229+0000","last_fullsized":"2024-03-28T16:49:37.594229+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:41:26.719819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594252+0000","last_change":"2024-03-28T16:49:04.740956+0000","last_active":"2024-03-28T16:49:37.594252+0000","last_peered":"2024-03-28T16:49:37.594252+0000","last_clean":"2024-03-28T16:49:37.594252+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:49:37.594252+0000","last_undegraded":"2024-03-28T16:49:37.594252+0000","last_fullsized":"2024-03-28T16:49:37.594252+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:55.907917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190179+0000","last_change":"2024-03-28T16:48:56.680906+0000","last_active":"2024-03-28T16:50:23.190179+0000","last_peered":"2024-03-28T16:50:23.190179+0000","last_clean":"2024-03-28T16:50:23.190179+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:50:23.190179+0000","last_undegraded":"2024-03-28T16:50:23.190179+0000","last_fullsized":"2024-03-28T16:50:23.190179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:05.166122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587082+0000","last_change":"2024-03-28T16:48:54.663088+0000","last_active":"2024-03-28T16:49:37.587082+0000","last_peered":"2024-03-28T16:49:37.587082+0000","last_clean":"2024-03-28T16:49:37.587082+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:49:37.587082+0000","last_undegraded":"2024-03-28T16:49:37.587082+0000","last_fullsized":"2024-03-28T16:49:37.587082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:12.659538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.606876+0000","last_change":"2024-03-28T16:49:37.606876+0000","last_active":"2024-03-28T16:49:37.606876+0000","last_peered":"2024-03-28T16:49:37.606876+0000","last_clean":"2024-03-28T16:49:37.606876+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:49:37.606876+0000","last_undegraded":"2024-03-28T16:49:37.606876+0000","last_fullsized":"2024-03-28T16:49:37.606876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:35:25.555426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'31","reported_seq":73,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.157002+0000","last_change":"2024-03-28T16:49:00.725615+0000","last_active":"2024-03-28T16:50:23.157002+0000","last_peered":"2024-03-28T16:50:23.157002+0000","last_clean":"2024-03-28T16:50:23.157002+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:50:23.157002+0000","last_undegraded":"2024-03-28T16:50:23.157002+0000","last_fullsized":"2024-03-28T16:50:23.157002+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:47:56.763507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582514+0000","last_change":"2024-03-28T16:49:02.739259+0000","last_active":"2024-03-28T16:49:37.582514+0000","last_peered":"2024-03-28T16:49:37.582514+0000","last_clean":"2024-03-28T16:49:37.582514+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:49:37.582514+0000","last_undegraded":"2024-03-28T16:49:37.582514+0000","last_fullsized":"2024-03-28T16:49:37.582514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:33.761245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'1","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586987+0000","last_change":"2024-03-28T16:49:04.723410+0000","last_active":"2024-03-28T16:49:37.586987+0000","last_peered":"2024-03-28T16:49:37.586987+0000","last_clean":"2024-03-28T16:49:37.586987+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:49:37.586987+0000","last_undegraded":"2024-03-28T16:49:37.586987+0000","last_fullsized":"2024-03-28T16:49:37.586987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:56:19.278765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.603811+0000","last_change":"2024-03-28T16:49:37.603811+0000","last_active":"2024-03-28T16:49:37.603811+0000","last_peered":"2024-03-28T16:49:37.603811+0000","last_clean":"2024-03-28T16:49:37.603811+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:49:37.603811+0000","last_undegraded":"2024-03-28T16:49:37.603811+0000","last_fullsized":"2024-03-28T16:49:37.603811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:44.503106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.833936+0000","last_change":"2024-03-28T16:48:54.657586+0000","last_active":"2024-03-28T16:50:28.833936+0000","last_peered":"2024-03-28T16:50:28.833936+0000","last_clean":"2024-03-28T16:50:28.833936+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:50:28.833936+0000","last_undegraded":"2024-03-28T16:50:28.833936+0000","last_fullsized":"2024-03-28T16:50:28.833936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:39:16.549224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594338+0000","last_change":"2024-03-28T16:48:58.696407+0000","last_active":"2024-03-28T16:49:37.594338+0000","last_peered":"2024-03-28T16:49:37.594338+0000","last_clean":"2024-03-28T16:49:37.594338+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:49:37.594338+0000","last_undegraded":"2024-03-28T16:49:37.594338+0000","last_fullsized":"2024-03-28T16:49:37.594338+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:24:45.492142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:15.707942+0000","last_change":"2024-03-28T16:49:00.701020+0000","last_active":"2024-03-28T16:50:15.707942+0000","last_peered":"2024-03-28T16:50:15.707942+0000","last_clean":"2024-03-28T16:50:15.707942+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:50:15.707942+0000","last_undegraded":"2024-03-28T16:50:15.707942+0000","last_fullsized":"2024-03-28T16:50:15.707942+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:42.652898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589597+0000","last_change":"2024-03-28T16:49:02.726748+0000","last_active":"2024-03-28T16:49:37.589597+0000","last_peered":"2024-03-28T16:49:37.589597+0000","last_clean":"2024-03-28T16:49:37.589597+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:49:37.589597+0000","last_undegraded":"2024-03-28T16:49:37.589597+0000","last_fullsized":"2024-03-28T16:49:37.589597+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:39.742691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586140+0000","last_change":"2024-03-28T16:49:04.744040+0000","last_active":"2024-03-28T16:49:37.586140+0000","last_peered":"2024-03-28T16:49:37.586140+0000","last_clean":"2024-03-28T16:49:37.586140+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:49:37.586140+0000","last_undegraded":"2024-03-28T16:49:37.586140+0000","last_fullsized":"2024-03-28T16:49:37.586140+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:51:53.769493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.730425+0000","last_change":"2024-03-28T16:48:56.676841+0000","last_active":"2024-03-28T16:50:22.730425+0000","last_peered":"2024-03-28T16:50:22.730425+0000","last_clean":"2024-03-28T16:50:22.730425+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:50:22.730425+0000","last_undegraded":"2024-03-28T16:50:22.730425+0000","last_fullsized":"2024-03-28T16:50:22.730425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:45.419272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.183347+0000","last_change":"2024-03-28T16:48:54.659093+0000","last_active":"2024-03-28T16:50:23.183347+0000","last_peered":"2024-03-28T16:50:23.183347+0000","last_clean":"2024-03-28T16:50:23.183347+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:50:23.183347+0000","last_undegraded":"2024-03-28T16:50:23.183347+0000","last_fullsized":"2024-03-28T16:50:23.183347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'2","reported_seq":198,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.023788+0000","last_change":"2024-03-28T16:48:58.690200+0000","last_active":"2024-03-28T16:50:42.023788+0000","last_peered":"2024-03-28T16:50:42.023788+0000","last_clean":"2024-03-28T16:50:42.023788+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:50:42.023788+0000","last_undegraded":"2024-03-28T16:50:42.023788+0000","last_fullsized":"2024-03-28T16:50:42.023788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:59.661181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":256,"num_read_kb":255,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'35","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.577197+0000","last_change":"2024-03-28T16:49:00.718425+0000","last_active":"2024-03-28T16:50:28.577197+0000","last_peered":"2024-03-28T16:50:28.577197+0000","last_clean":"2024-03-28T16:50:28.577197+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:28.577197+0000","last_undegraded":"2024-03-28T16:50:28.577197+0000","last_fullsized":"2024-03-28T16:50:28.577197+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:56:37.725635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":50,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585222+0000","last_change":"2024-03-28T16:49:02.715466+0000","last_active":"2024-03-28T16:49:37.585222+0000","last_peered":"2024-03-28T16:49:37.585222+0000","last_clean":"2024-03-28T16:49:37.585222+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:49:37.585222+0000","last_undegraded":"2024-03-28T16:49:37.585222+0000","last_fullsized":"2024-03-28T16:49:37.585222+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:53.894990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581973+0000","last_change":"2024-03-28T16:49:04.752570+0000","last_active":"2024-03-28T16:49:37.581973+0000","last_peered":"2024-03-28T16:49:37.581973+0000","last_clean":"2024-03-28T16:49:37.581973+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:49:37.581973+0000","last_undegraded":"2024-03-28T16:49:37.581973+0000","last_fullsized":"2024-03-28T16:49:37.581973+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:04:11.687874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189947+0000","last_change":"2024-03-28T16:48:56.683249+0000","last_active":"2024-03-28T16:50:23.189947+0000","last_peered":"2024-03-28T16:50:23.189947+0000","last_clean":"2024-03-28T16:50:23.189947+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:50:23.189947+0000","last_undegraded":"2024-03-28T16:50:23.189947+0000","last_fullsized":"2024-03-28T16:50:23.189947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:23.009747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602071+0000","last_change":"2024-03-28T16:48:54.674928+0000","last_active":"2024-03-28T16:49:37.602071+0000","last_peered":"2024-03-28T16:49:37.602071+0000","last_clean":"2024-03-28T16:49:37.602071+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:49:37.602071+0000","last_undegraded":"2024-03-28T16:49:37.602071+0000","last_fullsized":"2024-03-28T16:49:37.602071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593497+0000","last_change":"2024-03-28T16:48:58.680113+0000","last_active":"2024-03-28T16:49:37.593497+0000","last_peered":"2024-03-28T16:49:37.593497+0000","last_clean":"2024-03-28T16:49:37.593497+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:49:37.593497+0000","last_undegraded":"2024-03-28T16:49:37.593497+0000","last_fullsized":"2024-03-28T16:49:37.593497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:37:59.079328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'31","reported_seq":76,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.190940+0000","last_change":"2024-03-28T16:49:00.696647+0000","last_active":"2024-03-28T16:50:42.190940+0000","last_peered":"2024-03-28T16:50:42.190940+0000","last_clean":"2024-03-28T16:50:42.190940+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:50:42.190940+0000","last_undegraded":"2024-03-28T16:50:42.190940+0000","last_fullsized":"2024-03-28T16:50:42.190940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:15.665839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594972+0000","last_change":"2024-03-28T16:49:02.726334+0000","last_active":"2024-03-28T16:49:37.594972+0000","last_peered":"2024-03-28T16:49:37.594972+0000","last_clean":"2024-03-28T16:49:37.594972+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:49:37.594972+0000","last_undegraded":"2024-03-28T16:49:37.594972+0000","last_fullsized":"2024-03-28T16:49:37.594972+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:58.829170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600011+0000","last_change":"2024-03-28T16:48:56.675359+0000","last_active":"2024-03-28T16:49:37.600011+0000","last_peered":"2024-03-28T16:49:37.600011+0000","last_clean":"2024-03-28T16:49:37.600011+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:49:37.600011+0000","last_undegraded":"2024-03-28T16:49:37.600011+0000","last_fullsized":"2024-03-28T16:49:37.600011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:35.956973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601915+0000","last_change":"2024-03-28T16:49:37.600567+0000","last_active":"2024-03-28T16:49:37.601915+0000","last_peered":"2024-03-28T16:49:37.601915+0000","last_clean":"2024-03-28T16:49:37.601915+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:49:37.601915+0000","last_undegraded":"2024-03-28T16:49:37.601915+0000","last_fullsized":"2024-03-28T16:49:37.601915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:40:41.090124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591166+0000","last_change":"2024-03-28T16:48:58.687213+0000","last_active":"2024-03-28T16:49:37.591166+0000","last_peered":"2024-03-28T16:49:37.591166+0000","last_clean":"2024-03-28T16:49:37.591166+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:49:37.591166+0000","last_undegraded":"2024-03-28T16:49:37.591166+0000","last_fullsized":"2024-03-28T16:49:37.591166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:13.652414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'51","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.785801+0000","last_change":"2024-03-28T16:49:00.716007+0000","last_active":"2024-03-28T16:50:28.785801+0000","last_peered":"2024-03-28T16:50:28.785801+0000","last_clean":"2024-03-28T16:50:28.785801+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:28.785801+0000","last_undegraded":"2024-03-28T16:50:28.785801+0000","last_fullsized":"2024-03-28T16:50:28.785801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:56.812114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595955+0000","last_change":"2024-03-28T16:49:02.730156+0000","last_active":"2024-03-28T16:49:37.595955+0000","last_peered":"2024-03-28T16:49:37.595955+0000","last_clean":"2024-03-28T16:49:37.595955+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:49:37.595955+0000","last_undegraded":"2024-03-28T16:49:37.595955+0000","last_fullsized":"2024-03-28T16:49:37.595955+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:04:15.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'11","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.213440+0000","last_change":"2024-03-28T16:49:04.752525+0000","last_active":"2024-03-28T16:50:23.213440+0000","last_peered":"2024-03-28T16:50:23.213440+0000","last_clean":"2024-03-28T16:50:23.213440+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:50:23.213440+0000","last_undegraded":"2024-03-28T16:50:23.213440+0000","last_fullsized":"2024-03-28T16:50:23.213440+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:18.858161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588098+0000","last_change":"2024-03-28T16:48:56.697482+0000","last_active":"2024-03-28T16:49:37.588098+0000","last_peered":"2024-03-28T16:49:37.588098+0000","last_clean":"2024-03-28T16:49:37.588098+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:49:37.588098+0000","last_undegraded":"2024-03-28T16:49:37.588098+0000","last_fullsized":"2024-03-28T16:49:37.588098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:02.641119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584038+0000","last_change":"2024-03-28T16:48:54.668040+0000","last_active":"2024-03-28T16:49:37.584038+0000","last_peered":"2024-03-28T16:49:37.584038+0000","last_clean":"2024-03-28T16:49:37.584038+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:49:37.584038+0000","last_undegraded":"2024-03-28T16:49:37.584038+0000","last_fullsized":"2024-03-28T16:49:37.584038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.596417+0000","last_change":"2024-03-28T16:48:58.686377+0000","last_active":"2024-03-28T16:49:37.596417+0000","last_peered":"2024-03-28T16:49:37.596417+0000","last_clean":"2024-03-28T16:49:37.596417+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:49:37.596417+0000","last_undegraded":"2024-03-28T16:49:37.596417+0000","last_fullsized":"2024-03-28T16:49:37.596417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:01.340592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'34","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.138592+0000","last_change":"2024-03-28T16:49:00.716668+0000","last_active":"2024-03-28T16:50:23.138592+0000","last_peered":"2024-03-28T16:50:23.138592+0000","last_clean":"2024-03-28T16:50:23.138592+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:23.138592+0000","last_undegraded":"2024-03-28T16:50:23.138592+0000","last_fullsized":"2024-03-28T16:50:23.138592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:48.604008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587828+0000","last_change":"2024-03-28T16:49:02.731621+0000","last_active":"2024-03-28T16:49:37.587828+0000","last_peered":"2024-03-28T16:49:37.587828+0000","last_clean":"2024-03-28T16:49:37.587828+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:49:37.587828+0000","last_undegraded":"2024-03-28T16:49:37.587828+0000","last_fullsized":"2024-03-28T16:49:37.587828+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:30.494062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.557447+0000","last_change":"2024-03-28T16:49:04.754981+0000","last_active":"2024-03-28T16:49:40.557447+0000","last_peered":"2024-03-28T16:49:40.557447+0000","last_clean":"2024-03-28T16:49:40.557447+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:49:40.557447+0000","last_undegraded":"2024-03-28T16:49:40.557447+0000","last_fullsized":"2024-03-28T16:49:40.557447+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:33:25.577689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'3","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.189773+0000","last_change":"2024-03-28T16:48:56.694984+0000","last_active":"2024-03-28T16:50:23.189773+0000","last_peered":"2024-03-28T16:50:23.189773+0000","last_clean":"2024-03-28T16:50:23.189773+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:50:23.189773+0000","last_undegraded":"2024-03-28T16:50:23.189773+0000","last_fullsized":"2024-03-28T16:50:23.189773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:09.365959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585942+0000","last_change":"2024-03-28T16:48:54.654527+0000","last_active":"2024-03-28T16:49:37.585942+0000","last_peered":"2024-03-28T16:49:37.585942+0000","last_clean":"2024-03-28T16:49:37.585942+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:49:37.585942+0000","last_undegraded":"2024-03-28T16:49:37.585942+0000","last_fullsized":"2024-03-28T16:49:37.585942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.610755+0000","last_change":"2024-03-28T16:49:37.610755+0000","last_active":"2024-03-28T16:49:37.610755+0000","last_peered":"2024-03-28T16:49:37.610755+0000","last_clean":"2024-03-28T16:49:37.610755+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:49:37.610755+0000","last_undegraded":"2024-03-28T16:49:37.610755+0000","last_fullsized":"2024-03-28T16:49:37.610755+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:32:24.156961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'32","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.163251+0000","last_change":"2024-03-28T16:49:00.706941+0000","last_active":"2024-03-28T16:50:23.163251+0000","last_peered":"2024-03-28T16:50:23.163251+0000","last_clean":"2024-03-28T16:50:23.163251+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:23.163251+0000","last_undegraded":"2024-03-28T16:50:23.163251+0000","last_fullsized":"2024-03-28T16:50:23.163251+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:29:05.753628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":46,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587449+0000","last_change":"2024-03-28T16:49:02.734090+0000","last_active":"2024-03-28T16:49:37.587449+0000","last_peered":"2024-03-28T16:49:37.587449+0000","last_clean":"2024-03-28T16:49:37.587449+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:49:37.587449+0000","last_undegraded":"2024-03-28T16:49:37.587449+0000","last_fullsized":"2024-03-28T16:49:37.587449+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:39.107257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.835263+0000","last_change":"2024-03-28T16:49:04.748086+0000","last_active":"2024-03-28T16:49:39.835263+0000","last_peered":"2024-03-28T16:49:39.835263+0000","last_clean":"2024-03-28T16:49:39.835263+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:49:39.835263+0000","last_undegraded":"2024-03-28T16:49:39.835263+0000","last_fullsized":"2024-03-28T16:49:39.835263+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:44.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588233+0000","last_change":"2024-03-28T16:48:56.682850+0000","last_active":"2024-03-28T16:49:37.588233+0000","last_peered":"2024-03-28T16:49:37.588233+0000","last_clean":"2024-03-28T16:49:37.588233+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:49:37.588233+0000","last_undegraded":"2024-03-28T16:49:37.588233+0000","last_fullsized":"2024-03-28T16:49:37.588233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592598+0000","last_change":"2024-03-28T16:48:54.674069+0000","last_active":"2024-03-28T16:49:37.592598+0000","last_peered":"2024-03-28T16:49:37.592598+0000","last_clean":"2024-03-28T16:49:37.592598+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:49:37.592598+0000","last_undegraded":"2024-03-28T16:49:37.592598+0000","last_fullsized":"2024-03-28T16:49:37.592598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:43.406243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587070+0000","last_change":"2024-03-28T16:48:58.681774+0000","last_active":"2024-03-28T16:49:37.587070+0000","last_peered":"2024-03-28T16:49:37.587070+0000","last_clean":"2024-03-28T16:49:37.587070+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:49:37.587070+0000","last_undegraded":"2024-03-28T16:49:37.587070+0000","last_fullsized":"2024-03-28T16:49:37.587070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:26:24.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'10","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589169+0000","last_change":"2024-03-28T16:49:00.720072+0000","last_active":"2024-03-28T16:49:37.589169+0000","last_peered":"2024-03-28T16:49:37.589169+0000","last_clean":"2024-03-28T16:49:37.589169+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:49:37.589169+0000","last_undegraded":"2024-03-28T16:49:37.589169+0000","last_fullsized":"2024-03-28T16:49:37.589169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:22:49.287785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'249","reported_seq":300,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.059663+0000","last_change":"2024-03-28T16:49:02.734898+0000","last_active":"2024-03-28T16:50:42.059663+0000","last_peered":"2024-03-28T16:50:42.059663+0000","last_clean":"2024-03-28T16:50:42.059663+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:42.059663+0000","last_undegraded":"2024-03-28T16:50:42.059663+0000","last_fullsized":"2024-03-28T16:50:42.059663+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:05.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.579919+0000","last_change":"2024-03-28T16:49:04.748224+0000","last_active":"2024-03-28T16:49:37.579919+0000","last_peered":"2024-03-28T16:49:37.579919+0000","last_clean":"2024-03-28T16:49:37.579919+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:49:37.579919+0000","last_undegraded":"2024-03-28T16:49:37.579919+0000","last_fullsized":"2024-03-28T16:49:37.579919+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:26.578458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'5","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370813+0000","last_change":"2024-03-28T16:48:56.685308+0000","last_active":"2024-03-28T16:50:22.370813+0000","last_peered":"2024-03-28T16:50:22.370813+0000","last_clean":"2024-03-28T16:50:22.370813+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:50:22.370813+0000","last_undegraded":"2024-03-28T16:50:22.370813+0000","last_fullsized":"2024-03-28T16:50:22.370813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:39.524773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586373+0000","last_change":"2024-03-28T16:48:54.667648+0000","last_active":"2024-03-28T16:49:37.586373+0000","last_peered":"2024-03-28T16:49:37.586373+0000","last_clean":"2024-03-28T16:49:37.586373+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:49:37.586373+0000","last_undegraded":"2024-03-28T16:49:37.586373+0000","last_fullsized":"2024-03-28T16:49:37.586373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598430+0000","last_change":"2024-03-28T16:48:58.684892+0000","last_active":"2024-03-28T16:49:37.598430+0000","last_peered":"2024-03-28T16:49:37.598430+0000","last_clean":"2024-03-28T16:49:37.598430+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:49:37.598430+0000","last_undegraded":"2024-03-28T16:49:37.598430+0000","last_fullsized":"2024-03-28T16:49:37.598430+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:17:34.392979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'37","reported_seq":85,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.138684+0000","last_change":"2024-03-28T16:49:00.722124+0000","last_active":"2024-03-28T16:50:42.138684+0000","last_peered":"2024-03-28T16:50:42.138684+0000","last_clean":"2024-03-28T16:50:42.138684+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:42.138684+0000","last_undegraded":"2024-03-28T16:50:42.138684+0000","last_fullsized":"2024-03-28T16:50:42.138684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:39.985393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584276+0000","last_change":"2024-03-28T16:49:02.715093+0000","last_active":"2024-03-28T16:49:37.584276+0000","last_peered":"2024-03-28T16:49:37.584276+0000","last_clean":"2024-03-28T16:49:37.584276+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:49:37.584276+0000","last_undegraded":"2024-03-28T16:49:37.584276+0000","last_fullsized":"2024-03-28T16:49:37.584276+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:26.448039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601171+0000","last_change":"2024-03-28T16:49:04.745332+0000","last_active":"2024-03-28T16:49:37.601171+0000","last_peered":"2024-03-28T16:49:37.601171+0000","last_clean":"2024-03-28T16:49:37.601171+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:49:37.601171+0000","last_undegraded":"2024-03-28T16:49:37.601171+0000","last_fullsized":"2024-03-28T16:49:37.601171+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:11.521117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'3","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369950+0000","last_change":"2024-03-28T16:48:56.690217+0000","last_active":"2024-03-28T16:50:22.369950+0000","last_peered":"2024-03-28T16:50:22.369950+0000","last_clean":"2024-03-28T16:50:22.369950+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:50:22.369950+0000","last_undegraded":"2024-03-28T16:50:22.369950+0000","last_fullsized":"2024-03-28T16:50:22.369950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:15:13.722953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588654+0000","last_change":"2024-03-28T16:48:54.669871+0000","last_active":"2024-03-28T16:49:37.588654+0000","last_peered":"2024-03-28T16:49:37.588654+0000","last_clean":"2024-03-28T16:49:37.588654+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:49:37.588654+0000","last_undegraded":"2024-03-28T16:49:37.588654+0000","last_fullsized":"2024-03-28T16:49:37.588654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:27:58.495034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594708+0000","last_change":"2024-03-28T16:48:58.692783+0000","last_active":"2024-03-28T16:49:37.594708+0000","last_peered":"2024-03-28T16:49:37.594708+0000","last_clean":"2024-03-28T16:49:37.594708+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:49:37.594708+0000","last_undegraded":"2024-03-28T16:49:37.594708+0000","last_fullsized":"2024-03-28T16:49:37.594708+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:18:20.188622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'28","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.128390+0000","last_change":"2024-03-28T16:49:00.706559+0000","last_active":"2024-03-28T16:50:23.128390+0000","last_peered":"2024-03-28T16:50:23.128390+0000","last_clean":"2024-03-28T16:50:23.128390+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:23.128390+0000","last_undegraded":"2024-03-28T16:50:23.128390+0000","last_fullsized":"2024-03-28T16:50:23.128390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:54.800679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581059+0000","last_change":"2024-03-28T16:49:02.739372+0000","last_active":"2024-03-28T16:49:37.581059+0000","last_peered":"2024-03-28T16:49:37.581059+0000","last_clean":"2024-03-28T16:49:37.581059+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:49:37.581059+0000","last_undegraded":"2024-03-28T16:49:37.581059+0000","last_fullsized":"2024-03-28T16:49:37.581059+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:22:56.097286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582527+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:49:37.582527+0000","last_peered":"2024-03-28T16:49:37.582527+0000","last_clean":"2024-03-28T16:49:37.582527+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:49:37.582527+0000","last_undegraded":"2024-03-28T16:49:37.582527+0000","last_fullsized":"2024-03-28T16:49:37.582527+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:45:27.545220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593066+0000","last_change":"2024-03-28T16:48:56.666104+0000","last_active":"2024-03-28T16:49:37.593066+0000","last_peered":"2024-03-28T16:49:37.593066+0000","last_clean":"2024-03-28T16:49:37.593066+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:49:37.593066+0000","last_undegraded":"2024-03-28T16:49:37.593066+0000","last_fullsized":"2024-03-28T16:49:37.593066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:35:42.047564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588335+0000","last_change":"2024-03-28T16:48:54.665975+0000","last_active":"2024-03-28T16:49:37.588335+0000","last_peered":"2024-03-28T16:49:37.588335+0000","last_clean":"2024-03-28T16:49:37.588335+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:49:37.588335+0000","last_undegraded":"2024-03-28T16:49:37.588335+0000","last_fullsized":"2024-03-28T16:49:37.588335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:22:15.984808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585938+0000","last_change":"2024-03-28T16:48:58.688417+0000","last_active":"2024-03-28T16:49:37.585938+0000","last_peered":"2024-03-28T16:49:37.585938+0000","last_clean":"2024-03-28T16:49:37.585938+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:49:37.585938+0000","last_undegraded":"2024-03-28T16:49:37.585938+0000","last_fullsized":"2024-03-28T16:49:37.585938+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:57.154239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'24","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.159574+0000","last_change":"2024-03-28T16:49:00.717188+0000","last_active":"2024-03-28T16:50:23.159574+0000","last_peered":"2024-03-28T16:50:23.159574+0000","last_clean":"2024-03-28T16:50:23.159574+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:23.159574+0000","last_undegraded":"2024-03-28T16:50:23.159574+0000","last_fullsized":"2024-03-28T16:50:23.159574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:40:19.790152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581817+0000","last_change":"2024-03-28T16:49:02.739568+0000","last_active":"2024-03-28T16:49:37.581817+0000","last_peered":"2024-03-28T16:49:37.581817+0000","last_clean":"2024-03-28T16:49:37.581817+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:49:37.581817+0000","last_undegraded":"2024-03-28T16:49:37.581817+0000","last_fullsized":"2024-03-28T16:49:37.581817+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:41:32.279507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587161+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:49:37.587161+0000","last_peered":"2024-03-28T16:49:37.587161+0000","last_clean":"2024-03-28T16:49:37.587161+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:49:37.587161+0000","last_undegraded":"2024-03-28T16:49:37.587161+0000","last_fullsized":"2024-03-28T16:49:37.587161+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:55:59.156256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'11","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854243+0000","last_change":"2024-03-28T16:48:56.678271+0000","last_active":"2024-03-28T16:50:28.854243+0000","last_peered":"2024-03-28T16:50:28.854243+0000","last_clean":"2024-03-28T16:50:28.854243+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:50:28.854243+0000","last_undegraded":"2024-03-28T16:50:28.854243+0000","last_fullsized":"2024-03-28T16:50:28.854243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:31.748236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602333+0000","last_change":"2024-03-28T16:48:54.655009+0000","last_active":"2024-03-28T16:49:37.602333+0000","last_peered":"2024-03-28T16:49:37.602333+0000","last_clean":"2024-03-28T16:49:37.602333+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:49:37.602333+0000","last_undegraded":"2024-03-28T16:49:37.602333+0000","last_fullsized":"2024-03-28T16:49:37.602333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'2","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595281+0000","last_change":"2024-03-28T16:48:58.697773+0000","last_active":"2024-03-28T16:49:37.595281+0000","last_peered":"2024-03-28T16:49:37.595281+0000","last_clean":"2024-03-28T16:49:37.595281+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:49:37.595281+0000","last_undegraded":"2024-03-28T16:49:37.595281+0000","last_fullsized":"2024-03-28T16:49:37.595281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:25.024742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'29","reported_seq":70,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.816906+0000","last_change":"2024-03-28T16:49:00.720335+0000","last_active":"2024-03-28T16:50:28.816906+0000","last_peered":"2024-03-28T16:50:28.816906+0000","last_clean":"2024-03-28T16:50:28.816906+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:50:28.816906+0000","last_undegraded":"2024-03-28T16:50:28.816906+0000","last_fullsized":"2024-03-28T16:50:28.816906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:53.514285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":26,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586838+0000","last_change":"2024-03-28T16:49:02.714786+0000","last_active":"2024-03-28T16:49:37.586838+0000","last_peered":"2024-03-28T16:49:37.586838+0000","last_clean":"2024-03-28T16:49:37.586838+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:49:37.586838+0000","last_undegraded":"2024-03-28T16:49:37.586838+0000","last_fullsized":"2024-03-28T16:49:37.586838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:24:35.734813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582310+0000","last_change":"2024-03-28T16:49:04.753745+0000","last_active":"2024-03-28T16:49:37.582310+0000","last_peered":"2024-03-28T16:49:37.582310+0000","last_clean":"2024-03-28T16:49:37.582310+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:49:37.582310+0000","last_undegraded":"2024-03-28T16:49:37.582310+0000","last_fullsized":"2024-03-28T16:49:37.582310+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:59.796190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.619610+0000","last_change":"2024-03-28T16:49:37.618664+0000","last_active":"2024-03-28T16:49:37.619610+0000","last_peered":"2024-03-28T16:49:37.619610+0000","last_clean":"2024-03-28T16:49:37.619610+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:49:37.619610+0000","last_undegraded":"2024-03-28T16:49:37.619610+0000","last_fullsized":"2024-03-28T16:49:37.619610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:36.076211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588848+0000","last_change":"2024-03-28T16:48:54.658051+0000","last_active":"2024-03-28T16:49:37.588848+0000","last_peered":"2024-03-28T16:49:37.588848+0000","last_clean":"2024-03-28T16:49:37.588848+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:49:37.588848+0000","last_undegraded":"2024-03-28T16:49:37.588848+0000","last_fullsized":"2024-03-28T16:49:37.588848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:41:39.297224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190542+0000","last_change":"2024-03-28T16:48:56.682367+0000","last_active":"2024-03-28T16:50:23.190542+0000","last_peered":"2024-03-28T16:50:23.190542+0000","last_clean":"2024-03-28T16:50:23.190542+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:50:23.190542+0000","last_undegraded":"2024-03-28T16:50:23.190542+0000","last_fullsized":"2024-03-28T16:50:23.190542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:40.278052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584176+0000","last_change":"2024-03-28T16:48:54.655298+0000","last_active":"2024-03-28T16:49:37.584176+0000","last_peered":"2024-03-28T16:49:37.584176+0000","last_clean":"2024-03-28T16:49:37.584176+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:49:37.584176+0000","last_undegraded":"2024-03-28T16:49:37.584176+0000","last_fullsized":"2024-03-28T16:49:37.584176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:42:32.412284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'3","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.370059+0000","last_change":"2024-03-28T16:48:56.671798+0000","last_active":"2024-03-28T16:50:22.370059+0000","last_peered":"2024-03-28T16:50:22.370059+0000","last_clean":"2024-03-28T16:50:22.370059+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:50:22.370059+0000","last_undegraded":"2024-03-28T16:50:22.370059+0000","last_fullsized":"2024-03-28T16:50:22.370059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:59.926897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589042+0000","last_change":"2024-03-28T16:48:54.657491+0000","last_active":"2024-03-28T16:49:37.589042+0000","last_peered":"2024-03-28T16:49:37.589042+0000","last_clean":"2024-03-28T16:49:37.589042+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:49:37.589042+0000","last_undegraded":"2024-03-28T16:49:37.589042+0000","last_fullsized":"2024-03-28T16:49:37.589042+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:40:58.160099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586819+0000","last_change":"2024-03-28T16:48:56.683483+0000","last_active":"2024-03-28T16:49:37.586819+0000","last_peered":"2024-03-28T16:49:37.586819+0000","last_clean":"2024-03-28T16:49:37.586819+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:49:37.586819+0000","last_undegraded":"2024-03-28T16:49:37.586819+0000","last_fullsized":"2024-03-28T16:49:37.586819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:34:50.841625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588791+0000","last_change":"2024-03-28T16:48:54.665811+0000","last_active":"2024-03-28T16:49:37.588791+0000","last_peered":"2024-03-28T16:49:37.588791+0000","last_clean":"2024-03-28T16:49:37.588791+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:49:37.588791+0000","last_undegraded":"2024-03-28T16:49:37.588791+0000","last_fullsized":"2024-03-28T16:49:37.588791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:32:38.364800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591495+0000","last_change":"2024-03-28T16:48:56.677917+0000","last_active":"2024-03-28T16:49:37.591495+0000","last_peered":"2024-03-28T16:49:37.591495+0000","last_clean":"2024-03-28T16:49:37.591495+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:49:37.591495+0000","last_undegraded":"2024-03-28T16:49:37.591495+0000","last_fullsized":"2024-03-28T16:49:37.591495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587233+0000","last_change":"2024-03-28T16:48:54.655391+0000","last_active":"2024-03-28T16:49:37.587233+0000","last_peered":"2024-03-28T16:49:37.587233+0000","last_clean":"2024-03-28T16:49:37.587233+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:49:37.587233+0000","last_undegraded":"2024-03-28T16:49:37.587233+0000","last_fullsized":"2024-03-28T16:49:37.587233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:54.994902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840006+0000","last_change":"2024-03-28T16:48:56.678419+0000","last_active":"2024-03-28T16:50:28.840006+0000","last_peered":"2024-03-28T16:50:28.840006+0000","last_clean":"2024-03-28T16:50:28.840006+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:50:28.840006+0000","last_undegraded":"2024-03-28T16:50:28.840006+0000","last_fullsized":"2024-03-28T16:50:28.840006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:03.478538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.595229+0000","last_change":"2024-03-28T16:48:54.669640+0000","last_active":"2024-03-28T16:49:37.595229+0000","last_peered":"2024-03-28T16:49:37.595229+0000","last_clean":"2024-03-28T16:49:37.595229+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:49:37.595229+0000","last_undegraded":"2024-03-28T16:49:37.595229+0000","last_fullsized":"2024-03-28T16:49:37.595229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:29.919950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.527969+0000","last_change":"2024-03-28T16:48:56.689880+0000","last_active":"2024-03-28T16:49:39.527969+0000","last_peered":"2024-03-28T16:49:39.527969+0000","last_clean":"2024-03-28T16:49:39.527969+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:49:39.527969+0000","last_undegraded":"2024-03-28T16:49:39.527969+0000","last_fullsized":"2024-03-28T16:49:39.527969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:33:17.857470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598088+0000","last_change":"2024-03-28T16:48:54.661594+0000","last_active":"2024-03-28T16:49:37.598088+0000","last_peered":"2024-03-28T16:49:37.598088+0000","last_clean":"2024-03-28T16:49:37.598088+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:49:37.598088+0000","last_undegraded":"2024-03-28T16:49:37.598088+0000","last_fullsized":"2024-03-28T16:49:37.598088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'5","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839320+0000","last_change":"2024-03-28T16:48:56.679523+0000","last_active":"2024-03-28T16:50:28.839320+0000","last_peered":"2024-03-28T16:50:28.839320+0000","last_clean":"2024-03-28T16:50:28.839320+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:50:28.839320+0000","last_undegraded":"2024-03-28T16:50:28.839320+0000","last_fullsized":"2024-03-28T16:50:28.839320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:00.508063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588478+0000","last_change":"2024-03-28T16:48:54.659870+0000","last_active":"2024-03-28T16:49:37.588478+0000","last_peered":"2024-03-28T16:49:37.588478+0000","last_clean":"2024-03-28T16:49:37.588478+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:49:37.588478+0000","last_undegraded":"2024-03-28T16:49:37.588478+0000","last_fullsized":"2024-03-28T16:49:37.588478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:51:09.154229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.526775+0000","last_change":"2024-03-28T16:48:56.687788+0000","last_active":"2024-03-28T16:49:39.526775+0000","last_peered":"2024-03-28T16:49:39.526775+0000","last_clean":"2024-03-28T16:49:39.526775+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:49:39.526775+0000","last_undegraded":"2024-03-28T16:49:39.526775+0000","last_fullsized":"2024-03-28T16:49:39.526775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:05.073812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581474+0000","last_change":"2024-03-28T16:48:54.657662+0000","last_active":"2024-03-28T16:49:37.581474+0000","last_peered":"2024-03-28T16:49:37.581474+0000","last_clean":"2024-03-28T16:49:37.581474+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:49:37.581474+0000","last_undegraded":"2024-03-28T16:49:37.581474+0000","last_fullsized":"2024-03-28T16:49:37.581474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:59.369957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'1","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.369831+0000","last_change":"2024-03-28T16:48:56.682872+0000","last_active":"2024-03-28T16:50:22.369831+0000","last_peered":"2024-03-28T16:50:22.369831+0000","last_clean":"2024-03-28T16:50:22.369831+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:50:22.369831+0000","last_undegraded":"2024-03-28T16:50:22.369831+0000","last_fullsized":"2024-03-28T16:50:22.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:01:50.772258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584620+0000","last_change":"2024-03-28T16:48:54.663496+0000","last_active":"2024-03-28T16:49:37.584620+0000","last_peered":"2024-03-28T16:49:37.584620+0000","last_clean":"2024-03-28T16:49:37.584620+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:49:37.584620+0000","last_undegraded":"2024-03-28T16:49:37.584620+0000","last_fullsized":"2024-03-28T16:49:37.584620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:19.227186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587192+0000","last_change":"2024-03-28T16:48:56.678342+0000","last_active":"2024-03-28T16:49:37.587192+0000","last_peered":"2024-03-28T16:49:37.587192+0000","last_clean":"2024-03-28T16:49:37.587192+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:49:37.587192+0000","last_undegraded":"2024-03-28T16:49:37.587192+0000","last_fullsized":"2024-03-28T16:49:37.587192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:22.822597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585680+0000","last_change":"2024-03-28T16:48:54.675150+0000","last_active":"2024-03-28T16:49:37.585680+0000","last_peered":"2024-03-28T16:49:37.585680+0000","last_clean":"2024-03-28T16:49:37.585680+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:49:37.585680+0000","last_undegraded":"2024-03-28T16:49:37.585680+0000","last_fullsized":"2024-03-28T16:49:37.585680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:20:10.475957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.190164+0000","last_change":"2024-03-28T16:48:56.671157+0000","last_active":"2024-03-28T16:50:23.190164+0000","last_peered":"2024-03-28T16:50:23.190164+0000","last_clean":"2024-03-28T16:50:23.190164+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:50:23.190164+0000","last_undegraded":"2024-03-28T16:50:23.190164+0000","last_fullsized":"2024-03-28T16:50:23.190164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:59.325597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585793+0000","last_change":"2024-03-28T16:48:54.674973+0000","last_active":"2024-03-28T16:49:37.585793+0000","last_peered":"2024-03-28T16:49:37.585793+0000","last_clean":"2024-03-28T16:49:37.585793+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:49:37.585793+0000","last_undegraded":"2024-03-28T16:49:37.585793+0000","last_fullsized":"2024-03-28T16:49:37.585793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.840005+0000","last_change":"2024-03-28T16:48:56.683829+0000","last_active":"2024-03-28T16:50:28.840005+0000","last_peered":"2024-03-28T16:50:28.840005+0000","last_clean":"2024-03-28T16:50:28.840005+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:50:28.840005+0000","last_undegraded":"2024-03-28T16:50:28.840005+0000","last_fullsized":"2024-03-28T16:50:28.840005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:33.964587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587774+0000","last_change":"2024-03-28T16:48:54.676815+0000","last_active":"2024-03-28T16:49:37.587774+0000","last_peered":"2024-03-28T16:49:37.587774+0000","last_clean":"2024-03-28T16:49:37.587774+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:49:37.587774+0000","last_undegraded":"2024-03-28T16:49:37.587774+0000","last_fullsized":"2024-03-28T16:49:37.587774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:19.704472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.839390+0000","last_change":"2024-03-28T16:48:56.693060+0000","last_active":"2024-03-28T16:50:28.839390+0000","last_peered":"2024-03-28T16:50:28.839390+0000","last_clean":"2024-03-28T16:50:28.839390+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:50:28.839390+0000","last_undegraded":"2024-03-28T16:50:28.839390+0000","last_fullsized":"2024-03-28T16:50:28.839390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:09:01.881244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589164+0000","last_change":"2024-03-28T16:48:54.663003+0000","last_active":"2024-03-28T16:49:37.589164+0000","last_peered":"2024-03-28T16:49:37.589164+0000","last_clean":"2024-03-28T16:49:37.589164+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:49:37.589164+0000","last_undegraded":"2024-03-28T16:49:37.589164+0000","last_fullsized":"2024-03-28T16:49:37.589164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:00:43.737071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.078594+0000","last_change":"2024-03-28T16:48:56.685753+0000","last_active":"2024-03-28T16:49:39.078594+0000","last_peered":"2024-03-28T16:49:39.078594+0000","last_clean":"2024-03-28T16:49:39.078594+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:49:39.078594+0000","last_undegraded":"2024-03-28T16:49:39.078594+0000","last_fullsized":"2024-03-28T16:49:39.078594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:12.442441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598216+0000","last_change":"2024-03-28T16:48:54.661896+0000","last_active":"2024-03-28T16:49:37.598216+0000","last_peered":"2024-03-28T16:49:37.598216+0000","last_clean":"2024-03-28T16:49:37.598216+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:49:37.598216+0000","last_undegraded":"2024-03-28T16:49:37.598216+0000","last_fullsized":"2024-03-28T16:49:37.598216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:41.836208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'9","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877960+0000","last_change":"2024-03-28T16:48:56.680119+0000","last_active":"2024-03-28T16:49:39.877960+0000","last_peered":"2024-03-28T16:49:39.877960+0000","last_clean":"2024-03-28T16:49:39.877960+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:49:39.877960+0000","last_undegraded":"2024-03-28T16:49:39.877960+0000","last_fullsized":"2024-03-28T16:49:39.877960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588667+0000","last_change":"2024-03-28T16:48:54.665812+0000","last_active":"2024-03-28T16:49:37.588667+0000","last_peered":"2024-03-28T16:49:37.588667+0000","last_clean":"2024-03-28T16:49:37.588667+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:49:37.588667+0000","last_undegraded":"2024-03-28T16:49:37.588667+0000","last_fullsized":"2024-03-28T16:49:37.588667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:12:31.951558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581570+0000","last_change":"2024-03-28T16:48:54.665260+0000","last_active":"2024-03-28T16:49:37.581570+0000","last_peered":"2024-03-28T16:49:37.581570+0000","last_clean":"2024-03-28T16:49:37.581570+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:49:37.581570+0000","last_undegraded":"2024-03-28T16:49:37.581570+0000","last_fullsized":"2024-03-28T16:49:37.581570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:37.463264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591429+0000","last_change":"2024-03-28T16:48:56.690793+0000","last_active":"2024-03-28T16:49:37.591429+0000","last_peered":"2024-03-28T16:49:37.591429+0000","last_clean":"2024-03-28T16:49:37.591429+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:49:37.591429+0000","last_undegraded":"2024-03-28T16:49:37.591429+0000","last_fullsized":"2024-03-28T16:49:37.591429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.672567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584402+0000","last_change":"2024-03-28T16:48:54.663735+0000","last_active":"2024-03-28T16:49:37.584402+0000","last_peered":"2024-03-28T16:49:37.584402+0000","last_clean":"2024-03-28T16:49:37.584402+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:49:37.584402+0000","last_undegraded":"2024-03-28T16:49:37.584402+0000","last_fullsized":"2024-03-28T16:49:37.584402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:25.602713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587762+0000","last_change":"2024-03-28T16:48:56.683832+0000","last_active":"2024-03-28T16:49:37.587762+0000","last_peered":"2024-03-28T16:49:37.587762+0000","last_clean":"2024-03-28T16:49:37.587762+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:49:37.587762+0000","last_undegraded":"2024-03-28T16:49:37.587762+0000","last_fullsized":"2024-03-28T16:49:37.587762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:38.808030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1980,"ondisk_log_size":1980,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1425,"num_read_kb":1208,"num_write":543,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1216,"ondisk_log_size":1216,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":861,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705691,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39916,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331924,"statfs":{"total":96636764160,"available":96595890176,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.752}]},{"osd":2,"last update":"Thu Mar 28 16:49:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.51900000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705692,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Mar 28 16:49:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52300000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705692,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.45300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79600000000000004,"5min":0.878,"15min":0.878},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69399999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.83799999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.27100000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.86299999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68400000000000005}]}]},{"osd":0,"up_from":9,"seq":38654705691,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.83599999999999997}]},{"osd":2,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Mar 28 16:49:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:50:47.264 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.264 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.264 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.264 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.265 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.265 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.265 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.265 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.265 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.265 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.265 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.266 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.266 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.266 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.266 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.266 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.266 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.267 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.267 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.267 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.267 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.267 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.267 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.268 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.268 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.268 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.268 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.268 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.268 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.268 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.269 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.270 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.270 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.270 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.270 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.270 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.270 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.271 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.271 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.271 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.271 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.271 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.271 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.271 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.272 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.273 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.273 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.273 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.273 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.273 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.273 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.273 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.274 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.274 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.274 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.274 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.274 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.275 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.275 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.275 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.275 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.275 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.275 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.275 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.276 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.276 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.276 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.276 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.276 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.276 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.277 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.277 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.277 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.277 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.277 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.277 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.278 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.278 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.278 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.278 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.278 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.278 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.279 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.279 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.279 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.279 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.279 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.279 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-28T16:48:42.575180+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=42, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.279 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.280 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.280 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.280 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.280 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.280 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.280 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.281 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.281 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.281 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.281 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.281 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.281 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.281 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.282 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.282 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.282 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.282 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.282 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.282 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.283 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.283 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.283 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.283 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.283 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.283 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.283 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.284 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.284 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.284 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.284 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.284 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.284 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.285 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.285 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.285 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.285 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.285 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.285 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.286 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.286 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.286 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.286 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.286 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.286 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.287 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.287 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.287 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.287 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.287 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.287 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.288 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.288 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.288 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.288 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.288 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.288 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.288 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.289 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.289 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.289 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.289 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.289 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.289 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.289 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.290 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.290 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.290 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.290 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.290 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.290 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.291 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.291 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.291 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.291 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.291 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.291 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.291 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.292 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.292 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.292 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.292 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.292 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.292 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.292 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.293 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.293 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.293 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.293 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.293 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.293 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.294 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.294 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.294 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.294 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.294 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.294 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.295 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.295 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.295 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.295 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.295 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.295 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.296 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.296 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.296 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.296 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.296 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.296 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.297 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.297 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.297 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.297 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.297 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.297 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.297 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.298 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.298 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.298 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.298 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.298 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.298 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.299 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.299 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.299 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.299 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.299 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.299 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.300 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.300 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.300 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.300 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.300 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.300 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.301 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.302 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.302 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.302 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.302 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.302 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.302 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.303 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.303 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.303 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.303 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.303 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.303 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.303 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.304 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.304 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.304 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.304 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.304 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.304 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.304 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.305 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:50:47.305 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-28T16:51:07.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:51:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086f667640 1 Processor -- start 2024-03-28T16:51:07.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086f667640 1 -- start start 2024-03-28T16:51:07.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086f667640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0868092560 0x7f0868092940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:07.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086f667640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0868094180 con 0x7f086812a220 2024-03-28T16:51:07.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086f667640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680958f0 con 0x7f0868092560 2024-03-28T16:51:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086d403640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0868092560 0x7f0868092940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086cc02640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f086812a220 0x7f086812a6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51414/0 (socket says 172.21.15.119:51414) 2024-03-28T16:51:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086cc02640 1 -- 172.21.15.119:0/1536151827 learned_addr learned my addr 172.21.15.119:0/1536151827 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:51:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086d403640 1 -- 172.21.15.119:0/1536151827 >> v1:172.21.15.119:6789/0 conn(0x7f086812a220 legacy=0x7f086812a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:07.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086d403640 1 -- 172.21.15.119:0/1536151827 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086809c160 con 0x7f0868092560 2024-03-28T16:51:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f086d403640 1 --2- 172.21.15.119:0/1536151827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f0868092940 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0864009740 tx=0x7f086401d480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08254cc77148efb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f085bfff640 1 -- 172.21.15.119:0/1536151827 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08640a8070 con 0x7f0868092560 2024-03-28T16:51:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f085bfff640 1 -- 172.21.15.119:0/1536151827 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08640ac560 con 0x7f0868092560 2024-03-28T16:51:07.439 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.437+0000 7f085bfff640 1 -- 172.21.15.119:0/1536151827 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08640ac820 con 0x7f0868092560 2024-03-28T16:51:07.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/1536151827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 msgr2=0x7f0868092940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:07.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 --2- 172.21.15.119:0/1536151827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f0868092940 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0864009740 tx=0x7f086401d480 comp rx=0 tx=0).stop 2024-03-28T16:51:07.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/1536151827 shutdown_connections 2024-03-28T16:51:07.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 --2- 172.21.15.119:0/1536151827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f0868092940 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:07.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/1536151827 >> 172.21.15.119:0/1536151827 conn(0x7f086809a2f0 msgr2=0x7f086809a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:51:07.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/1536151827 shutdown_connections 2024-03-28T16:51:07.441 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/1536151827 wait complete. 2024-03-28T16:51:07.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 Processor -- start 2024-03-28T16:51:07.442 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- start start 2024-03-28T16:51:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f086808a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086808b4a0 con 0x7f0868092560 2024-03-28T16:51:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086d403640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f086808a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086d403640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f086808a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36884/0 (socket says 172.21.15.119:36884) 2024-03-28T16:51:07.444 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086d403640 1 -- 172.21.15.119:0/3060633686 learned_addr learned my addr 172.21.15.119:0/3060633686 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086d403640 1 -- 172.21.15.119:0/3060633686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0864009420 con 0x7f0868092560 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086d403640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f086808a350 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f08640ae040 tx=0x7f086401a220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f0859ffb640 1 -- 172.21.15.119:0/3060633686 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08640aba00 con 0x7f0868092560 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f0859ffb640 1 -- 172.21.15.119:0/3060633686 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08640abba0 con 0x7f0868092560 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0868093000 con 0x7f0868092560 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f0859ffb640 1 -- 172.21.15.119:0/3060633686 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08640abe80 con 0x7f0868092560 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086808c7f0 con 0x7f0868092560 2024-03-28T16:51:07.445 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0868067e00 con 0x7f0868092560 2024-03-28T16:51:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f0859ffb640 1 -- 172.21.15.119:0/3060633686 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f08640c6450 con 0x7f0868092560 2024-03-28T16:51:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.441+0000 7f0859ffb640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f084c0359c0 0x7f084c037e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.445+0000 7f0859ffb640 1 -- 172.21.15.119:0/3060633686 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f08640b9070 con 0x7f0868092560 2024-03-28T16:51:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.445+0000 7f086cc02640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f084c0359c0 0x7f084c037e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:07.446 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.445+0000 7f086cc02640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f084c0359c0 0x7f084c037e80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f085c005890 tx=0x7f085c007a90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:07.448 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.449+0000 7f0859ffb640 1 -- 172.21.15.119:0/3060633686 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08640c4710 con 0x7f0868092560 2024-03-28T16:51:07.602 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.601+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0868092e80 con 0x7f084c0359c0 2024-03-28T16:51:07.646 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.645+0000 7f0859ffb640 1 -- 172.21.15.119:0/3060633686 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+666170 (secure 0 0 0) 0x7f085c001c00 con 0x7f084c0359c0 2024-03-28T16:51:07.655 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:51:07.662 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:51:07.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f084c0359c0 msgr2=0x7f084c037e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:07.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f084c0359c0 0x7f084c037e80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f085c005890 tx=0x7f085c007a90 comp rx=0 tx=0).stop 2024-03-28T16:51:07.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 msgr2=0x7f086808a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:07.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f086808a350 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f08640ae040 tx=0x7f086401a220 comp rx=0 tx=0).stop 2024-03-28T16:51:07.664 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 shutdown_connections 2024-03-28T16:51:07.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f084c0359c0 0x7f084c037e80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:07.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 --2- 172.21.15.119:0/3060633686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0868092560 0x7f086808a350 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:07.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 >> 172.21.15.119:0/3060633686 conn(0x7f086809a2f0 msgr2=0x7f0868070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:51:07.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 shutdown_connections 2024-03-28T16:51:07.665 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:07.653+0000 7f086f667640 1 -- 172.21.15.119:0/3060633686 wait complete. 2024-03-28T16:51:07.676 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-03-28T16:51:06.546368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":4029,"num_read_kb":3679,"num_write":1543,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":769,"num_omap_keys":5,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3982,"ondisk_log_size":3982,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":160768,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":157945,"kb_avail":377326592,"statfs":{"total":386547056640,"available":386382430208,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":161736470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":419,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005927"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412125+0000","last_change":"2024-03-28T16:48:54.656021+0000","last_active":"2024-03-28T16:50:46.412125+0000","last_peered":"2024-03-28T16:50:46.412125+0000","last_clean":"2024-03-28T16:50:46.412125+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:50:46.412125+0000","last_undegraded":"2024-03-28T16:50:46.412125+0000","last_fullsized":"2024-03-28T16:50:46.412125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585390+0000","last_change":"2024-03-28T16:48:56.687475+0000","last_active":"2024-03-28T16:49:37.585390+0000","last_peered":"2024-03-28T16:49:37.585390+0000","last_clean":"2024-03-28T16:49:37.585390+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:49:37.585390+0000","last_undegraded":"2024-03-28T16:49:37.585390+0000","last_fullsized":"2024-03-28T16:49:37.585390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532031+0000","last_change":"2024-03-28T16:48:54.666372+0000","last_active":"2024-03-28T16:50:45.532031+0000","last_peered":"2024-03-28T16:50:45.532031+0000","last_clean":"2024-03-28T16:50:45.532031+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:50:45.532031+0000","last_undegraded":"2024-03-28T16:50:45.532031+0000","last_fullsized":"2024-03-28T16:50:45.532031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.563876+0000","last_change":"2024-03-28T16:48:56.688050+0000","last_active":"2024-03-28T16:49:40.563876+0000","last_peered":"2024-03-28T16:49:40.563876+0000","last_clean":"2024-03-28T16:49:40.563876+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:49:40.563876+0000","last_undegraded":"2024-03-28T16:49:40.563876+0000","last_fullsized":"2024-03-28T16:49:40.563876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871923+0000","last_change":"2024-03-28T16:48:54.665976+0000","last_active":"2024-03-28T16:50:46.871923+0000","last_peered":"2024-03-28T16:50:46.871923+0000","last_clean":"2024-03-28T16:50:46.871923+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:50:46.871923+0000","last_undegraded":"2024-03-28T16:50:46.871923+0000","last_fullsized":"2024-03-28T16:50:46.871923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'3","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729842+0000","last_change":"2024-03-28T16:48:56.690734+0000","last_active":"2024-03-28T16:50:22.729842+0000","last_peered":"2024-03-28T16:50:22.729842+0000","last_clean":"2024-03-28T16:50:22.729842+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:50:22.729842+0000","last_undegraded":"2024-03-28T16:50:22.729842+0000","last_fullsized":"2024-03-28T16:50:22.729842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'2","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412164+0000","last_change":"2024-03-28T16:48:54.656958+0000","last_active":"2024-03-28T16:50:46.412164+0000","last_peered":"2024-03-28T16:50:46.412164+0000","last_clean":"2024-03-28T16:50:46.412164+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:50:46.412164+0000","last_undegraded":"2024-03-28T16:50:46.412164+0000","last_fullsized":"2024-03-28T16:50:46.412164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.575289+0000","last_change":"2024-03-28T16:49:37.602262+0000","last_active":"2024-03-28T16:49:40.575289+0000","last_peered":"2024-03-28T16:49:40.575289+0000","last_clean":"2024-03-28T16:49:40.575289+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:49:40.575289+0000","last_undegraded":"2024-03-28T16:49:40.575289+0000","last_fullsized":"2024-03-28T16:49:40.575289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:38.404379+0000","last_change":"2024-03-28T16:48:54.661971+0000","last_active":"2024-03-28T16:49:38.404379+0000","last_peered":"2024-03-28T16:49:38.404379+0000","last_clean":"2024-03-28T16:49:38.404379+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:49:38.404379+0000","last_undegraded":"2024-03-28T16:49:38.404379+0000","last_fullsized":"2024-03-28T16:49:38.404379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985852+0000","last_change":"2024-03-28T16:48:56.671464+0000","last_active":"2024-03-28T16:50:45.985852+0000","last_peered":"2024-03-28T16:50:45.985852+0000","last_clean":"2024-03-28T16:50:45.985852+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:50:45.985852+0000","last_undegraded":"2024-03-28T16:50:45.985852+0000","last_fullsized":"2024-03-28T16:50:45.985852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.590201+0000","last_change":"2024-03-28T16:48:54.665454+0000","last_active":"2024-03-28T16:49:37.590201+0000","last_peered":"2024-03-28T16:49:37.590201+0000","last_clean":"2024-03-28T16:49:37.590201+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:49:37.590201+0000","last_undegraded":"2024-03-28T16:49:37.590201+0000","last_fullsized":"2024-03-28T16:49:37.590201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872739+0000","last_change":"2024-03-28T16:48:56.688375+0000","last_active":"2024-03-28T16:50:46.872739+0000","last_peered":"2024-03-28T16:50:46.872739+0000","last_clean":"2024-03-28T16:50:46.872739+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:50:46.872739+0000","last_undegraded":"2024-03-28T16:50:46.872739+0000","last_fullsized":"2024-03-28T16:50:46.872739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.862019+0000","last_change":"2024-03-28T16:48:54.672203+0000","last_active":"2024-03-28T16:49:39.862019+0000","last_peered":"2024-03-28T16:49:39.862019+0000","last_clean":"2024-03-28T16:49:39.862019+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:49:39.862019+0000","last_undegraded":"2024-03-28T16:49:39.862019+0000","last_fullsized":"2024-03-28T16:49:39.862019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.413182+0000","last_change":"2024-03-28T16:48:56.695671+0000","last_active":"2024-03-28T16:50:46.413182+0000","last_peered":"2024-03-28T16:50:46.413182+0000","last_clean":"2024-03-28T16:50:46.413182+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:50:46.413182+0000","last_undegraded":"2024-03-28T16:50:46.413182+0000","last_fullsized":"2024-03-28T16:50:46.413182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585269+0000","last_change":"2024-03-28T16:48:54.658053+0000","last_active":"2024-03-28T16:49:37.585269+0000","last_peered":"2024-03-28T16:49:37.585269+0000","last_clean":"2024-03-28T16:49:37.585269+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:49:37.585269+0000","last_undegraded":"2024-03-28T16:49:37.585269+0000","last_fullsized":"2024-03-28T16:49:37.585269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.413228+0000","last_change":"2024-03-28T16:48:56.681114+0000","last_active":"2024-03-28T16:50:46.413228+0000","last_peered":"2024-03-28T16:50:46.413228+0000","last_clean":"2024-03-28T16:50:46.413228+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:50:46.413228+0000","last_undegraded":"2024-03-28T16:50:46.413228+0000","last_fullsized":"2024-03-28T16:50:46.413228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.537442+0000","last_change":"2024-03-28T16:48:56.677244+0000","last_active":"2024-03-28T16:49:39.537442+0000","last_peered":"2024-03-28T16:49:39.537442+0000","last_clean":"2024-03-28T16:49:39.537442+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:49:39.537442+0000","last_undegraded":"2024-03-28T16:49:39.537442+0000","last_fullsized":"2024-03-28T16:49:39.537442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:38.400218+0000","last_change":"2024-03-28T16:48:54.671518+0000","last_active":"2024-03-28T16:49:38.400218+0000","last_peered":"2024-03-28T16:49:38.400218+0000","last_clean":"2024-03-28T16:49:38.400218+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:49:38.400218+0000","last_undegraded":"2024-03-28T16:49:38.400218+0000","last_fullsized":"2024-03-28T16:49:38.400218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'9","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469323+0000","last_change":"2024-03-28T16:48:56.692955+0000","last_active":"2024-03-28T16:50:35.469323+0000","last_peered":"2024-03-28T16:50:35.469323+0000","last_clean":"2024-03-28T16:50:35.469323+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:50:35.469323+0000","last_undegraded":"2024-03-28T16:50:35.469323+0000","last_fullsized":"2024-03-28T16:50:35.469323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.612449+0000","last_change":"2024-03-28T16:49:37.610918+0000","last_active":"2024-03-28T16:49:37.612449+0000","last_peered":"2024-03-28T16:49:37.612449+0000","last_clean":"2024-03-28T16:49:37.612449+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:49:37.612449+0000","last_undegraded":"2024-03-28T16:49:37.612449+0000","last_fullsized":"2024-03-28T16:49:37.612449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854727+0000","last_change":"2024-03-28T16:48:56.687940+0000","last_active":"2024-03-28T16:50:28.854727+0000","last_peered":"2024-03-28T16:50:28.854727+0000","last_clean":"2024-03-28T16:50:28.854727+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:50:28.854727+0000","last_undegraded":"2024-03-28T16:50:28.854727+0000","last_fullsized":"2024-03-28T16:50:28.854727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586148+0000","last_change":"2024-03-28T16:48:54.670148+0000","last_active":"2024-03-28T16:49:37.586148+0000","last_peered":"2024-03-28T16:49:37.586148+0000","last_clean":"2024-03-28T16:49:37.586148+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:49:37.586148+0000","last_undegraded":"2024-03-28T16:49:37.586148+0000","last_fullsized":"2024-03-28T16:49:37.586148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877477+0000","last_change":"2024-03-28T16:48:56.678642+0000","last_active":"2024-03-28T16:49:39.877477+0000","last_peered":"2024-03-28T16:49:39.877477+0000","last_clean":"2024-03-28T16:49:39.877477+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:49:39.877477+0000","last_undegraded":"2024-03-28T16:49:39.877477+0000","last_fullsized":"2024-03-28T16:49:39.877477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586252+0000","last_change":"2024-03-28T16:48:54.655014+0000","last_active":"2024-03-28T16:49:37.586252+0000","last_peered":"2024-03-28T16:49:37.586252+0000","last_clean":"2024-03-28T16:49:37.586252+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:49:37.586252+0000","last_undegraded":"2024-03-28T16:49:37.586252+0000","last_fullsized":"2024-03-28T16:49:37.586252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877451+0000","last_change":"2024-03-28T16:48:56.680884+0000","last_active":"2024-03-28T16:49:39.877451+0000","last_peered":"2024-03-28T16:49:39.877451+0000","last_clean":"2024-03-28T16:49:39.877451+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:49:39.877451+0000","last_undegraded":"2024-03-28T16:49:39.877451+0000","last_fullsized":"2024-03-28T16:49:39.877451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.846084+0000","last_change":"2024-03-28T16:48:54.670541+0000","last_active":"2024-03-28T16:49:39.846084+0000","last_peered":"2024-03-28T16:49:39.846084+0000","last_clean":"2024-03-28T16:49:39.846084+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:49:39.846084+0000","last_undegraded":"2024-03-28T16:49:39.846084+0000","last_fullsized":"2024-03-28T16:49:39.846084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'5","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.468799+0000","last_change":"2024-03-28T16:48:56.671588+0000","last_active":"2024-03-28T16:50:35.468799+0000","last_peered":"2024-03-28T16:50:35.468799+0000","last_clean":"2024-03-28T16:50:35.468799+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:50:35.468799+0000","last_undegraded":"2024-03-28T16:50:35.468799+0000","last_fullsized":"2024-03-28T16:50:35.468799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582776+0000","last_change":"2024-03-28T16:48:54.673730+0000","last_active":"2024-03-28T16:49:37.582776+0000","last_peered":"2024-03-28T16:49:37.582776+0000","last_clean":"2024-03-28T16:49:37.582776+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:49:37.582776+0000","last_undegraded":"2024-03-28T16:49:37.582776+0000","last_fullsized":"2024-03-28T16:49:37.582776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'19","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.872837+0000","last_change":"2024-03-28T16:49:04.742579+0000","last_active":"2024-03-28T16:50:28.872837+0000","last_peered":"2024-03-28T16:50:28.872837+0000","last_clean":"2024-03-28T16:50:28.872837+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:50:28.872837+0000","last_undegraded":"2024-03-28T16:50:28.872837+0000","last_fullsized":"2024-03-28T16:50:28.872837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854866+0000","last_change":"2024-03-28T16:48:56.683934+0000","last_active":"2024-03-28T16:50:28.854866+0000","last_peered":"2024-03-28T16:50:28.854866+0000","last_clean":"2024-03-28T16:50:28.854866+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:50:28.854866+0000","last_undegraded":"2024-03-28T16:50:28.854866+0000","last_fullsized":"2024-03-28T16:50:28.854866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592706+0000","last_change":"2024-03-28T16:48:54.670023+0000","last_active":"2024-03-28T16:49:37.592706+0000","last_peered":"2024-03-28T16:49:37.592706+0000","last_clean":"2024-03-28T16:49:37.592706+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:49:37.592706+0000","last_undegraded":"2024-03-28T16:49:37.592706+0000","last_fullsized":"2024-03-28T16:49:37.592706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.137952+0000","last_change":"2024-03-28T16:50:50.138003+0000","last_active":"2024-03-28T16:50:50.137952+0000","last_peered":"2024-03-28T16:50:50.137952+0000","last_clean":"2024-03-28T16:50:50.137952+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:50:50.137952+0000","last_undegraded":"2024-03-28T16:50:50.137952+0000","last_fullsized":"2024-03-28T16:50:50.137952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.137765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:11.908122+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'31","reported_seq":77,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.257758+0000","last_change":"2024-03-28T16:49:00.727669+0000","last_active":"2024-03-28T16:50:42.257758+0000","last_peered":"2024-03-28T16:50:42.257758+0000","last_clean":"2024-03-28T16:50:42.257758+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:50:42.257758+0000","last_undegraded":"2024-03-28T16:50:42.257758+0000","last_fullsized":"2024-03-28T16:50:42.257758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.251316+0000","last_change":"2024-03-28T16:50:57.251369+0000","last_active":"2024-03-28T16:50:57.251316+0000","last_peered":"2024-03-28T16:50:57.251316+0000","last_clean":"2024-03-28T16:50:57.251316+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:50:57.251316+0000","last_undegraded":"2024-03-28T16:50:57.251316+0000","last_fullsized":"2024-03-28T16:50:57.251316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.251130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:45.232439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'6","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.940810+0000","last_change":"2024-03-28T16:49:04.742984+0000","last_active":"2024-03-28T16:50:22.940810+0000","last_peered":"2024-03-28T16:50:22.940810+0000","last_clean":"2024-03-28T16:50:22.940810+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:50:22.940810+0000","last_undegraded":"2024-03-28T16:50:22.940810+0000","last_fullsized":"2024-03-28T16:50:22.940810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":66,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854780+0000","last_change":"2024-03-28T16:48:56.671488+0000","last_active":"2024-03-28T16:50:28.854780+0000","last_peered":"2024-03-28T16:50:28.854780+0000","last_clean":"2024-03-28T16:50:28.854780+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:50:28.854780+0000","last_undegraded":"2024-03-28T16:50:28.854780+0000","last_fullsized":"2024-03-28T16:50:28.854780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591338+0000","last_change":"2024-03-28T16:48:54.655701+0000","last_active":"2024-03-28T16:49:37.591338+0000","last_peered":"2024-03-28T16:49:37.591338+0000","last_clean":"2024-03-28T16:49:37.591338+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:49:37.591338+0000","last_undegraded":"2024-03-28T16:49:37.591338+0000","last_fullsized":"2024-03-28T16:49:37.591338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586014+0000","last_change":"2024-03-28T16:48:58.689758+0000","last_active":"2024-03-28T16:49:37.586014+0000","last_peered":"2024-03-28T16:49:37.586014+0000","last_clean":"2024-03-28T16:49:37.586014+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:49:37.586014+0000","last_undegraded":"2024-03-28T16:49:37.586014+0000","last_fullsized":"2024-03-28T16:49:37.586014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"31'10","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.284126+0000","last_change":"2024-03-28T16:50:55.284276+0000","last_active":"2024-03-28T16:50:55.284126+0000","last_peered":"2024-03-28T16:50:55.284126+0000","last_clean":"2024-03-28T16:50:55.284126+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:50:55.284126+0000","last_undegraded":"2024-03-28T16:50:55.284126+0000","last_fullsized":"2024-03-28T16:50:55.284126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.283889+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:07.952223+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.156551+0000","last_change":"2024-03-28T16:50:49.156607+0000","last_active":"2024-03-28T16:50:49.156551+0000","last_peered":"2024-03-28T16:50:49.156551+0000","last_clean":"2024-03-28T16:50:49.156551+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:50:49.156551+0000","last_undegraded":"2024-03-28T16:50:49.156551+0000","last_fullsized":"2024-03-28T16:50:49.156551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.156374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:19.754691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584478+0000","last_change":"2024-03-28T16:49:04.743161+0000","last_active":"2024-03-28T16:49:37.584478+0000","last_peered":"2024-03-28T16:49:37.584478+0000","last_clean":"2024-03-28T16:49:37.584478+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:49:37.584478+0000","last_undegraded":"2024-03-28T16:49:37.584478+0000","last_fullsized":"2024-03-28T16:49:37.584478+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.204056+0000","last_change":"2024-03-28T16:48:56.676331+0000","last_active":"2024-03-28T16:50:23.204056+0000","last_peered":"2024-03-28T16:50:23.204056+0000","last_clean":"2024-03-28T16:50:23.204056+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:50:23.204056+0000","last_undegraded":"2024-03-28T16:50:23.204056+0000","last_fullsized":"2024-03-28T16:50:23.204056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.593084+0000","last_change":"2024-03-28T16:48:54.659330+0000","last_active":"2024-03-28T16:49:37.593084+0000","last_peered":"2024-03-28T16:49:37.593084+0000","last_clean":"2024-03-28T16:49:37.593084+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:49:37.593084+0000","last_undegraded":"2024-03-28T16:49:37.593084+0000","last_fullsized":"2024-03-28T16:49:37.593084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'5","reported_seq":206,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.038836+0000","last_change":"2024-03-28T16:48:58.687156+0000","last_active":"2024-03-28T16:50:42.038836+0000","last_peered":"2024-03-28T16:50:42.038836+0000","last_clean":"2024-03-28T16:50:42.038836+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:50:42.038836+0000","last_undegraded":"2024-03-28T16:50:42.038836+0000","last_fullsized":"2024-03-28T16:50:42.038836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":270,"num_read_kb":269,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'16","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.357611+0000","last_change":"2024-03-28T16:50:59.357686+0000","last_active":"2024-03-28T16:50:59.357611+0000","last_peered":"2024-03-28T16:50:59.357611+0000","last_clean":"2024-03-28T16:50:59.357611+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:50:59.357611+0000","last_undegraded":"2024-03-28T16:50:59.357611+0000","last_fullsized":"2024-03-28T16:50:59.357611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.357393+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:54.820495+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'249","reported_seq":333,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.382533+0000","last_change":"2024-03-28T16:49:02.737451+0000","last_active":"2024-03-28T16:50:42.382533+0000","last_peered":"2024-03-28T16:50:42.382533+0000","last_clean":"2024-03-28T16:50:42.382533+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:50:42.382533+0000","last_undegraded":"2024-03-28T16:50:42.382533+0000","last_fullsized":"2024-03-28T16:50:42.382533+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.350859+0000","last_change":"2024-03-28T16:51:04.350955+0000","last_active":"2024-03-28T16:51:04.350859+0000","last_peered":"2024-03-28T16:51:04.350859+0000","last_clean":"2024-03-28T16:51:04.350859+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:51:04.350859+0000","last_undegraded":"2024-03-28T16:51:04.350859+0000","last_fullsized":"2024-03-28T16:51:04.350859+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.350588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:52.331897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.591132+0000","last_change":"2024-03-28T16:48:56.697774+0000","last_active":"2024-03-28T16:49:37.591132+0000","last_peered":"2024-03-28T16:49:37.591132+0000","last_clean":"2024-03-28T16:49:37.591132+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:49:37.591132+0000","last_undegraded":"2024-03-28T16:49:37.591132+0000","last_fullsized":"2024-03-28T16:49:37.591132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.583691+0000","last_change":"2024-03-28T16:48:54.659035+0000","last_active":"2024-03-28T16:49:37.583691+0000","last_peered":"2024-03-28T16:49:37.583691+0000","last_clean":"2024-03-28T16:49:37.583691+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:49:37.583691+0000","last_undegraded":"2024-03-28T16:49:37.583691+0000","last_fullsized":"2024-03-28T16:49:37.583691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.124205+0000","last_change":"2024-03-28T16:50:49.124265+0000","last_active":"2024-03-28T16:50:49.124205+0000","last_peered":"2024-03-28T16:50:49.124205+0000","last_clean":"2024-03-28T16:50:49.124205+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:50:49.124205+0000","last_undegraded":"2024-03-28T16:50:49.124205+0000","last_fullsized":"2024-03-28T16:50:49.124205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.124014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.365150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'34","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.425129+0000","last_change":"2024-03-28T16:50:57.425199+0000","last_active":"2024-03-28T16:50:57.425129+0000","last_peered":"2024-03-28T16:50:57.425129+0000","last_clean":"2024-03-28T16:50:57.425129+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:57.425129+0000","last_undegraded":"2024-03-28T16:50:57.425129+0000","last_fullsized":"2024-03-28T16:50:57.425129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.424946+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:19.309481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'249","reported_seq":348,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.393933+0000","last_change":"2024-03-28T16:49:02.734376+0000","last_active":"2024-03-28T16:50:42.393933+0000","last_peered":"2024-03-28T16:50:42.393933+0000","last_clean":"2024-03-28T16:50:42.393933+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:50:42.393933+0000","last_undegraded":"2024-03-28T16:50:42.393933+0000","last_fullsized":"2024-03-28T16:50:42.393933+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581741+0000","last_change":"2024-03-28T16:48:56.693376+0000","last_active":"2024-03-28T16:49:37.581741+0000","last_peered":"2024-03-28T16:49:37.581741+0000","last_clean":"2024-03-28T16:49:37.581741+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:49:37.581741+0000","last_undegraded":"2024-03-28T16:49:37.581741+0000","last_fullsized":"2024-03-28T16:49:37.581741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.600292+0000","last_change":"2024-03-28T16:48:54.671982+0000","last_active":"2024-03-28T16:49:37.600292+0000","last_peered":"2024-03-28T16:49:37.600292+0000","last_clean":"2024-03-28T16:49:37.600292+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:49:37.600292+0000","last_undegraded":"2024-03-28T16:49:37.600292+0000","last_fullsized":"2024-03-28T16:49:37.600292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.302360+0000","last_change":"2024-03-28T16:51:00.302414+0000","last_active":"2024-03-28T16:51:00.302360+0000","last_peered":"2024-03-28T16:51:00.302360+0000","last_clean":"2024-03-28T16:51:00.302360+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:51:00.302360+0000","last_undegraded":"2024-03-28T16:51:00.302360+0000","last_fullsized":"2024-03-28T16:51:00.302360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.302183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:46.487988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'49","reported_seq":108,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.444761+0000","last_change":"2024-03-28T16:50:55.444852+0000","last_active":"2024-03-28T16:50:55.444761+0000","last_peered":"2024-03-28T16:50:55.444761+0000","last_clean":"2024-03-28T16:50:55.444761+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:55.444761+0000","last_undegraded":"2024-03-28T16:50:55.444761+0000","last_fullsized":"2024-03-28T16:50:55.444761+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'49","last_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_deep_scrub":"31'49","last_deep_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.444545+0000","objects_scrubbed":9,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:56.707599+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.195088+0000","last_change":"2024-03-28T16:50:53.195172+0000","last_active":"2024-03-28T16:50:53.195088+0000","last_peered":"2024-03-28T16:50:53.195088+0000","last_clean":"2024-03-28T16:50:53.195088+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:50:53.195088+0000","last_undegraded":"2024-03-28T16:50:53.195088+0000","last_fullsized":"2024-03-28T16:50:53.195088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.194894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:05.863228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.397727+0000","last_change":"2024-03-28T16:50:56.397864+0000","last_active":"2024-03-28T16:50:56.397727+0000","last_peered":"2024-03-28T16:50:56.397727+0000","last_clean":"2024-03-28T16:50:56.397727+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:56.397727+0000","last_undegraded":"2024-03-28T16:50:56.397727+0000","last_fullsized":"2024-03-28T16:50:56.397727+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.397465+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:26.995782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'22","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.730052+0000","last_change":"2024-03-28T16:48:56.678278+0000","last_active":"2024-03-28T16:50:22.730052+0000","last_peered":"2024-03-28T16:50:22.730052+0000","last_clean":"2024-03-28T16:50:22.730052+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:50:22.730052+0000","last_undegraded":"2024-03-28T16:50:22.730052+0000","last_fullsized":"2024-03-28T16:50:22.730052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.055056+0000","last_change":"2024-03-28T16:48:54.672277+0000","last_active":"2024-03-28T16:49:39.055056+0000","last_peered":"2024-03-28T16:49:39.055056+0000","last_clean":"2024-03-28T16:49:39.055056+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:49:39.055056+0000","last_undegraded":"2024-03-28T16:49:39.055056+0000","last_fullsized":"2024-03-28T16:49:39.055056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586227+0000","last_change":"2024-03-28T16:48:58.690263+0000","last_active":"2024-03-28T16:49:37.586227+0000","last_peered":"2024-03-28T16:49:37.586227+0000","last_clean":"2024-03-28T16:49:37.586227+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:49:37.586227+0000","last_undegraded":"2024-03-28T16:49:37.586227+0000","last_fullsized":"2024-03-28T16:49:37.586227+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'31","reported_seq":77,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.155098+0000","last_change":"2024-03-28T16:49:00.727195+0000","last_active":"2024-03-28T16:50:42.155098+0000","last_peered":"2024-03-28T16:50:42.155098+0000","last_clean":"2024-03-28T16:50:42.155098+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:50:42.155098+0000","last_undegraded":"2024-03-28T16:50:42.155098+0000","last_fullsized":"2024-03-28T16:50:42.155098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.182897+0000","last_change":"2024-03-28T16:50:51.183048+0000","last_active":"2024-03-28T16:50:51.182897+0000","last_peered":"2024-03-28T16:50:51.182897+0000","last_clean":"2024-03-28T16:50:51.182897+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:50:51.182897+0000","last_undegraded":"2024-03-28T16:50:51.182897+0000","last_fullsized":"2024-03-28T16:50:51.182897+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.182678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:48.071769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'4","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.320284+0000","last_change":"2024-03-28T16:51:05.320355+0000","last_active":"2024-03-28T16:51:05.320284+0000","last_peered":"2024-03-28T16:51:05.320284+0000","last_clean":"2024-03-28T16:51:05.320284+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:51:05.320284+0000","last_undegraded":"2024-03-28T16:51:05.320284+0000","last_fullsized":"2024-03-28T16:51:05.320284+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.320076+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:34.995213+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'3","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.468563+0000","last_change":"2024-03-28T16:48:56.681600+0000","last_active":"2024-03-28T16:50:35.468563+0000","last_peered":"2024-03-28T16:50:35.468563+0000","last_clean":"2024-03-28T16:50:35.468563+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:50:35.468563+0000","last_undegraded":"2024-03-28T16:50:35.468563+0000","last_fullsized":"2024-03-28T16:50:35.468563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588302+0000","last_change":"2024-03-28T16:48:54.676553+0000","last_active":"2024-03-28T16:49:37.588302+0000","last_peered":"2024-03-28T16:49:37.588302+0000","last_clean":"2024-03-28T16:49:37.588302+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:49:37.588302+0000","last_undegraded":"2024-03-28T16:49:37.588302+0000","last_fullsized":"2024-03-28T16:49:37.588302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592100+0000","last_change":"2024-03-28T16:48:58.682276+0000","last_active":"2024-03-28T16:49:37.592100+0000","last_peered":"2024-03-28T16:49:37.592100+0000","last_clean":"2024-03-28T16:49:37.592100+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:49:37.592100+0000","last_undegraded":"2024-03-28T16:49:37.592100+0000","last_fullsized":"2024-03-28T16:49:37.592100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'37","reported_seq":86,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.300574+0000","last_change":"2024-03-28T16:49:00.720840+0000","last_active":"2024-03-28T16:50:42.300574+0000","last_peered":"2024-03-28T16:50:42.300574+0000","last_clean":"2024-03-28T16:50:42.300574+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:50:42.300574+0000","last_undegraded":"2024-03-28T16:50:42.300574+0000","last_fullsized":"2024-03-28T16:50:42.300574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'249","reported_seq":351,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.171682+0000","last_change":"2024-03-28T16:50:52.171801+0000","last_active":"2024-03-28T16:50:52.171682+0000","last_peered":"2024-03-28T16:50:52.171682+0000","last_clean":"2024-03-28T16:50:52.171682+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:52.171682+0000","last_undegraded":"2024-03-28T16:50:52.171682+0000","last_fullsized":"2024-03-28T16:50:52.171682+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.171465+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:19.659817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.851699+0000","last_change":"2024-03-28T16:49:04.743207+0000","last_active":"2024-03-28T16:50:28.851699+0000","last_peered":"2024-03-28T16:50:28.851699+0000","last_clean":"2024-03-28T16:50:28.851699+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:50:28.851699+0000","last_undegraded":"2024-03-28T16:50:28.851699+0000","last_fullsized":"2024-03-28T16:50:28.851699+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.878763+0000","last_change":"2024-03-28T16:48:56.685083+0000","last_active":"2024-03-28T16:49:39.878763+0000","last_peered":"2024-03-28T16:49:39.878763+0000","last_clean":"2024-03-28T16:49:39.878763+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:49:39.878763+0000","last_undegraded":"2024-03-28T16:49:39.878763+0000","last_fullsized":"2024-03-28T16:49:39.878763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:38.402607+0000","last_change":"2024-03-28T16:48:54.658093+0000","last_active":"2024-03-28T16:49:38.402607+0000","last_peered":"2024-03-28T16:49:38.402607+0000","last_clean":"2024-03-28T16:49:38.402607+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:49:38.402607+0000","last_undegraded":"2024-03-28T16:49:38.402607+0000","last_fullsized":"2024-03-28T16:49:38.402607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586341+0000","last_change":"2024-03-28T16:48:58.682237+0000","last_active":"2024-03-28T16:49:37.586341+0000","last_peered":"2024-03-28T16:49:37.586341+0000","last_clean":"2024-03-28T16:49:37.586341+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:49:37.586341+0000","last_undegraded":"2024-03-28T16:49:37.586341+0000","last_fullsized":"2024-03-28T16:49:37.586341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'56","reported_seq":120,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.314011+0000","last_change":"2024-03-28T16:49:00.725827+0000","last_active":"2024-03-28T16:50:42.314011+0000","last_peered":"2024-03-28T16:50:42.314011+0000","last_clean":"2024-03-28T16:50:42.314011+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:50:42.314011+0000","last_undegraded":"2024-03-28T16:50:42.314011+0000","last_fullsized":"2024-03-28T16:50:42.314011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.180987+0000","last_change":"2024-03-28T16:50:54.181042+0000","last_active":"2024-03-28T16:50:54.180987+0000","last_peered":"2024-03-28T16:50:54.180987+0000","last_clean":"2024-03-28T16:50:54.180987+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:50:54.180987+0000","last_undegraded":"2024-03-28T16:50:54.180987+0000","last_fullsized":"2024-03-28T16:50:54.180987+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.180810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:17.080003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.864035+0000","last_change":"2024-03-28T16:49:04.723184+0000","last_active":"2024-03-28T16:50:28.864035+0000","last_peered":"2024-03-28T16:50:28.864035+0000","last_clean":"2024-03-28T16:50:28.864035+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:50:28.864035+0000","last_undegraded":"2024-03-28T16:50:28.864035+0000","last_fullsized":"2024-03-28T16:50:28.864035+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.877791+0000","last_change":"2024-03-28T16:48:56.683578+0000","last_active":"2024-03-28T16:49:39.877791+0000","last_peered":"2024-03-28T16:49:39.877791+0000","last_clean":"2024-03-28T16:49:39.877791+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:49:39.877791+0000","last_undegraded":"2024-03-28T16:49:39.877791+0000","last_fullsized":"2024-03-28T16:49:39.877791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588548+0000","last_change":"2024-03-28T16:48:54.664924+0000","last_active":"2024-03-28T16:49:37.588548+0000","last_peered":"2024-03-28T16:49:37.588548+0000","last_clean":"2024-03-28T16:49:37.588548+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:49:37.588548+0000","last_undegraded":"2024-03-28T16:49:37.588548+0000","last_fullsized":"2024-03-28T16:49:37.588548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.364981+0000","last_change":"2024-03-28T16:51:05.365035+0000","last_active":"2024-03-28T16:51:05.364981+0000","last_peered":"2024-03-28T16:51:05.364981+0000","last_clean":"2024-03-28T16:51:05.364981+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:51:05.364981+0000","last_undegraded":"2024-03-28T16:51:05.364981+0000","last_fullsized":"2024-03-28T16:51:05.364981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.364765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:50.121796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'37","reported_seq":92,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.221133+0000","last_change":"2024-03-28T16:50:51.221214+0000","last_active":"2024-03-28T16:50:51.221133+0000","last_peered":"2024-03-28T16:50:51.221133+0000","last_clean":"2024-03-28T16:50:51.221133+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:51.221133+0000","last_undegraded":"2024-03-28T16:50:51.221133+0000","last_fullsized":"2024-03-28T16:50:51.221133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'37","last_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_deep_scrub":"31'37","last_deep_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.220960+0000","objects_scrubbed":5,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:47.323997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587933+0000","last_change":"2024-03-28T16:49:02.739236+0000","last_active":"2024-03-28T16:49:37.587933+0000","last_peered":"2024-03-28T16:49:37.587933+0000","last_clean":"2024-03-28T16:49:37.587933+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:49:37.587933+0000","last_undegraded":"2024-03-28T16:49:37.587933+0000","last_fullsized":"2024-03-28T16:49:37.587933+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580236+0000","last_change":"2024-03-28T16:49:04.748645+0000","last_active":"2024-03-28T16:49:37.580236+0000","last_peered":"2024-03-28T16:49:37.580236+0000","last_clean":"2024-03-28T16:49:37.580236+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:49:37.580236+0000","last_undegraded":"2024-03-28T16:49:37.580236+0000","last_fullsized":"2024-03-28T16:49:37.580236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.193629+0000","last_change":"2024-03-28T16:51:00.193692+0000","last_active":"2024-03-28T16:51:00.193629+0000","last_peered":"2024-03-28T16:51:00.193629+0000","last_clean":"2024-03-28T16:51:00.193629+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:51:00.193629+0000","last_undegraded":"2024-03-28T16:51:00.193629+0000","last_fullsized":"2024-03-28T16:51:00.193629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.193439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:35.074661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594151+0000","last_change":"2024-03-28T16:48:54.673299+0000","last_active":"2024-03-28T16:49:37.594151+0000","last_peered":"2024-03-28T16:49:37.594151+0000","last_clean":"2024-03-28T16:49:37.594151+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:49:37.594151+0000","last_undegraded":"2024-03-28T16:49:37.594151+0000","last_fullsized":"2024-03-28T16:49:37.594151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589032+0000","last_change":"2024-03-28T16:48:58.683855+0000","last_active":"2024-03-28T16:49:37.589032+0000","last_peered":"2024-03-28T16:49:37.589032+0000","last_clean":"2024-03-28T16:49:37.589032+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:49:37.589032+0000","last_undegraded":"2024-03-28T16:49:37.589032+0000","last_fullsized":"2024-03-28T16:49:37.589032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'44","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.389378+0000","last_change":"2024-03-28T16:51:01.389452+0000","last_active":"2024-03-28T16:51:01.389378+0000","last_peered":"2024-03-28T16:51:01.389378+0000","last_clean":"2024-03-28T16:51:01.389378+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:51:01.389378+0000","last_undegraded":"2024-03-28T16:51:01.389378+0000","last_fullsized":"2024-03-28T16:51:01.389378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.389173+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:28.877525+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'249","reported_seq":334,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.428702+0000","last_change":"2024-03-28T16:50:47.428834+0000","last_active":"2024-03-28T16:50:47.428702+0000","last_peered":"2024-03-28T16:50:47.428702+0000","last_clean":"2024-03-28T16:50:47.428702+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:47.428702+0000","last_undegraded":"2024-03-28T16:50:47.428702+0000","last_fullsized":"2024-03-28T16:50:47.428702+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.428456+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:01.404074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'13","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.484553+0000","last_change":"2024-03-28T16:49:04.743257+0000","last_active":"2024-03-28T16:50:35.484553+0000","last_peered":"2024-03-28T16:50:35.484553+0000","last_clean":"2024-03-28T16:50:35.484553+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:50:35.484553+0000","last_undegraded":"2024-03-28T16:50:35.484553+0000","last_fullsized":"2024-03-28T16:50:35.484553+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854218+0000","last_change":"2024-03-28T16:48:56.690053+0000","last_active":"2024-03-28T16:50:28.854218+0000","last_peered":"2024-03-28T16:50:28.854218+0000","last_clean":"2024-03-28T16:50:28.854218+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:50:28.854218+0000","last_undegraded":"2024-03-28T16:50:28.854218+0000","last_fullsized":"2024-03-28T16:50:28.854218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589149+0000","last_change":"2024-03-28T16:48:54.664571+0000","last_active":"2024-03-28T16:49:37.589149+0000","last_peered":"2024-03-28T16:49:37.589149+0000","last_clean":"2024-03-28T16:49:37.589149+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:49:37.589149+0000","last_undegraded":"2024-03-28T16:49:37.589149+0000","last_fullsized":"2024-03-28T16:49:37.589149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'3","reported_seq":383,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.054359+0000","last_change":"2024-03-28T16:49:37.622013+0000","last_active":"2024-03-28T16:50:42.054359+0000","last_peered":"2024-03-28T16:50:42.054359+0000","last_clean":"2024-03-28T16:50:42.054359+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:50:42.054359+0000","last_undegraded":"2024-03-28T16:50:42.054359+0000","last_fullsized":"2024-03-28T16:50:42.054359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":340,"num_read_kb":340,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'10","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587380+0000","last_change":"2024-03-28T16:49:00.725908+0000","last_active":"2024-03-28T16:49:37.587380+0000","last_peered":"2024-03-28T16:49:37.587380+0000","last_clean":"2024-03-28T16:49:37.587380+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:49:37.587380+0000","last_undegraded":"2024-03-28T16:49:37.587380+0000","last_fullsized":"2024-03-28T16:49:37.587380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'249","reported_seq":357,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.109623+0000","last_change":"2024-03-28T16:50:48.109690+0000","last_active":"2024-03-28T16:50:48.109623+0000","last_peered":"2024-03-28T16:50:48.109623+0000","last_clean":"2024-03-28T16:50:48.109623+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:48.109623+0000","last_undegraded":"2024-03-28T16:50:48.109623+0000","last_fullsized":"2024-03-28T16:50:48.109623+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.109433+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:49.372487+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589822+0000","last_change":"2024-03-28T16:49:04.746465+0000","last_active":"2024-03-28T16:49:37.589822+0000","last_peered":"2024-03-28T16:49:37.589822+0000","last_clean":"2024-03-28T16:49:37.589822+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:49:37.589822+0000","last_undegraded":"2024-03-28T16:49:37.589822+0000","last_fullsized":"2024-03-28T16:49:37.589822+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.203599+0000","last_change":"2024-03-28T16:48:56.681161+0000","last_active":"2024-03-28T16:50:23.203599+0000","last_peered":"2024-03-28T16:50:23.203599+0000","last_clean":"2024-03-28T16:50:23.203599+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:50:23.203599+0000","last_undegraded":"2024-03-28T16:50:23.203599+0000","last_fullsized":"2024-03-28T16:50:23.203599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580694+0000","last_change":"2024-03-28T16:48:54.676487+0000","last_active":"2024-03-28T16:49:37.580694+0000","last_peered":"2024-03-28T16:49:37.580694+0000","last_clean":"2024-03-28T16:49:37.580694+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:49:37.580694+0000","last_undegraded":"2024-03-28T16:49:37.580694+0000","last_fullsized":"2024-03-28T16:49:37.580694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.165428+0000","last_change":"2024-03-28T16:50:48.165481+0000","last_active":"2024-03-28T16:50:48.165428+0000","last_peered":"2024-03-28T16:50:48.165428+0000","last_clean":"2024-03-28T16:50:48.165428+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:50:48.165428+0000","last_undegraded":"2024-03-28T16:50:48.165428+0000","last_fullsized":"2024-03-28T16:50:48.165428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.165239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:40:14.898173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'46","reported_seq":104,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.272546+0000","last_change":"2024-03-28T16:50:56.272646+0000","last_active":"2024-03-28T16:50:56.272546+0000","last_peered":"2024-03-28T16:50:56.272546+0000","last_clean":"2024-03-28T16:50:56.272546+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:56.272546+0000","last_undegraded":"2024-03-28T16:50:56.272546+0000","last_fullsized":"2024-03-28T16:50:56.272546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'46","last_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_deep_scrub":"31'46","last_deep_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.272353+0000","objects_scrubbed":8,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:19.171546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.431476+0000","last_change":"2024-03-28T16:50:53.431537+0000","last_active":"2024-03-28T16:50:53.431476+0000","last_peered":"2024-03-28T16:50:53.431476+0000","last_clean":"2024-03-28T16:50:53.431476+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:50:53.431476+0000","last_undegraded":"2024-03-28T16:50:53.431476+0000","last_fullsized":"2024-03-28T16:50:53.431476+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.431293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:35.681957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.348931+0000","last_change":"2024-03-28T16:51:01.348998+0000","last_active":"2024-03-28T16:51:01.348931+0000","last_peered":"2024-03-28T16:51:01.348931+0000","last_clean":"2024-03-28T16:51:01.348931+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:51:01.348931+0000","last_undegraded":"2024-03-28T16:51:01.348931+0000","last_fullsized":"2024-03-28T16:51:01.348931+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.348719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:30.188118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871992+0000","last_change":"2024-03-28T16:48:56.693994+0000","last_active":"2024-03-28T16:50:46.871992+0000","last_peered":"2024-03-28T16:50:46.871992+0000","last_clean":"2024-03-28T16:50:46.871992+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:50:46.871992+0000","last_undegraded":"2024-03-28T16:50:46.871992+0000","last_fullsized":"2024-03-28T16:50:46.871992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":200,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.592395+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:50:45.592395+0000","last_peered":"2024-03-28T16:50:45.592395+0000","last_clean":"2024-03-28T16:50:45.592395+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:50:45.592395+0000","last_undegraded":"2024-03-28T16:50:45.592395+0000","last_fullsized":"2024-03-28T16:50:45.592395+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870781+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:50:46.870781+0000","last_peered":"2024-03-28T16:50:46.870781+0000","last_clean":"2024-03-28T16:50:46.870781+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:50:46.870781+0000","last_undegraded":"2024-03-28T16:50:46.870781+0000","last_fullsized":"2024-03-28T16:50:46.870781+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870947+0000","last_change":"2024-03-28T16:48:54.666099+0000","last_active":"2024-03-28T16:50:46.870947+0000","last_peered":"2024-03-28T16:50:46.870947+0000","last_clean":"2024-03-28T16:50:46.870947+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:50:46.870947+0000","last_undegraded":"2024-03-28T16:50:46.870947+0000","last_fullsized":"2024-03-28T16:50:46.870947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.295412+0000","last_change":"2024-03-28T16:51:03.295412+0000","last_active":"2024-03-28T16:51:03.295412+0000","last_peered":"2024-03-28T16:51:03.295412+0000","last_clean":"2024-03-28T16:51:03.295412+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:51:03.295412+0000","last_undegraded":"2024-03-28T16:51:03.295412+0000","last_fullsized":"2024-03-28T16:51:03.295412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.295139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:08.548996+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'81","reported_seq":170,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.160003+0000","last_change":"2024-03-28T16:50:53.160163+0000","last_active":"2024-03-28T16:50:53.160003+0000","last_peered":"2024-03-28T16:50:53.160003+0000","last_clean":"2024-03-28T16:50:53.160003+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:53.160003+0000","last_undegraded":"2024-03-28T16:50:53.160003+0000","last_fullsized":"2024-03-28T16:50:53.160003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'81","last_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_deep_scrub":"31'81","last_deep_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.159787+0000","objects_scrubbed":5,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:32.543735+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.290614+0000","last_change":"2024-03-28T16:50:57.290671+0000","last_active":"2024-03-28T16:50:57.290614+0000","last_peered":"2024-03-28T16:50:57.290614+0000","last_clean":"2024-03-28T16:50:57.290614+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:50:57.290614+0000","last_undegraded":"2024-03-28T16:50:57.290614+0000","last_fullsized":"2024-03-28T16:50:57.290614+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.290416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:32.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'52","reported_seq":115,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.478660+0000","last_change":"2024-03-28T16:49:04.752277+0000","last_active":"2024-03-28T16:50:35.478660+0000","last_peered":"2024-03-28T16:50:35.478660+0000","last_clean":"2024-03-28T16:50:35.478660+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:50:35.478660+0000","last_undegraded":"2024-03-28T16:50:35.478660+0000","last_fullsized":"2024-03-28T16:50:35.478660+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'11","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469786+0000","last_change":"2024-03-28T16:49:37.614693+0000","last_active":"2024-03-28T16:50:35.469786+0000","last_peered":"2024-03-28T16:50:35.469786+0000","last_clean":"2024-03-28T16:50:35.469786+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:50:35.469786+0000","last_undegraded":"2024-03-28T16:50:35.469786+0000","last_fullsized":"2024-03-28T16:50:35.469786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870684+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:50:46.870684+0000","last_peered":"2024-03-28T16:50:46.870684+0000","last_clean":"2024-03-28T16:50:46.870684+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:50:46.870684+0000","last_undegraded":"2024-03-28T16:50:46.870684+0000","last_fullsized":"2024-03-28T16:50:46.870684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532168+0000","last_change":"2024-03-28T16:48:54.662603+0000","last_active":"2024-03-28T16:50:45.532168+0000","last_peered":"2024-03-28T16:50:45.532168+0000","last_clean":"2024-03-28T16:50:45.532168+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:50:45.532168+0000","last_undegraded":"2024-03-28T16:50:45.532168+0000","last_fullsized":"2024-03-28T16:50:45.532168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.191021+0000","last_change":"2024-03-28T16:50:47.191091+0000","last_active":"2024-03-28T16:50:47.191021+0000","last_peered":"2024-03-28T16:50:47.191021+0000","last_clean":"2024-03-28T16:50:47.191021+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:50:47.191021+0000","last_undegraded":"2024-03-28T16:50:47.191021+0000","last_fullsized":"2024-03-28T16:50:47.191021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.190844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:09.946965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'260","reported_seq":455,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.259946+0000","last_change":"2024-03-28T16:50:54.259946+0000","last_active":"2024-03-28T16:50:54.259946+0000","last_peered":"2024-03-28T16:50:54.259946+0000","last_clean":"2024-03-28T16:50:54.259946+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:54.259946+0000","last_undegraded":"2024-03-28T16:50:54.259946+0000","last_fullsized":"2024-03-28T16:50:54.259946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'260","last_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_deep_scrub":"31'260","last_deep_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.259667+0000","objects_scrubbed":10,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:21.748018+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":342,"num_read_kb":327,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582733+0000","last_change":"2024-03-28T16:49:02.740444+0000","last_active":"2024-03-28T16:49:37.582733+0000","last_peered":"2024-03-28T16:49:37.582733+0000","last_clean":"2024-03-28T16:49:37.582733+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:49:37.582733+0000","last_undegraded":"2024-03-28T16:49:37.582733+0000","last_fullsized":"2024-03-28T16:49:37.582733+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'4","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.424691+0000","last_change":"2024-03-28T16:49:04.744802+0000","last_active":"2024-03-28T16:50:30.424691+0000","last_peered":"2024-03-28T16:50:30.424691+0000","last_clean":"2024-03-28T16:50:30.424691+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:50:30.424691+0000","last_undegraded":"2024-03-28T16:50:30.424691+0000","last_fullsized":"2024-03-28T16:50:30.424691+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.563689+0000","last_change":"2024-03-28T16:48:56.689836+0000","last_active":"2024-03-28T16:49:40.563689+0000","last_peered":"2024-03-28T16:49:40.563689+0000","last_clean":"2024-03-28T16:49:40.563689+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:49:40.563689+0000","last_undegraded":"2024-03-28T16:49:40.563689+0000","last_fullsized":"2024-03-28T16:49:40.563689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.411785+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:50:46.411785+0000","last_peered":"2024-03-28T16:50:46.411785+0000","last_clean":"2024-03-28T16:50:46.411785+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:50:46.411785+0000","last_undegraded":"2024-03-28T16:50:46.411785+0000","last_fullsized":"2024-03-28T16:50:46.411785+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531810+0000","last_change":"2024-03-28T16:48:54.663058+0000","last_active":"2024-03-28T16:50:45.531810+0000","last_peered":"2024-03-28T16:50:45.531810+0000","last_clean":"2024-03-28T16:50:45.531810+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:50:45.531810+0000","last_undegraded":"2024-03-28T16:50:45.531810+0000","last_fullsized":"2024-03-28T16:50:45.531810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"22'1","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.599460+0000","last_change":"2024-03-28T16:48:58.687188+0000","last_active":"2024-03-28T16:49:37.599460+0000","last_peered":"2024-03-28T16:49:37.599460+0000","last_clean":"2024-03-28T16:49:37.599460+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:49:37.599460+0000","last_undegraded":"2024-03-28T16:49:37.599460+0000","last_fullsized":"2024-03-28T16:49:37.599460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'23","reported_seq":82,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.409006+0000","last_change":"2024-03-28T16:50:54.409006+0000","last_active":"2024-03-28T16:50:54.409006+0000","last_peered":"2024-03-28T16:50:54.409006+0000","last_clean":"2024-03-28T16:50:54.409006+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:54.409006+0000","last_undegraded":"2024-03-28T16:50:54.409006+0000","last_fullsized":"2024-03-28T16:50:54.409006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'23","last_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_deep_scrub":"31'23","last_deep_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.408718+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:56.786363+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.341112+0000","last_change":"2024-03-28T16:51:01.341170+0000","last_active":"2024-03-28T16:51:01.341112+0000","last_peered":"2024-03-28T16:51:01.341112+0000","last_clean":"2024-03-28T16:51:01.341112+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:51:01.341112+0000","last_undegraded":"2024-03-28T16:51:01.341112+0000","last_fullsized":"2024-03-28T16:51:01.341112+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.340925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:06.594781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582631+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:49:37.582631+0000","last_peered":"2024-03-28T16:49:37.582631+0000","last_clean":"2024-03-28T16:49:37.582631+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:49:37.582631+0000","last_undegraded":"2024-03-28T16:49:37.582631+0000","last_fullsized":"2024-03-28T16:49:37.582631+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532146+0000","last_change":"2024-03-28T16:48:56.683769+0000","last_active":"2024-03-28T16:50:45.532146+0000","last_peered":"2024-03-28T16:50:45.532146+0000","last_clean":"2024-03-28T16:50:45.532146+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:50:45.532146+0000","last_undegraded":"2024-03-28T16:50:45.532146+0000","last_fullsized":"2024-03-28T16:50:45.532146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870800+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:50:46.870800+0000","last_peered":"2024-03-28T16:50:46.870800+0000","last_clean":"2024-03-28T16:50:46.870800+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:50:46.870800+0000","last_undegraded":"2024-03-28T16:50:46.870800+0000","last_fullsized":"2024-03-28T16:50:46.870800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.411963+0000","last_change":"2024-03-28T16:48:54.666472+0000","last_active":"2024-03-28T16:50:46.411963+0000","last_peered":"2024-03-28T16:50:46.411963+0000","last_clean":"2024-03-28T16:50:46.411963+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:50:46.411963+0000","last_undegraded":"2024-03-28T16:50:46.411963+0000","last_fullsized":"2024-03-28T16:50:46.411963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587487+0000","last_change":"2024-03-28T16:48:58.688005+0000","last_active":"2024-03-28T16:49:37.587487+0000","last_peered":"2024-03-28T16:49:37.587487+0000","last_clean":"2024-03-28T16:49:37.587487+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:49:37.587487+0000","last_undegraded":"2024-03-28T16:49:37.587487+0000","last_fullsized":"2024-03-28T16:49:37.587487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'19","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.195703+0000","last_change":"2024-03-28T16:50:51.195780+0000","last_active":"2024-03-28T16:50:51.195703+0000","last_peered":"2024-03-28T16:50:51.195703+0000","last_clean":"2024-03-28T16:50:51.195703+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:50:51.195703+0000","last_undegraded":"2024-03-28T16:50:51.195703+0000","last_fullsized":"2024-03-28T16:50:51.195703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'19","last_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_deep_scrub":"31'19","last_deep_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.195522+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:16.598840+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588166+0000","last_change":"2024-03-28T16:49:02.741194+0000","last_active":"2024-03-28T16:49:37.588166+0000","last_peered":"2024-03-28T16:49:37.588166+0000","last_clean":"2024-03-28T16:49:37.588166+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:49:37.588166+0000","last_undegraded":"2024-03-28T16:49:37.588166+0000","last_fullsized":"2024-03-28T16:49:37.588166+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585964+0000","last_change":"2024-03-28T16:49:04.741719+0000","last_active":"2024-03-28T16:49:37.585964+0000","last_peered":"2024-03-28T16:49:37.585964+0000","last_clean":"2024-03-28T16:49:37.585964+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:49:37.585964+0000","last_undegraded":"2024-03-28T16:49:37.585964+0000","last_fullsized":"2024-03-28T16:49:37.585964+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985149+0000","last_change":"2024-03-28T16:48:56.687243+0000","last_active":"2024-03-28T16:50:45.985149+0000","last_peered":"2024-03-28T16:50:45.985149+0000","last_clean":"2024-03-28T16:50:45.985149+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:50:45.985149+0000","last_undegraded":"2024-03-28T16:50:45.985149+0000","last_fullsized":"2024-03-28T16:50:45.985149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.411817+0000","last_change":"2024-03-28T16:48:54.657105+0000","last_active":"2024-03-28T16:50:46.411817+0000","last_peered":"2024-03-28T16:50:46.411817+0000","last_clean":"2024-03-28T16:50:46.411817+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:50:46.411817+0000","last_undegraded":"2024-03-28T16:50:46.411817+0000","last_fullsized":"2024-03-28T16:50:46.411817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531684+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:50:45.531684+0000","last_peered":"2024-03-28T16:50:45.531684+0000","last_clean":"2024-03-28T16:50:45.531684+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:50:45.531684+0000","last_undegraded":"2024-03-28T16:50:45.531684+0000","last_fullsized":"2024-03-28T16:50:45.531684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589014+0000","last_change":"2024-03-28T16:48:58.690031+0000","last_active":"2024-03-28T16:49:37.589014+0000","last_peered":"2024-03-28T16:49:37.589014+0000","last_clean":"2024-03-28T16:49:37.589014+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:49:37.589014+0000","last_undegraded":"2024-03-28T16:49:37.589014+0000","last_fullsized":"2024-03-28T16:49:37.589014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'69","reported_seq":145,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.306941+0000","last_change":"2024-03-28T16:50:46.306941+0000","last_active":"2024-03-28T16:50:46.306941+0000","last_peered":"2024-03-28T16:50:46.306941+0000","last_clean":"2024-03-28T16:50:46.306941+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:46.306941+0000","last_undegraded":"2024-03-28T16:50:46.306941+0000","last_fullsized":"2024-03-28T16:50:46.306941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'69","last_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_deep_scrub":"31'69","last_deep_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.306710+0000","objects_scrubbed":8,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:28.604401+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.441235+0000","last_change":"2024-03-28T16:50:51.441291+0000","last_active":"2024-03-28T16:50:51.441235+0000","last_peered":"2024-03-28T16:50:51.441235+0000","last_clean":"2024-03-28T16:50:51.441235+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:50:51.441235+0000","last_undegraded":"2024-03-28T16:50:51.441235+0000","last_fullsized":"2024-03-28T16:50:51.441235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.441051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:13:01.864029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'9","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.421996+0000","last_change":"2024-03-28T16:49:04.741551+0000","last_active":"2024-03-28T16:50:30.421996+0000","last_peered":"2024-03-28T16:50:30.421996+0000","last_clean":"2024-03-28T16:50:30.421996+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:50:30.421996+0000","last_undegraded":"2024-03-28T16:50:30.421996+0000","last_fullsized":"2024-03-28T16:50:30.421996+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985239+0000","last_change":"2024-03-28T16:48:56.683849+0000","last_active":"2024-03-28T16:50:45.985239+0000","last_peered":"2024-03-28T16:50:45.985239+0000","last_clean":"2024-03-28T16:50:45.985239+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:50:45.985239+0000","last_undegraded":"2024-03-28T16:50:45.985239+0000","last_fullsized":"2024-03-28T16:50:45.985239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531753+0000","last_change":"2024-03-28T16:48:54.663076+0000","last_active":"2024-03-28T16:50:45.531753+0000","last_peered":"2024-03-28T16:50:45.531753+0000","last_clean":"2024-03-28T16:50:45.531753+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:50:45.531753+0000","last_undegraded":"2024-03-28T16:50:45.531753+0000","last_fullsized":"2024-03-28T16:50:45.531753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870790+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:50:46.870790+0000","last_peered":"2024-03-28T16:50:46.870790+0000","last_clean":"2024-03-28T16:50:46.870790+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:50:46.870790+0000","last_undegraded":"2024-03-28T16:50:46.870790+0000","last_fullsized":"2024-03-28T16:50:46.870790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.158975+0000","last_change":"2024-03-28T16:50:55.159039+0000","last_active":"2024-03-28T16:50:55.158975+0000","last_peered":"2024-03-28T16:50:55.158975+0000","last_clean":"2024-03-28T16:50:55.158975+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:50:55.158975+0000","last_undegraded":"2024-03-28T16:50:55.158975+0000","last_fullsized":"2024-03-28T16:50:55.158975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.158776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:53:10.542165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"31'21","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.252537+0000","last_change":"2024-03-28T16:50:52.252537+0000","last_active":"2024-03-28T16:50:52.252537+0000","last_peered":"2024-03-28T16:50:52.252537+0000","last_clean":"2024-03-28T16:50:52.252537+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:50:52.252537+0000","last_undegraded":"2024-03-28T16:50:52.252537+0000","last_fullsized":"2024-03-28T16:50:52.252537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'21","last_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_deep_scrub":"31'21","last_deep_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.252240+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:47.715342+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'249","reported_seq":304,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.104945+0000","last_change":"2024-03-28T16:50:46.104945+0000","last_active":"2024-03-28T16:50:46.104945+0000","last_peered":"2024-03-28T16:50:46.104945+0000","last_clean":"2024-03-28T16:50:46.104945+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:46.104945+0000","last_undegraded":"2024-03-28T16:50:46.104945+0000","last_fullsized":"2024-03-28T16:50:46.104945+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.104655+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:39.533970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.200250+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:23.200250+0000","last_peered":"2024-03-28T16:50:23.200250+0000","last_clean":"2024-03-28T16:50:23.200250+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:23.200250+0000","last_undegraded":"2024-03-28T16:50:23.200250+0000","last_fullsized":"2024-03-28T16:50:23.200250+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871954+0000","last_change":"2024-03-28T16:48:56.693531+0000","last_active":"2024-03-28T16:50:46.871954+0000","last_peered":"2024-03-28T16:50:46.871954+0000","last_clean":"2024-03-28T16:50:46.871954+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:50:46.871954+0000","last_undegraded":"2024-03-28T16:50:46.871954+0000","last_fullsized":"2024-03-28T16:50:46.871954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.854745+0000","last_change":"2024-03-28T16:48:54.662217+0000","last_active":"2024-03-28T16:49:39.854745+0000","last_peered":"2024-03-28T16:49:39.854745+0000","last_clean":"2024-03-28T16:49:39.854745+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:49:39.854745+0000","last_undegraded":"2024-03-28T16:49:39.854745+0000","last_fullsized":"2024-03-28T16:49:39.854745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531684+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:50:45.531684+0000","last_peered":"2024-03-28T16:50:45.531684+0000","last_clean":"2024-03-28T16:50:45.531684+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:50:45.531684+0000","last_undegraded":"2024-03-28T16:50:45.531684+0000","last_fullsized":"2024-03-28T16:50:45.531684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.163811+0000","last_change":"2024-03-28T16:50:54.163965+0000","last_active":"2024-03-28T16:50:54.163811+0000","last_peered":"2024-03-28T16:50:54.163811+0000","last_clean":"2024-03-28T16:50:54.163811+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:50:54.163811+0000","last_undegraded":"2024-03-28T16:50:54.163811+0000","last_fullsized":"2024-03-28T16:50:54.163811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.163592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:27.253015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'10","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589752+0000","last_change":"2024-03-28T16:49:00.721989+0000","last_active":"2024-03-28T16:49:37.589752+0000","last_peered":"2024-03-28T16:49:37.589752+0000","last_clean":"2024-03-28T16:49:37.589752+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:49:37.589752+0000","last_undegraded":"2024-03-28T16:49:37.589752+0000","last_fullsized":"2024-03-28T16:49:37.589752+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'249","reported_seq":336,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.232977+0000","last_change":"2024-03-28T16:50:49.233041+0000","last_active":"2024-03-28T16:50:49.232977+0000","last_peered":"2024-03-28T16:50:49.232977+0000","last_clean":"2024-03-28T16:50:49.232977+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:49.232977+0000","last_undegraded":"2024-03-28T16:50:49.232977+0000","last_fullsized":"2024-03-28T16:50:49.232977+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.232798+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:57.227518+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585250+0000","last_change":"2024-03-28T16:49:04.739259+0000","last_active":"2024-03-28T16:49:37.585250+0000","last_peered":"2024-03-28T16:49:37.585250+0000","last_clean":"2024-03-28T16:49:37.585250+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:49:37.585250+0000","last_undegraded":"2024-03-28T16:49:37.585250+0000","last_fullsized":"2024-03-28T16:49:37.585250+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":66,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412168+0000","last_change":"2024-03-28T16:48:56.682608+0000","last_active":"2024-03-28T16:50:46.412168+0000","last_peered":"2024-03-28T16:50:46.412168+0000","last_clean":"2024-03-28T16:50:46.412168+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:50:46.412168+0000","last_undegraded":"2024-03-28T16:50:46.412168+0000","last_fullsized":"2024-03-28T16:50:46.412168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984622+0000","last_change":"2024-03-28T16:48:54.675316+0000","last_active":"2024-03-28T16:50:45.984622+0000","last_peered":"2024-03-28T16:50:45.984622+0000","last_clean":"2024-03-28T16:50:45.984622+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:50:45.984622+0000","last_undegraded":"2024-03-28T16:50:45.984622+0000","last_fullsized":"2024-03-28T16:50:45.984622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870766+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:50:46.870766+0000","last_peered":"2024-03-28T16:50:46.870766+0000","last_clean":"2024-03-28T16:50:46.870766+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:50:46.870766+0000","last_undegraded":"2024-03-28T16:50:46.870766+0000","last_fullsized":"2024-03-28T16:50:46.870766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.166685+0000","last_change":"2024-03-28T16:50:56.166685+0000","last_active":"2024-03-28T16:50:56.166685+0000","last_peered":"2024-03-28T16:50:56.166685+0000","last_clean":"2024-03-28T16:50:56.166685+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:50:56.166685+0000","last_undegraded":"2024-03-28T16:50:56.166685+0000","last_fullsized":"2024-03-28T16:50:56.166685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.166428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:06.567829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'90","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.384285+0000","last_change":"2024-03-28T16:50:58.384285+0000","last_active":"2024-03-28T16:50:58.384285+0000","last_peered":"2024-03-28T16:50:58.384285+0000","last_clean":"2024-03-28T16:50:58.384285+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:58.384285+0000","last_undegraded":"2024-03-28T16:50:58.384285+0000","last_fullsized":"2024-03-28T16:50:58.384285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'90","last_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_deep_scrub":"31'90","last_deep_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.383995+0000","objects_scrubbed":8,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:54.487031+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.285302+0000","last_change":"2024-03-28T16:50:47.285325+0000","last_active":"2024-03-28T16:50:47.285302+0000","last_peered":"2024-03-28T16:50:47.285302+0000","last_clean":"2024-03-28T16:50:47.285302+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:50:47.285302+0000","last_undegraded":"2024-03-28T16:50:47.285302+0000","last_fullsized":"2024-03-28T16:50:47.285302+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.285202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:46:10.170506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.594355+0000","last_change":"2024-03-28T16:49:04.754190+0000","last_active":"2024-03-28T16:49:37.594355+0000","last_peered":"2024-03-28T16:49:37.594355+0000","last_clean":"2024-03-28T16:49:37.594355+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:49:37.594355+0000","last_undegraded":"2024-03-28T16:49:37.594355+0000","last_fullsized":"2024-03-28T16:49:37.594355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.344559+0000","last_change":"2024-03-28T16:51:00.344614+0000","last_active":"2024-03-28T16:51:00.344559+0000","last_peered":"2024-03-28T16:51:00.344559+0000","last_clean":"2024-03-28T16:51:00.344559+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:51:00.344559+0000","last_undegraded":"2024-03-28T16:51:00.344559+0000","last_fullsized":"2024-03-28T16:51:00.344559+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.344363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:57.233454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.204232+0000","last_change":"2024-03-28T16:48:56.680906+0000","last_active":"2024-03-28T16:50:23.204232+0000","last_peered":"2024-03-28T16:50:23.204232+0000","last_clean":"2024-03-28T16:50:23.204232+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:50:23.204232+0000","last_undegraded":"2024-03-28T16:50:23.204232+0000","last_fullsized":"2024-03-28T16:50:23.204232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587214+0000","last_change":"2024-03-28T16:48:54.663088+0000","last_active":"2024-03-28T16:49:37.587214+0000","last_peered":"2024-03-28T16:49:37.587214+0000","last_clean":"2024-03-28T16:49:37.587214+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:49:37.587214+0000","last_undegraded":"2024-03-28T16:49:37.587214+0000","last_fullsized":"2024-03-28T16:49:37.587214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.608975+0000","last_change":"2024-03-28T16:49:37.606876+0000","last_active":"2024-03-28T16:49:37.608975+0000","last_peered":"2024-03-28T16:49:37.608975+0000","last_clean":"2024-03-28T16:49:37.608975+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:49:37.608975+0000","last_undegraded":"2024-03-28T16:49:37.608975+0000","last_fullsized":"2024-03-28T16:49:37.608975+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'34","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.293562+0000","last_change":"2024-03-28T16:49:00.725615+0000","last_active":"2024-03-28T16:50:42.293562+0000","last_peered":"2024-03-28T16:50:42.293562+0000","last_clean":"2024-03-28T16:50:42.293562+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:50:42.293562+0000","last_undegraded":"2024-03-28T16:50:42.293562+0000","last_fullsized":"2024-03-28T16:50:42.293562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582668+0000","last_change":"2024-03-28T16:49:02.739259+0000","last_active":"2024-03-28T16:49:37.582668+0000","last_peered":"2024-03-28T16:49:37.582668+0000","last_clean":"2024-03-28T16:49:37.582668+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:49:37.582668+0000","last_undegraded":"2024-03-28T16:49:37.582668+0000","last_fullsized":"2024-03-28T16:49:37.582668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'1","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.473838+0000","last_change":"2024-03-28T16:49:04.723410+0000","last_active":"2024-03-28T16:50:35.473838+0000","last_peered":"2024-03-28T16:50:35.473838+0000","last_clean":"2024-03-28T16:50:35.473838+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:50:35.473838+0000","last_undegraded":"2024-03-28T16:50:35.473838+0000","last_fullsized":"2024-03-28T16:50:35.473838+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.607248+0000","last_change":"2024-03-28T16:49:37.603811+0000","last_active":"2024-03-28T16:49:37.607248+0000","last_peered":"2024-03-28T16:49:37.607248+0000","last_clean":"2024-03-28T16:49:37.607248+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:49:37.607248+0000","last_undegraded":"2024-03-28T16:49:37.607248+0000","last_fullsized":"2024-03-28T16:49:37.607248+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.836685+0000","last_change":"2024-03-28T16:48:54.657586+0000","last_active":"2024-03-28T16:50:28.836685+0000","last_peered":"2024-03-28T16:50:28.836685+0000","last_clean":"2024-03-28T16:50:28.836685+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:50:28.836685+0000","last_undegraded":"2024-03-28T16:50:28.836685+0000","last_fullsized":"2024-03-28T16:50:28.836685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.178756+0000","last_change":"2024-03-28T16:50:52.178814+0000","last_active":"2024-03-28T16:50:52.178756+0000","last_peered":"2024-03-28T16:50:52.178756+0000","last_clean":"2024-03-28T16:50:52.178756+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:50:52.178756+0000","last_undegraded":"2024-03-28T16:50:52.178756+0000","last_fullsized":"2024-03-28T16:50:52.178756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.178561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:00.951220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"31'8","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.353761+0000","last_change":"2024-03-28T16:51:00.353835+0000","last_active":"2024-03-28T16:51:00.353761+0000","last_peered":"2024-03-28T16:51:00.353761+0000","last_clean":"2024-03-28T16:51:00.353761+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:51:00.353761+0000","last_undegraded":"2024-03-28T16:51:00.353761+0000","last_fullsized":"2024-03-28T16:51:00.353761+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.353557+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:18.400713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.443244+0000","last_change":"2024-03-28T16:50:49.443394+0000","last_active":"2024-03-28T16:50:49.443244+0000","last_peered":"2024-03-28T16:50:49.443244+0000","last_clean":"2024-03-28T16:50:49.443244+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:50:49.443244+0000","last_undegraded":"2024-03-28T16:50:49.443244+0000","last_fullsized":"2024-03-28T16:50:49.443244+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.443035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:33:21.896718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586273+0000","last_change":"2024-03-28T16:49:04.744040+0000","last_active":"2024-03-28T16:49:37.586273+0000","last_peered":"2024-03-28T16:49:37.586273+0000","last_clean":"2024-03-28T16:49:37.586273+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:49:37.586273+0000","last_undegraded":"2024-03-28T16:49:37.586273+0000","last_fullsized":"2024-03-28T16:49:37.586273+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.730889+0000","last_change":"2024-03-28T16:48:56.676841+0000","last_active":"2024-03-28T16:50:22.730889+0000","last_peered":"2024-03-28T16:50:22.730889+0000","last_clean":"2024-03-28T16:50:22.730889+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:50:22.730889+0000","last_undegraded":"2024-03-28T16:50:22.730889+0000","last_fullsized":"2024-03-28T16:50:22.730889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.186964+0000","last_change":"2024-03-28T16:48:54.659093+0000","last_active":"2024-03-28T16:50:23.186964+0000","last_peered":"2024-03-28T16:50:23.186964+0000","last_clean":"2024-03-28T16:50:23.186964+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:50:23.186964+0000","last_undegraded":"2024-03-28T16:50:23.186964+0000","last_fullsized":"2024-03-28T16:50:23.186964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'2","reported_seq":199,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.054717+0000","last_change":"2024-03-28T16:48:58.690200+0000","last_active":"2024-03-28T16:50:42.054717+0000","last_peered":"2024-03-28T16:50:42.054717+0000","last_clean":"2024-03-28T16:50:42.054717+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:50:42.054717+0000","last_undegraded":"2024-03-28T16:50:42.054717+0000","last_fullsized":"2024-03-28T16:50:42.054717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":256,"num_read_kb":255,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'42","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.233444+0000","last_change":"2024-03-28T16:50:55.233580+0000","last_active":"2024-03-28T16:50:55.233444+0000","last_peered":"2024-03-28T16:50:55.233444+0000","last_clean":"2024-03-28T16:50:55.233444+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:55.233444+0000","last_undegraded":"2024-03-28T16:50:55.233444+0000","last_fullsized":"2024-03-28T16:50:55.233444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'42","last_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_deep_scrub":"31'42","last_deep_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.233264+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:43.214573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":56,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.129888+0000","last_change":"2024-03-28T16:50:47.129943+0000","last_active":"2024-03-28T16:50:47.129888+0000","last_peered":"2024-03-28T16:50:47.129888+0000","last_clean":"2024-03-28T16:50:47.129888+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:50:47.129888+0000","last_undegraded":"2024-03-28T16:50:47.129888+0000","last_fullsized":"2024-03-28T16:50:47.129888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.129698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:49.507343+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582135+0000","last_change":"2024-03-28T16:49:04.752570+0000","last_active":"2024-03-28T16:49:37.582135+0000","last_peered":"2024-03-28T16:49:37.582135+0000","last_clean":"2024-03-28T16:49:37.582135+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:49:37.582135+0000","last_undegraded":"2024-03-28T16:49:37.582135+0000","last_fullsized":"2024-03-28T16:49:37.582135+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.203748+0000","last_change":"2024-03-28T16:48:56.683249+0000","last_active":"2024-03-28T16:50:23.203748+0000","last_peered":"2024-03-28T16:50:23.203748+0000","last_clean":"2024-03-28T16:50:23.203748+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:50:23.203748+0000","last_undegraded":"2024-03-28T16:50:23.203748+0000","last_fullsized":"2024-03-28T16:50:23.203748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.602191+0000","last_change":"2024-03-28T16:48:54.674928+0000","last_active":"2024-03-28T16:49:37.602191+0000","last_peered":"2024-03-28T16:49:37.602191+0000","last_clean":"2024-03-28T16:49:37.602191+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:49:37.602191+0000","last_undegraded":"2024-03-28T16:49:37.602191+0000","last_fullsized":"2024-03-28T16:49:37.602191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.166412+0000","last_change":"2024-03-28T16:50:58.166465+0000","last_active":"2024-03-28T16:50:58.166412+0000","last_peered":"2024-03-28T16:50:58.166412+0000","last_clean":"2024-03-28T16:50:58.166412+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:50:58.166412+0000","last_undegraded":"2024-03-28T16:50:58.166412+0000","last_fullsized":"2024-03-28T16:50:58.166412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.166221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:21.547629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'34","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.370627+0000","last_change":"2024-03-28T16:51:02.370703+0000","last_active":"2024-03-28T16:51:02.370627+0000","last_peered":"2024-03-28T16:51:02.370627+0000","last_clean":"2024-03-28T16:51:02.370627+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:51:02.370627+0000","last_undegraded":"2024-03-28T16:51:02.370627+0000","last_fullsized":"2024-03-28T16:51:02.370627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.370450+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:15.038783+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.448107+0000","last_change":"2024-03-28T16:50:50.448158+0000","last_active":"2024-03-28T16:50:50.448107+0000","last_peered":"2024-03-28T16:50:50.448107+0000","last_clean":"2024-03-28T16:50:50.448107+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:50:50.448107+0000","last_undegraded":"2024-03-28T16:50:50.448107+0000","last_fullsized":"2024-03-28T16:50:50.448107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.447911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:45.521899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872459+0000","last_change":"2024-03-28T16:48:56.675359+0000","last_active":"2024-03-28T16:50:46.872459+0000","last_peered":"2024-03-28T16:50:46.872459+0000","last_clean":"2024-03-28T16:50:46.872459+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:50:46.872459+0000","last_undegraded":"2024-03-28T16:50:46.872459+0000","last_fullsized":"2024-03-28T16:50:46.872459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:41.566962+0000","last_change":"2024-03-28T16:49:37.600567+0000","last_active":"2024-03-28T16:49:41.566962+0000","last_peered":"2024-03-28T16:49:41.566962+0000","last_clean":"2024-03-28T16:49:41.566962+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:49:41.566962+0000","last_undegraded":"2024-03-28T16:49:41.566962+0000","last_fullsized":"2024-03-28T16:49:41.566962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532495+0000","last_change":"2024-03-28T16:48:58.687213+0000","last_active":"2024-03-28T16:50:45.532495+0000","last_peered":"2024-03-28T16:50:45.532495+0000","last_clean":"2024-03-28T16:50:45.532495+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:50:45.532495+0000","last_undegraded":"2024-03-28T16:50:45.532495+0000","last_fullsized":"2024-03-28T16:50:45.532495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'60","reported_seq":132,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.252481+0000","last_change":"2024-03-28T16:50:56.252559+0000","last_active":"2024-03-28T16:50:56.252481+0000","last_peered":"2024-03-28T16:50:56.252481+0000","last_clean":"2024-03-28T16:50:56.252481+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:56.252481+0000","last_undegraded":"2024-03-28T16:50:56.252481+0000","last_fullsized":"2024-03-28T16:50:56.252481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'60","last_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_deep_scrub":"31'60","last_deep_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.252305+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:41.009336+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.444508+0000","last_change":"2024-03-28T16:50:48.444559+0000","last_active":"2024-03-28T16:50:48.444508+0000","last_peered":"2024-03-28T16:50:48.444508+0000","last_clean":"2024-03-28T16:50:48.444508+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:50:48.444508+0000","last_undegraded":"2024-03-28T16:50:48.444508+0000","last_fullsized":"2024-03-28T16:50:48.444508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.444297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:24:42.861356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'11","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.216050+0000","last_change":"2024-03-28T16:49:04.752525+0000","last_active":"2024-03-28T16:50:30.216050+0000","last_peered":"2024-03-28T16:50:30.216050+0000","last_clean":"2024-03-28T16:50:30.216050+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:50:30.216050+0000","last_undegraded":"2024-03-28T16:50:30.216050+0000","last_fullsized":"2024-03-28T16:50:30.216050+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532234+0000","last_change":"2024-03-28T16:48:56.697482+0000","last_active":"2024-03-28T16:50:45.532234+0000","last_peered":"2024-03-28T16:50:45.532234+0000","last_clean":"2024-03-28T16:50:45.532234+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:50:45.532234+0000","last_undegraded":"2024-03-28T16:50:45.532234+0000","last_fullsized":"2024-03-28T16:50:45.532234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584146+0000","last_change":"2024-03-28T16:48:54.668040+0000","last_active":"2024-03-28T16:49:37.584146+0000","last_peered":"2024-03-28T16:49:37.584146+0000","last_clean":"2024-03-28T16:49:37.584146+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:49:37.584146+0000","last_undegraded":"2024-03-28T16:49:37.584146+0000","last_fullsized":"2024-03-28T16:49:37.584146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.413459+0000","last_change":"2024-03-28T16:48:58.686377+0000","last_active":"2024-03-28T16:50:46.413459+0000","last_peered":"2024-03-28T16:50:46.413459+0000","last_clean":"2024-03-28T16:50:46.413459+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:50:46.413459+0000","last_undegraded":"2024-03-28T16:50:46.413459+0000","last_fullsized":"2024-03-28T16:50:46.413459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'43","reported_seq":100,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.182779+0000","last_change":"2024-03-28T16:50:57.182861+0000","last_active":"2024-03-28T16:50:57.182779+0000","last_peered":"2024-03-28T16:50:57.182779+0000","last_clean":"2024-03-28T16:50:57.182779+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:57.182779+0000","last_undegraded":"2024-03-28T16:50:57.182779+0000","last_fullsized":"2024-03-28T16:50:57.182779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.182585+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.262964+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.238718+0000","last_change":"2024-03-28T16:50:53.238776+0000","last_active":"2024-03-28T16:50:53.238718+0000","last_peered":"2024-03-28T16:50:53.238718+0000","last_clean":"2024-03-28T16:50:53.238718+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:50:53.238718+0000","last_undegraded":"2024-03-28T16:50:53.238718+0000","last_fullsized":"2024-03-28T16:50:53.238718+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.238528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:50.127619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.567266+0000","last_change":"2024-03-28T16:49:04.754981+0000","last_active":"2024-03-28T16:49:40.567266+0000","last_peered":"2024-03-28T16:49:40.567266+0000","last_clean":"2024-03-28T16:49:40.567266+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:49:40.567266+0000","last_undegraded":"2024-03-28T16:49:40.567266+0000","last_fullsized":"2024-03-28T16:49:40.567266+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'3","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.203425+0000","last_change":"2024-03-28T16:48:56.694984+0000","last_active":"2024-03-28T16:50:23.203425+0000","last_peered":"2024-03-28T16:50:23.203425+0000","last_clean":"2024-03-28T16:50:23.203425+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:50:23.203425+0000","last_undegraded":"2024-03-28T16:50:23.203425+0000","last_fullsized":"2024-03-28T16:50:23.203425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586092+0000","last_change":"2024-03-28T16:48:54.654527+0000","last_active":"2024-03-28T16:49:37.586092+0000","last_peered":"2024-03-28T16:49:37.586092+0000","last_clean":"2024-03-28T16:49:37.586092+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:49:37.586092+0000","last_undegraded":"2024-03-28T16:49:37.586092+0000","last_fullsized":"2024-03-28T16:49:37.586092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.612252+0000","last_change":"2024-03-28T16:49:37.610755+0000","last_active":"2024-03-28T16:49:37.612252+0000","last_peered":"2024-03-28T16:49:37.612252+0000","last_clean":"2024-03-28T16:49:37.612252+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:49:37.612252+0000","last_undegraded":"2024-03-28T16:49:37.612252+0000","last_fullsized":"2024-03-28T16:49:37.612252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'39","reported_seq":94,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.341978+0000","last_change":"2024-03-28T16:50:58.342055+0000","last_active":"2024-03-28T16:50:58.341978+0000","last_peered":"2024-03-28T16:50:58.341978+0000","last_clean":"2024-03-28T16:50:58.341978+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:58.341978+0000","last_undegraded":"2024-03-28T16:50:58.341978+0000","last_fullsized":"2024-03-28T16:50:58.341978+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.341815+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:35.862752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587559+0000","last_change":"2024-03-28T16:49:02.734090+0000","last_active":"2024-03-28T16:49:37.587559+0000","last_peered":"2024-03-28T16:49:37.587559+0000","last_clean":"2024-03-28T16:49:37.587559+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:49:37.587559+0000","last_undegraded":"2024-03-28T16:49:37.587559+0000","last_fullsized":"2024-03-28T16:49:37.587559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.874641+0000","last_change":"2024-03-28T16:49:04.748086+0000","last_active":"2024-03-28T16:49:39.874641+0000","last_peered":"2024-03-28T16:49:39.874641+0000","last_clean":"2024-03-28T16:49:39.874641+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:49:39.874641+0000","last_undegraded":"2024-03-28T16:49:39.874641+0000","last_fullsized":"2024-03-28T16:49:39.874641+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'3","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469109+0000","last_change":"2024-03-28T16:48:56.682850+0000","last_active":"2024-03-28T16:50:35.469109+0000","last_peered":"2024-03-28T16:50:35.469109+0000","last_clean":"2024-03-28T16:50:35.469109+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:50:35.469109+0000","last_undegraded":"2024-03-28T16:50:35.469109+0000","last_fullsized":"2024-03-28T16:50:35.469109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592742+0000","last_change":"2024-03-28T16:48:54.674069+0000","last_active":"2024-03-28T16:49:37.592742+0000","last_peered":"2024-03-28T16:49:37.592742+0000","last_clean":"2024-03-28T16:49:37.592742+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:49:37.592742+0000","last_undegraded":"2024-03-28T16:49:37.592742+0000","last_fullsized":"2024-03-28T16:49:37.592742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.303927+0000","last_change":"2024-03-28T16:51:04.303980+0000","last_active":"2024-03-28T16:51:04.303927+0000","last_peered":"2024-03-28T16:51:04.303927+0000","last_clean":"2024-03-28T16:51:04.303927+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:51:04.303927+0000","last_undegraded":"2024-03-28T16:51:04.303927+0000","last_fullsized":"2024-03-28T16:51:04.303927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.303737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:22.080932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.229100+0000","last_change":"2024-03-28T16:51:01.229100+0000","last_active":"2024-03-28T16:51:01.229100+0000","last_peered":"2024-03-28T16:51:01.229100+0000","last_clean":"2024-03-28T16:51:01.229100+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:51:01.229100+0000","last_undegraded":"2024-03-28T16:51:01.229100+0000","last_fullsized":"2024-03-28T16:51:01.229100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.228821+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:13.034030+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'249","reported_seq":309,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.440306+0000","last_change":"2024-03-28T16:50:52.440393+0000","last_active":"2024-03-28T16:50:52.440306+0000","last_peered":"2024-03-28T16:50:52.440306+0000","last_clean":"2024-03-28T16:50:52.440306+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:52.440306+0000","last_undegraded":"2024-03-28T16:50:52.440306+0000","last_fullsized":"2024-03-28T16:50:52.440306+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.440130+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:54:49.215852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580073+0000","last_change":"2024-03-28T16:49:04.748224+0000","last_active":"2024-03-28T16:49:37.580073+0000","last_peered":"2024-03-28T16:49:37.580073+0000","last_clean":"2024-03-28T16:49:37.580073+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:49:37.580073+0000","last_undegraded":"2024-03-28T16:49:37.580073+0000","last_fullsized":"2024-03-28T16:49:37.580073+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'5","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729798+0000","last_change":"2024-03-28T16:48:56.685308+0000","last_active":"2024-03-28T16:50:22.729798+0000","last_peered":"2024-03-28T16:50:22.729798+0000","last_clean":"2024-03-28T16:50:22.729798+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:50:22.729798+0000","last_undegraded":"2024-03-28T16:50:22.729798+0000","last_fullsized":"2024-03-28T16:50:22.729798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586470+0000","last_change":"2024-03-28T16:48:54.667648+0000","last_active":"2024-03-28T16:49:37.586470+0000","last_peered":"2024-03-28T16:49:37.586470+0000","last_clean":"2024-03-28T16:49:37.586470+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:49:37.586470+0000","last_undegraded":"2024-03-28T16:49:37.586470+0000","last_fullsized":"2024-03-28T16:49:37.586470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.331599+0000","last_change":"2024-03-28T16:51:02.331654+0000","last_active":"2024-03-28T16:51:02.331599+0000","last_peered":"2024-03-28T16:51:02.331599+0000","last_clean":"2024-03-28T16:51:02.331599+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:51:02.331599+0000","last_undegraded":"2024-03-28T16:51:02.331599+0000","last_fullsized":"2024-03-28T16:51:02.331599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.331416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:20.378572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'43","reported_seq":100,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.202301+0000","last_change":"2024-03-28T16:50:50.202390+0000","last_active":"2024-03-28T16:50:50.202301+0000","last_peered":"2024-03-28T16:50:50.202301+0000","last_clean":"2024-03-28T16:50:50.202301+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:50.202301+0000","last_undegraded":"2024-03-28T16:50:50.202301+0000","last_fullsized":"2024-03-28T16:50:50.202301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.202141+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.086676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.276885+0000","last_change":"2024-03-28T16:50:48.276938+0000","last_active":"2024-03-28T16:50:48.276885+0000","last_peered":"2024-03-28T16:50:48.276885+0000","last_clean":"2024-03-28T16:50:48.276885+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:50:48.276885+0000","last_undegraded":"2024-03-28T16:50:48.276885+0000","last_fullsized":"2024-03-28T16:50:48.276885+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.276698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:29.219774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601299+0000","last_change":"2024-03-28T16:49:04.745332+0000","last_active":"2024-03-28T16:49:37.601299+0000","last_peered":"2024-03-28T16:49:37.601299+0000","last_clean":"2024-03-28T16:49:37.601299+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:49:37.601299+0000","last_undegraded":"2024-03-28T16:49:37.601299+0000","last_fullsized":"2024-03-28T16:49:37.601299+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'3","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729795+0000","last_change":"2024-03-28T16:48:56.690217+0000","last_active":"2024-03-28T16:50:22.729795+0000","last_peered":"2024-03-28T16:50:22.729795+0000","last_clean":"2024-03-28T16:50:22.729795+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:50:22.729795+0000","last_undegraded":"2024-03-28T16:50:22.729795+0000","last_fullsized":"2024-03-28T16:50:22.729795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588778+0000","last_change":"2024-03-28T16:48:54.669871+0000","last_active":"2024-03-28T16:49:37.588778+0000","last_peered":"2024-03-28T16:49:37.588778+0000","last_clean":"2024-03-28T16:49:37.588778+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:49:37.588778+0000","last_undegraded":"2024-03-28T16:49:37.588778+0000","last_fullsized":"2024-03-28T16:49:37.588778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.396440+0000","last_change":"2024-03-28T16:51:03.396495+0000","last_active":"2024-03-28T16:51:03.396440+0000","last_peered":"2024-03-28T16:51:03.396440+0000","last_clean":"2024-03-28T16:51:03.396440+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:51:03.396440+0000","last_undegraded":"2024-03-28T16:51:03.396440+0000","last_fullsized":"2024-03-28T16:51:03.396440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.396253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:26.295446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'34","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.307684+0000","last_change":"2024-03-28T16:50:59.307759+0000","last_active":"2024-03-28T16:50:59.307684+0000","last_peered":"2024-03-28T16:50:59.307684+0000","last_clean":"2024-03-28T16:50:59.307684+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:59.307684+0000","last_undegraded":"2024-03-28T16:50:59.307684+0000","last_fullsized":"2024-03-28T16:50:59.307684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.307494+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:28.146893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581179+0000","last_change":"2024-03-28T16:49:02.739372+0000","last_active":"2024-03-28T16:49:37.581179+0000","last_peered":"2024-03-28T16:49:37.581179+0000","last_clean":"2024-03-28T16:49:37.581179+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:49:37.581179+0000","last_undegraded":"2024-03-28T16:49:37.581179+0000","last_fullsized":"2024-03-28T16:49:37.581179+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582903+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:49:37.582903+0000","last_peered":"2024-03-28T16:49:37.582903+0000","last_clean":"2024-03-28T16:49:37.582903+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:49:37.582903+0000","last_undegraded":"2024-03-28T16:49:37.582903+0000","last_fullsized":"2024-03-28T16:49:37.582903+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'1","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.467847+0000","last_change":"2024-03-28T16:48:56.666104+0000","last_active":"2024-03-28T16:50:35.467847+0000","last_peered":"2024-03-28T16:50:35.467847+0000","last_clean":"2024-03-28T16:50:35.467847+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:50:35.467847+0000","last_undegraded":"2024-03-28T16:50:35.467847+0000","last_fullsized":"2024-03-28T16:50:35.467847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531809+0000","last_change":"2024-03-28T16:48:54.665975+0000","last_active":"2024-03-28T16:50:45.531809+0000","last_peered":"2024-03-28T16:50:45.531809+0000","last_clean":"2024-03-28T16:50:45.531809+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:50:45.531809+0000","last_undegraded":"2024-03-28T16:50:45.531809+0000","last_fullsized":"2024-03-28T16:50:45.531809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.256547+0000","last_change":"2024-03-28T16:50:59.256599+0000","last_active":"2024-03-28T16:50:59.256547+0000","last_peered":"2024-03-28T16:50:59.256547+0000","last_clean":"2024-03-28T16:50:59.256547+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:50:59.256547+0000","last_undegraded":"2024-03-28T16:50:59.256547+0000","last_fullsized":"2024-03-28T16:50:59.256547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.256359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:34.399651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.213370+0000","last_change":"2024-03-28T16:50:59.213439+0000","last_active":"2024-03-28T16:50:59.213370+0000","last_peered":"2024-03-28T16:50:59.213370+0000","last_clean":"2024-03-28T16:50:59.213370+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:59.213370+0000","last_undegraded":"2024-03-28T16:50:59.213370+0000","last_fullsized":"2024-03-28T16:50:59.213370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'28","last_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_deep_scrub":"31'28","last_deep_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.213187+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:38.889215+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581949+0000","last_change":"2024-03-28T16:49:02.739568+0000","last_active":"2024-03-28T16:49:37.581949+0000","last_peered":"2024-03-28T16:49:37.581949+0000","last_clean":"2024-03-28T16:49:37.581949+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:49:37.581949+0000","last_undegraded":"2024-03-28T16:49:37.581949+0000","last_fullsized":"2024-03-28T16:49:37.581949+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587298+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:49:37.587298+0000","last_peered":"2024-03-28T16:49:37.587298+0000","last_clean":"2024-03-28T16:49:37.587298+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:49:37.587298+0000","last_undegraded":"2024-03-28T16:49:37.587298+0000","last_fullsized":"2024-03-28T16:49:37.587298+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'11","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469404+0000","last_change":"2024-03-28T16:48:56.678271+0000","last_active":"2024-03-28T16:50:35.469404+0000","last_peered":"2024-03-28T16:50:35.469404+0000","last_clean":"2024-03-28T16:50:35.469404+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:50:35.469404+0000","last_undegraded":"2024-03-28T16:50:35.469404+0000","last_fullsized":"2024-03-28T16:50:35.469404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984783+0000","last_change":"2024-03-28T16:48:54.655009+0000","last_active":"2024-03-28T16:50:45.984783+0000","last_peered":"2024-03-28T16:50:45.984783+0000","last_clean":"2024-03-28T16:50:45.984783+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:50:45.984783+0000","last_undegraded":"2024-03-28T16:50:45.984783+0000","last_fullsized":"2024-03-28T16:50:45.984783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'3","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.215833+0000","last_change":"2024-03-28T16:50:58.215895+0000","last_active":"2024-03-28T16:50:58.215833+0000","last_peered":"2024-03-28T16:50:58.215833+0000","last_clean":"2024-03-28T16:50:58.215833+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:50:58.215833+0000","last_undegraded":"2024-03-28T16:50:58.215833+0000","last_fullsized":"2024-03-28T16:50:58.215833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.215651+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:42.972682+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'32","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:42.329497+0000","last_change":"2024-03-28T16:49:00.720335+0000","last_active":"2024-03-28T16:50:42.329497+0000","last_peered":"2024-03-28T16:50:42.329497+0000","last_clean":"2024-03-28T16:50:42.329497+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:50:42.329497+0000","last_undegraded":"2024-03-28T16:50:42.329497+0000","last_fullsized":"2024-03-28T16:50:42.329497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:59.663171+0000","last_clean_scrub_stamp":"2024-03-28T16:48:59.663171+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.209217+0000","last_change":"2024-03-28T16:50:50.209342+0000","last_active":"2024-03-28T16:50:50.209217+0000","last_peered":"2024-03-28T16:50:50.209217+0000","last_clean":"2024-03-28T16:50:50.209217+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:50:50.209217+0000","last_undegraded":"2024-03-28T16:50:50.209217+0000","last_fullsized":"2024-03-28T16:50:50.209217+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.209010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:45.672112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582486+0000","last_change":"2024-03-28T16:49:04.753745+0000","last_active":"2024-03-28T16:49:37.582486+0000","last_peered":"2024-03-28T16:49:37.582486+0000","last_clean":"2024-03-28T16:49:37.582486+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:49:37.582486+0000","last_undegraded":"2024-03-28T16:49:37.582486+0000","last_fullsized":"2024-03-28T16:49:37.582486+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.575349+0000","last_change":"2024-03-28T16:49:37.618664+0000","last_active":"2024-03-28T16:49:40.575349+0000","last_peered":"2024-03-28T16:49:40.575349+0000","last_clean":"2024-03-28T16:49:40.575349+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:49:40.575349+0000","last_undegraded":"2024-03-28T16:49:40.575349+0000","last_fullsized":"2024-03-28T16:49:40.575349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412173+0000","last_change":"2024-03-28T16:48:54.658051+0000","last_active":"2024-03-28T16:50:46.412173+0000","last_peered":"2024-03-28T16:50:46.412173+0000","last_clean":"2024-03-28T16:50:46.412173+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:50:46.412173+0000","last_undegraded":"2024-03-28T16:50:46.412173+0000","last_fullsized":"2024-03-28T16:50:46.412173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.203345+0000","last_change":"2024-03-28T16:48:56.682367+0000","last_active":"2024-03-28T16:50:23.203345+0000","last_peered":"2024-03-28T16:50:23.203345+0000","last_clean":"2024-03-28T16:50:23.203345+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:50:23.203345+0000","last_undegraded":"2024-03-28T16:50:23.203345+0000","last_fullsized":"2024-03-28T16:50:23.203345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532018+0000","last_change":"2024-03-28T16:48:54.655298+0000","last_active":"2024-03-28T16:50:45.532018+0000","last_peered":"2024-03-28T16:50:45.532018+0000","last_clean":"2024-03-28T16:50:45.532018+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:50:45.532018+0000","last_undegraded":"2024-03-28T16:50:45.532018+0000","last_fullsized":"2024-03-28T16:50:45.532018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'3","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729872+0000","last_change":"2024-03-28T16:48:56.671798+0000","last_active":"2024-03-28T16:50:22.729872+0000","last_peered":"2024-03-28T16:50:22.729872+0000","last_clean":"2024-03-28T16:50:22.729872+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:50:22.729872+0000","last_undegraded":"2024-03-28T16:50:22.729872+0000","last_fullsized":"2024-03-28T16:50:22.729872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871513+0000","last_change":"2024-03-28T16:48:54.657491+0000","last_active":"2024-03-28T16:50:46.871513+0000","last_peered":"2024-03-28T16:50:46.871513+0000","last_clean":"2024-03-28T16:50:46.871513+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:50:46.871513+0000","last_undegraded":"2024-03-28T16:50:46.871513+0000","last_fullsized":"2024-03-28T16:50:46.871513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586940+0000","last_change":"2024-03-28T16:48:56.683483+0000","last_active":"2024-03-28T16:49:37.586940+0000","last_peered":"2024-03-28T16:49:37.586940+0000","last_clean":"2024-03-28T16:49:37.586940+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:49:37.586940+0000","last_undegraded":"2024-03-28T16:49:37.586940+0000","last_fullsized":"2024-03-28T16:49:37.586940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531959+0000","last_change":"2024-03-28T16:48:54.665811+0000","last_active":"2024-03-28T16:50:45.531959+0000","last_peered":"2024-03-28T16:50:45.531959+0000","last_clean":"2024-03-28T16:50:45.531959+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:50:45.531959+0000","last_undegraded":"2024-03-28T16:50:45.531959+0000","last_fullsized":"2024-03-28T16:50:45.531959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532474+0000","last_change":"2024-03-28T16:48:56.677917+0000","last_active":"2024-03-28T16:50:45.532474+0000","last_peered":"2024-03-28T16:50:45.532474+0000","last_clean":"2024-03-28T16:50:45.532474+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:50:45.532474+0000","last_undegraded":"2024-03-28T16:50:45.532474+0000","last_fullsized":"2024-03-28T16:50:45.532474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587372+0000","last_change":"2024-03-28T16:48:54.655391+0000","last_active":"2024-03-28T16:49:37.587372+0000","last_peered":"2024-03-28T16:49:37.587372+0000","last_clean":"2024-03-28T16:49:37.587372+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:49:37.587372+0000","last_undegraded":"2024-03-28T16:49:37.587372+0000","last_fullsized":"2024-03-28T16:49:37.587372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985530+0000","last_change":"2024-03-28T16:48:56.678419+0000","last_active":"2024-03-28T16:50:45.985530+0000","last_peered":"2024-03-28T16:50:45.985530+0000","last_clean":"2024-03-28T16:50:45.985530+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:50:45.985530+0000","last_undegraded":"2024-03-28T16:50:45.985530+0000","last_fullsized":"2024-03-28T16:50:45.985530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984798+0000","last_change":"2024-03-28T16:48:54.669640+0000","last_active":"2024-03-28T16:50:45.984798+0000","last_peered":"2024-03-28T16:50:45.984798+0000","last_clean":"2024-03-28T16:50:45.984798+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:50:45.984798+0000","last_undegraded":"2024-03-28T16:50:45.984798+0000","last_fullsized":"2024-03-28T16:50:45.984798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532278+0000","last_change":"2024-03-28T16:48:56.689880+0000","last_active":"2024-03-28T16:50:45.532278+0000","last_peered":"2024-03-28T16:50:45.532278+0000","last_clean":"2024-03-28T16:50:45.532278+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:50:45.532278+0000","last_undegraded":"2024-03-28T16:50:45.532278+0000","last_fullsized":"2024-03-28T16:50:45.532278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412210+0000","last_change":"2024-03-28T16:48:54.661594+0000","last_active":"2024-03-28T16:50:46.412210+0000","last_peered":"2024-03-28T16:50:46.412210+0000","last_clean":"2024-03-28T16:50:46.412210+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:50:46.412210+0000","last_undegraded":"2024-03-28T16:50:46.412210+0000","last_fullsized":"2024-03-28T16:50:46.412210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'5","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985537+0000","last_change":"2024-03-28T16:48:56.679523+0000","last_active":"2024-03-28T16:50:45.985537+0000","last_peered":"2024-03-28T16:50:45.985537+0000","last_clean":"2024-03-28T16:50:45.985537+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:50:45.985537+0000","last_undegraded":"2024-03-28T16:50:45.985537+0000","last_fullsized":"2024-03-28T16:50:45.985537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588596+0000","last_change":"2024-03-28T16:48:54.659870+0000","last_active":"2024-03-28T16:49:37.588596+0000","last_peered":"2024-03-28T16:49:37.588596+0000","last_clean":"2024-03-28T16:49:37.588596+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:49:37.588596+0000","last_undegraded":"2024-03-28T16:49:37.588596+0000","last_fullsized":"2024-03-28T16:49:37.588596+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.537599+0000","last_change":"2024-03-28T16:48:56.687788+0000","last_active":"2024-03-28T16:49:39.537599+0000","last_peered":"2024-03-28T16:49:39.537599+0000","last_clean":"2024-03-28T16:49:39.537599+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:49:39.537599+0000","last_undegraded":"2024-03-28T16:49:39.537599+0000","last_fullsized":"2024-03-28T16:49:39.537599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:38.398340+0000","last_change":"2024-03-28T16:48:54.657662+0000","last_active":"2024-03-28T16:49:38.398340+0000","last_peered":"2024-03-28T16:49:38.398340+0000","last_clean":"2024-03-28T16:49:38.398340+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:49:38.398340+0000","last_undegraded":"2024-03-28T16:49:38.398340+0000","last_fullsized":"2024-03-28T16:49:38.398340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'1","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.729785+0000","last_change":"2024-03-28T16:48:56.682872+0000","last_active":"2024-03-28T16:50:22.729785+0000","last_peered":"2024-03-28T16:50:22.729785+0000","last_clean":"2024-03-28T16:50:22.729785+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:50:22.729785+0000","last_undegraded":"2024-03-28T16:50:22.729785+0000","last_fullsized":"2024-03-28T16:50:22.729785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584735+0000","last_change":"2024-03-28T16:48:54.663496+0000","last_active":"2024-03-28T16:49:37.584735+0000","last_peered":"2024-03-28T16:49:37.584735+0000","last_clean":"2024-03-28T16:49:37.584735+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:49:37.584735+0000","last_undegraded":"2024-03-28T16:49:37.584735+0000","last_fullsized":"2024-03-28T16:49:37.584735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'1","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.468994+0000","last_change":"2024-03-28T16:48:56.678342+0000","last_active":"2024-03-28T16:50:35.468994+0000","last_peered":"2024-03-28T16:50:35.468994+0000","last_clean":"2024-03-28T16:50:35.468994+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:50:35.468994+0000","last_undegraded":"2024-03-28T16:50:35.468994+0000","last_fullsized":"2024-03-28T16:50:35.468994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585797+0000","last_change":"2024-03-28T16:48:54.675150+0000","last_active":"2024-03-28T16:49:37.585797+0000","last_peered":"2024-03-28T16:49:37.585797+0000","last_clean":"2024-03-28T16:49:37.585797+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:49:37.585797+0000","last_undegraded":"2024-03-28T16:49:37.585797+0000","last_fullsized":"2024-03-28T16:49:37.585797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:23.203891+0000","last_change":"2024-03-28T16:48:56.671157+0000","last_active":"2024-03-28T16:50:23.203891+0000","last_peered":"2024-03-28T16:50:23.203891+0000","last_clean":"2024-03-28T16:50:23.203891+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:50:23.203891+0000","last_undegraded":"2024-03-28T16:50:23.203891+0000","last_fullsized":"2024-03-28T16:50:23.203891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585905+0000","last_change":"2024-03-28T16:48:54.674973+0000","last_active":"2024-03-28T16:49:37.585905+0000","last_peered":"2024-03-28T16:49:37.585905+0000","last_clean":"2024-03-28T16:49:37.585905+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:49:37.585905+0000","last_undegraded":"2024-03-28T16:49:37.585905+0000","last_fullsized":"2024-03-28T16:49:37.585905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.854475+0000","last_change":"2024-03-28T16:48:56.683829+0000","last_active":"2024-03-28T16:50:28.854475+0000","last_peered":"2024-03-28T16:50:28.854475+0000","last_clean":"2024-03-28T16:50:28.854475+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:50:28.854475+0000","last_undegraded":"2024-03-28T16:50:28.854475+0000","last_fullsized":"2024-03-28T16:50:28.854475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587881+0000","last_change":"2024-03-28T16:48:54.676815+0000","last_active":"2024-03-28T16:49:37.587881+0000","last_peered":"2024-03-28T16:49:37.587881+0000","last_clean":"2024-03-28T16:49:37.587881+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:49:37.587881+0000","last_undegraded":"2024-03-28T16:49:37.587881+0000","last_fullsized":"2024-03-28T16:49:37.587881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.853813+0000","last_change":"2024-03-28T16:48:56.693060+0000","last_active":"2024-03-28T16:50:28.853813+0000","last_peered":"2024-03-28T16:50:28.853813+0000","last_clean":"2024-03-28T16:50:28.853813+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:50:28.853813+0000","last_undegraded":"2024-03-28T16:50:28.853813+0000","last_fullsized":"2024-03-28T16:50:28.853813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.589304+0000","last_change":"2024-03-28T16:48:54.663003+0000","last_active":"2024-03-28T16:49:37.589304+0000","last_peered":"2024-03-28T16:49:37.589304+0000","last_clean":"2024-03-28T16:49:37.589304+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:49:37.589304+0000","last_undegraded":"2024-03-28T16:49:37.589304+0000","last_fullsized":"2024-03-28T16:49:37.589304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.090733+0000","last_change":"2024-03-28T16:48:56.685753+0000","last_active":"2024-03-28T16:49:39.090733+0000","last_peered":"2024-03-28T16:49:39.090733+0000","last_clean":"2024-03-28T16:49:39.090733+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:49:39.090733+0000","last_undegraded":"2024-03-28T16:49:39.090733+0000","last_fullsized":"2024-03-28T16:49:39.090733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.598266+0000","last_change":"2024-03-28T16:48:54.661896+0000","last_active":"2024-03-28T16:49:37.598266+0000","last_peered":"2024-03-28T16:49:37.598266+0000","last_clean":"2024-03-28T16:49:37.598266+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:49:37.598266+0000","last_undegraded":"2024-03-28T16:49:37.598266+0000","last_fullsized":"2024-03-28T16:49:37.598266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'9","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.468611+0000","last_change":"2024-03-28T16:48:56.680119+0000","last_active":"2024-03-28T16:50:35.468611+0000","last_peered":"2024-03-28T16:50:35.468611+0000","last_clean":"2024-03-28T16:50:35.468611+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:50:35.468611+0000","last_undegraded":"2024-03-28T16:50:35.468611+0000","last_fullsized":"2024-03-28T16:50:35.468611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588779+0000","last_change":"2024-03-28T16:48:54.665812+0000","last_active":"2024-03-28T16:49:37.588779+0000","last_peered":"2024-03-28T16:49:37.588779+0000","last_clean":"2024-03-28T16:49:37.588779+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:49:37.588779+0000","last_undegraded":"2024-03-28T16:49:37.588779+0000","last_fullsized":"2024-03-28T16:49:37.588779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.581682+0000","last_change":"2024-03-28T16:48:54.665260+0000","last_active":"2024-03-28T16:49:37.581682+0000","last_peered":"2024-03-28T16:49:37.581682+0000","last_clean":"2024-03-28T16:49:37.581682+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:49:37.581682+0000","last_undegraded":"2024-03-28T16:49:37.581682+0000","last_fullsized":"2024-03-28T16:49:37.581682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'3","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469849+0000","last_change":"2024-03-28T16:48:56.690793+0000","last_active":"2024-03-28T16:50:35.469849+0000","last_peered":"2024-03-28T16:50:35.469849+0000","last_clean":"2024-03-28T16:50:35.469849+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:50:35.469849+0000","last_undegraded":"2024-03-28T16:50:35.469849+0000","last_fullsized":"2024-03-28T16:50:35.469849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584469+0000","last_change":"2024-03-28T16:48:54.663735+0000","last_active":"2024-03-28T16:49:37.584469+0000","last_peered":"2024-03-28T16:49:37.584469+0000","last_clean":"2024-03-28T16:49:37.584469+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:49:37.584469+0000","last_undegraded":"2024-03-28T16:49:37.584469+0000","last_fullsized":"2024-03-28T16:49:37.584469+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'1","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469842+0000","last_change":"2024-03-28T16:48:56.683832+0000","last_active":"2024-03-28T16:50:35.469842+0000","last_peered":"2024-03-28T16:50:35.469842+0000","last_clean":"2024-03-28T16:50:35.469842+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:50:35.469842+0000","last_undegraded":"2024-03-28T16:50:35.469842+0000","last_fullsized":"2024-03-28T16:50:35.469842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1992,"ondisk_log_size":1992,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1539,"num_read_kb":1325,"num_write":581,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":600,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1386,"ondisk_log_size":1386,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":866,"num_read_kb":864,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705695,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39916,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331924,"statfs":{"total":96636764160,"available":96595890176,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":1.246,"15min":1.246},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81899999999999995,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.69399999999999995}]},{"osd":1,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.26700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.79000000000000004,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.76600000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77400000000000002,"5min":1.671,"15min":1.671},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76400000000000001,"5min":1.331,"15min":1.331},"last":0.54900000000000004}]}]},{"osd":2,"up_from":9,"seq":38654705696,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.82999999999999996}]},{"osd":1,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.26900000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.56100000000000005,"15min":0.54000000000000004},"min":{"1min":0.28599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.95899999999999996}]},{"osd":3,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.30399999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.32200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.78600000000000003}]}]},{"osd":1,"up_from":9,"seq":38654705696,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.45300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79600000000000004,"5min":0.878,"15min":0.878},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69399999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.68799999999999994}]},{"osd":3,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.83799999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.27100000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.86299999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72799999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705695,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.40600000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90000000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90800000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.117},"last":0.747}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:51:07.688 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.688 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.688 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.688 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.688 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.688 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.689 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.689 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.689 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.689 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.689 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.689 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.689 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.690 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.690 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.690 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.690 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.690 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.690 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.691 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.691 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.691 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.691 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.691 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.691 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.692 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.692 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.692 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.692 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.692 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.692 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.692 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.693 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.693 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.693 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.693 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.693 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.693 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.694 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.694 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.694 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.694 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.694 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.694 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.695 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.695 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.695 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.695 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.695 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.695 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.696 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.696 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.696 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.696 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.696 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.696 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.697 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.697 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.697 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.697 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.697 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.697 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.698 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.698 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.698 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.698 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.698 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.699 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.699 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.699 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.699 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.699 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.699 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.700 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.700 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.700 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-28T16:48:42.575180+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=42, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.700 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.700 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.700 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.701 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.701 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.701 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.701 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.701 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.701 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.701 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.702 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.702 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.702 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.702 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.702 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.703 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.703 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.703 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.703 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.703 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.703 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.704 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.704 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.704 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.704 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.704 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.704 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.704 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.705 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.705 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.705 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.705 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.705 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.705 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.706 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.706 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.706 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.706 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.706 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.707 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.707 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.707 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.707 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.707 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.707 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.707 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.708 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.708 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.708 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.708 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.708 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.708 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.709 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.709 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.709 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.709 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.709 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.709 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.709 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.710 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.710 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.710 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.710 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.710 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.711 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.711 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.711 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.711 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.711 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.712 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.712 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.712 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.712 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.712 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.712 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.713 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.713 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.713 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.713 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.713 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.713 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.714 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-03-28T16:48:59.663171+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=59, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.714 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.714 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.714 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.714 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.714 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.715 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.715 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.715 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.715 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.715 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.716 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.716 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.716 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.716 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.716 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.716 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.717 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.717 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.717 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.717 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.717 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.717 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.717 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.718 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.718 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.718 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.718 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.718 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.718 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.719 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.719 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.719 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.719 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.719 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.719 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.720 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.720 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:07.720 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-28T16:51:27.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:51:27.851 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f10a10af640 1 Processor -- start 2024-03-28T16:51:27.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f10a10af640 1 -- start start 2024-03-28T16:51:27.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f10a10af640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f109c098830 0x7f109c098c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:27.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f10a10af640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c09a490 con 0x7f109c099150 2024-03-28T16:51:27.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f10a10af640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c072800 con 0x7f109c098830 2024-03-28T16:51:27.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f109ad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f109c098830 0x7f109c098c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:27.852 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f109a59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f109c099150 0x7f109c096490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40362/0 (socket says 172.21.15.119:40362) 2024-03-28T16:51:27.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.849+0000 7f109a59c640 1 -- 172.21.15.119:0/978331678 learned_addr learned my addr 172.21.15.119:0/978331678 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:51:27.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f109ad9d640 1 -- 172.21.15.119:0/978331678 >> v1:172.21.15.119:6789/0 conn(0x7f109c099150 legacy=0x7f109c096490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:27.853 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f109ad9d640 1 -- 172.21.15.119:0/978331678 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c09d790 con 0x7f109c098830 2024-03-28T16:51:27.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f109ad9d640 1 --2- 172.21.15.119:0/978331678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c098c10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1084009740 tx=0x7f108401c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b743fda3c4750695 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:27.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f1099d9b640 1 -- 172.21.15.119:0/978331678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10840ab530 con 0x7f109c098830 2024-03-28T16:51:27.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f1099d9b640 1 -- 172.21.15.119:0/978331678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10840a7050 con 0x7f109c098830 2024-03-28T16:51:27.854 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f1099d9b640 1 -- 172.21.15.119:0/978331678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10840ab7e0 con 0x7f109c098830 2024-03-28T16:51:27.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f10a10af640 1 -- 172.21.15.119:0/978331678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 msgr2=0x7f109c098c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:27.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f10a10af640 1 --2- 172.21.15.119:0/978331678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c098c10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1084009740 tx=0x7f108401c370 comp rx=0 tx=0).stop 2024-03-28T16:51:27.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f10a10af640 1 -- 172.21.15.119:0/978331678 shutdown_connections 2024-03-28T16:51:27.856 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f10a10af640 1 --2- 172.21.15.119:0/978331678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c098c10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f10a10af640 1 -- 172.21.15.119:0/978331678 >> 172.21.15.119:0/978331678 conn(0x7f109c0a0920 msgr2=0x7f109c09ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f10a10af640 1 -- 172.21.15.119:0/978331678 shutdown_connections 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.853+0000 7f10a10af640 1 -- 172.21.15.119:0/978331678 wait complete. 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10a10af640 1 Processor -- start 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10a10af640 1 -- start start 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10a10af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c1c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10a10af640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c131f20 con 0x7f109c098830 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f109ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c1c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:27.857 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f109ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c1c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58090/0 (socket says 172.21.15.119:58090) 2024-03-28T16:51:27.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f109ad9d640 1 -- 172.21.15.119:0/2604348512 learned_addr learned my addr 172.21.15.119:0/2604348512 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:51:27.858 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f109ad9d640 1 -- 172.21.15.119:0/2604348512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1084009420 con 0x7f109c098830 2024-03-28T16:51:27.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f109ad9d640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c1c5750 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1084019fa0 tx=0x7f108401abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:27.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10937fe640 1 -- 172.21.15.119:0/2604348512 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10840b7450 con 0x7f109c098830 2024-03-28T16:51:27.859 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10937fe640 1 -- 172.21.15.119:0/2604348512 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10840b75b0 con 0x7f109c098830 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c131610 con 0x7f109c098830 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10937fe640 1 -- 172.21.15.119:0/2604348512 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10840b7850 con 0x7f109c098830 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c131b50 con 0x7f109c098830 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c133100 con 0x7f109c098830 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10937fe640 1 -- 172.21.15.119:0/2604348512 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f10840b79b0 con 0x7f109c098830 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10937fe640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f107c035a10 0x7f107c037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f10937fe640 1 -- 172.21.15.119:0/2604348512 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f10840e33f0 con 0x7f109c098830 2024-03-28T16:51:27.860 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f109a59c640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f107c035a10 0x7f107c037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:27.861 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.857+0000 7f109a59c640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f107c035a10 0x7f107c037ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f108c005890 tx=0x7f108c007a90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:27.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:27.861+0000 7f10937fe640 1 -- 172.21.15.119:0/2604348512 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f109c133100 con 0x7f109c098830 2024-03-28T16:51:28.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.013+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f109c132ef0 con 0x7f107c035a10 2024-03-28T16:51:28.060 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.061+0000 7f10937fe640 1 -- 172.21.15.119:0/2604348512 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+669277 (secure 0 0 0) 0x7f108c001c00 con 0x7f107c035a10 2024-03-28T16:51:28.068 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:51:28.073 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:51:28.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f107c035a10 msgr2=0x7f107c037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:28.074 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f107c035a10 0x7f107c037ed0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f108c005890 tx=0x7f108c007a90 comp rx=0 tx=0).stop 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 msgr2=0x7f109c1c5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c1c5750 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1084019fa0 tx=0x7f108401abb0 comp rx=0 tx=0).stop 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 shutdown_connections 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f107c035a10 0x7f107c037ed0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 --2- 172.21.15.119:0/2604348512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f109c098830 0x7f109c1c5750 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 >> 172.21.15.119:0/2604348512 conn(0x7f109c0a0920 msgr2=0x7f109c12daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 shutdown_connections 2024-03-28T16:51:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:28.069+0000 7f10a10af640 1 -- 172.21.15.119:0/2604348512 wait complete. 2024-03-28T16:51:28.087 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-28T16:51:26.556629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":4029,"num_read_kb":3679,"num_write":1543,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4037,"ondisk_log_size":4037,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":160768,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":157945,"kb_avail":377326592,"statfs":{"total":386547056640,"available":386382430208,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":161736470},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005793"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412125+0000","last_change":"2024-03-28T16:48:54.656021+0000","last_active":"2024-03-28T16:50:46.412125+0000","last_peered":"2024-03-28T16:50:46.412125+0000","last_clean":"2024-03-28T16:50:46.412125+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:50:46.412125+0000","last_undegraded":"2024-03-28T16:50:46.412125+0000","last_fullsized":"2024-03-28T16:50:46.412125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.303489+0000","last_change":"2024-03-28T16:51:11.303646+0000","last_active":"2024-03-28T16:51:11.303489+0000","last_peered":"2024-03-28T16:51:11.303489+0000","last_clean":"2024-03-28T16:51:11.303489+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:51:11.303489+0000","last_undegraded":"2024-03-28T16:51:11.303489+0000","last_fullsized":"2024-03-28T16:51:11.303489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.303263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:59.401107+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532031+0000","last_change":"2024-03-28T16:48:54.666372+0000","last_active":"2024-03-28T16:50:45.532031+0000","last_peered":"2024-03-28T16:50:45.532031+0000","last_clean":"2024-03-28T16:50:45.532031+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:50:45.532031+0000","last_undegraded":"2024-03-28T16:50:45.532031+0000","last_fullsized":"2024-03-28T16:50:45.532031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.563876+0000","last_change":"2024-03-28T16:48:56.688050+0000","last_active":"2024-03-28T16:49:40.563876+0000","last_peered":"2024-03-28T16:49:40.563876+0000","last_clean":"2024-03-28T16:49:40.563876+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:49:40.563876+0000","last_undegraded":"2024-03-28T16:49:40.563876+0000","last_fullsized":"2024-03-28T16:49:40.563876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871923+0000","last_change":"2024-03-28T16:48:54.665976+0000","last_active":"2024-03-28T16:50:46.871923+0000","last_peered":"2024-03-28T16:50:46.871923+0000","last_clean":"2024-03-28T16:50:46.871923+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:50:46.871923+0000","last_undegraded":"2024-03-28T16:50:46.871923+0000","last_fullsized":"2024-03-28T16:50:46.871923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'4","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.105476+0000","last_change":"2024-03-28T16:51:19.105540+0000","last_active":"2024-03-28T16:51:19.105476+0000","last_peered":"2024-03-28T16:51:19.105476+0000","last_clean":"2024-03-28T16:51:19.105476+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:51:19.105476+0000","last_undegraded":"2024-03-28T16:51:19.105476+0000","last_fullsized":"2024-03-28T16:51:19.105476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.105244+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:35:56.759918+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'2","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412164+0000","last_change":"2024-03-28T16:48:54.656958+0000","last_active":"2024-03-28T16:50:46.412164+0000","last_peered":"2024-03-28T16:50:46.412164+0000","last_clean":"2024-03-28T16:50:46.412164+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:50:46.412164+0000","last_undegraded":"2024-03-28T16:50:46.412164+0000","last_fullsized":"2024-03-28T16:50:46.412164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.575289+0000","last_change":"2024-03-28T16:49:37.602262+0000","last_active":"2024-03-28T16:49:40.575289+0000","last_peered":"2024-03-28T16:49:40.575289+0000","last_clean":"2024-03-28T16:49:40.575289+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:49:40.575289+0000","last_undegraded":"2024-03-28T16:49:40.575289+0000","last_fullsized":"2024-03-28T16:49:40.575289+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:38.404379+0000","last_change":"2024-03-28T16:48:54.661971+0000","last_active":"2024-03-28T16:49:38.404379+0000","last_peered":"2024-03-28T16:49:38.404379+0000","last_clean":"2024-03-28T16:49:38.404379+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:49:38.404379+0000","last_undegraded":"2024-03-28T16:49:38.404379+0000","last_fullsized":"2024-03-28T16:49:38.404379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.305502+0000","last_change":"2024-03-28T16:51:16.305559+0000","last_active":"2024-03-28T16:51:16.305502+0000","last_peered":"2024-03-28T16:51:16.305502+0000","last_clean":"2024-03-28T16:51:16.305502+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:51:16.305502+0000","last_undegraded":"2024-03-28T16:51:16.305502+0000","last_fullsized":"2024-03-28T16:51:16.305502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.305318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:43.571649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.321896+0000","last_change":"2024-03-28T16:51:21.321896+0000","last_active":"2024-03-28T16:51:21.321896+0000","last_peered":"2024-03-28T16:51:21.321896+0000","last_clean":"2024-03-28T16:51:21.321896+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:51:21.321896+0000","last_undegraded":"2024-03-28T16:51:21.321896+0000","last_fullsized":"2024-03-28T16:51:21.321896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.321658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:30.540834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.057665+0000","last_change":"2024-03-28T16:51:20.057727+0000","last_active":"2024-03-28T16:51:20.057665+0000","last_peered":"2024-03-28T16:51:20.057665+0000","last_clean":"2024-03-28T16:51:20.057665+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:51:20.057665+0000","last_undegraded":"2024-03-28T16:51:20.057665+0000","last_fullsized":"2024-03-28T16:51:20.057665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.057477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:08.878405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.862019+0000","last_change":"2024-03-28T16:48:54.672203+0000","last_active":"2024-03-28T16:49:39.862019+0000","last_peered":"2024-03-28T16:49:39.862019+0000","last_clean":"2024-03-28T16:49:39.862019+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:49:39.862019+0000","last_undegraded":"2024-03-28T16:49:39.862019+0000","last_fullsized":"2024-03-28T16:49:39.862019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.413182+0000","last_change":"2024-03-28T16:48:56.695671+0000","last_active":"2024-03-28T16:50:46.413182+0000","last_peered":"2024-03-28T16:50:46.413182+0000","last_clean":"2024-03-28T16:50:46.413182+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:50:46.413182+0000","last_undegraded":"2024-03-28T16:50:46.413182+0000","last_fullsized":"2024-03-28T16:50:46.413182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.585269+0000","last_change":"2024-03-28T16:48:54.658053+0000","last_active":"2024-03-28T16:49:37.585269+0000","last_peered":"2024-03-28T16:49:37.585269+0000","last_clean":"2024-03-28T16:49:37.585269+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:49:37.585269+0000","last_undegraded":"2024-03-28T16:49:37.585269+0000","last_fullsized":"2024-03-28T16:49:37.585269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.353532+0000","last_change":"2024-03-28T16:51:12.353604+0000","last_active":"2024-03-28T16:51:12.353532+0000","last_peered":"2024-03-28T16:51:12.353532+0000","last_clean":"2024-03-28T16:51:12.353532+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:51:12.353532+0000","last_undegraded":"2024-03-28T16:51:12.353532+0000","last_fullsized":"2024-03-28T16:51:12.353532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.353309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:42.028446+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872583+0000","last_change":"2024-03-28T16:48:56.677244+0000","last_active":"2024-03-28T16:50:46.872583+0000","last_peered":"2024-03-28T16:50:46.872583+0000","last_clean":"2024-03-28T16:50:46.872583+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:50:46.872583+0000","last_undegraded":"2024-03-28T16:50:46.872583+0000","last_fullsized":"2024-03-28T16:50:46.872583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985133+0000","last_change":"2024-03-28T16:48:54.671518+0000","last_active":"2024-03-28T16:50:45.985133+0000","last_peered":"2024-03-28T16:50:45.985133+0000","last_clean":"2024-03-28T16:50:45.985133+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:50:45.985133+0000","last_undegraded":"2024-03-28T16:50:45.985133+0000","last_fullsized":"2024-03-28T16:50:45.985133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'10","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.246752+0000","last_change":"2024-03-28T16:51:07.246752+0000","last_active":"2024-03-28T16:51:07.246752+0000","last_peered":"2024-03-28T16:51:07.246752+0000","last_clean":"2024-03-28T16:51:07.246752+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:51:07.246752+0000","last_undegraded":"2024-03-28T16:51:07.246752+0000","last_fullsized":"2024-03-28T16:51:07.246752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.246384+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:42.040943+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412498+0000","last_change":"2024-03-28T16:49:37.610918+0000","last_active":"2024-03-28T16:50:46.412498+0000","last_peered":"2024-03-28T16:50:46.412498+0000","last_clean":"2024-03-28T16:50:46.412498+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:50:46.412498+0000","last_undegraded":"2024-03-28T16:50:46.412498+0000","last_fullsized":"2024-03-28T16:50:46.412498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.251563+0000","last_change":"2024-03-28T16:51:08.251563+0000","last_active":"2024-03-28T16:51:08.251563+0000","last_peered":"2024-03-28T16:51:08.251563+0000","last_clean":"2024-03-28T16:51:08.251563+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:51:08.251563+0000","last_undegraded":"2024-03-28T16:51:08.251563+0000","last_fullsized":"2024-03-28T16:51:08.251563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.251179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:34.785916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985156+0000","last_change":"2024-03-28T16:48:54.670148+0000","last_active":"2024-03-28T16:50:45.985156+0000","last_peered":"2024-03-28T16:50:45.985156+0000","last_clean":"2024-03-28T16:50:45.985156+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:50:45.985156+0000","last_undegraded":"2024-03-28T16:50:45.985156+0000","last_fullsized":"2024-03-28T16:50:45.985156+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532598+0000","last_change":"2024-03-28T16:48:56.678642+0000","last_active":"2024-03-28T16:50:45.532598+0000","last_peered":"2024-03-28T16:50:45.532598+0000","last_clean":"2024-03-28T16:50:45.532598+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:50:45.532598+0000","last_undegraded":"2024-03-28T16:50:45.532598+0000","last_fullsized":"2024-03-28T16:50:45.532598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985047+0000","last_change":"2024-03-28T16:48:54.655014+0000","last_active":"2024-03-28T16:50:45.985047+0000","last_peered":"2024-03-28T16:50:45.985047+0000","last_clean":"2024-03-28T16:50:45.985047+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:50:45.985047+0000","last_undegraded":"2024-03-28T16:50:45.985047+0000","last_fullsized":"2024-03-28T16:50:45.985047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.320871+0000","last_change":"2024-03-28T16:51:11.320871+0000","last_active":"2024-03-28T16:51:11.320871+0000","last_peered":"2024-03-28T16:51:11.320871+0000","last_clean":"2024-03-28T16:51:11.320871+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:51:11.320871+0000","last_undegraded":"2024-03-28T16:51:11.320871+0000","last_fullsized":"2024-03-28T16:51:11.320871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.320440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:29.097635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.480322+0000","last_change":"2024-03-28T16:51:20.480322+0000","last_active":"2024-03-28T16:51:20.480322+0000","last_peered":"2024-03-28T16:51:20.480322+0000","last_clean":"2024-03-28T16:51:20.480322+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:51:20.480322+0000","last_undegraded":"2024-03-28T16:51:20.480322+0000","last_fullsized":"2024-03-28T16:51:20.480322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.479971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:37.855992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.332059+0000","last_change":"2024-03-28T16:51:15.332059+0000","last_active":"2024-03-28T16:51:15.332059+0000","last_peered":"2024-03-28T16:51:15.332059+0000","last_clean":"2024-03-28T16:51:15.332059+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:51:15.332059+0000","last_undegraded":"2024-03-28T16:51:15.332059+0000","last_fullsized":"2024-03-28T16:51:15.332059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.331796+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:37:44.933269+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871718+0000","last_change":"2024-03-28T16:48:54.673730+0000","last_active":"2024-03-28T16:50:46.871718+0000","last_peered":"2024-03-28T16:50:46.871718+0000","last_clean":"2024-03-28T16:50:46.871718+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:50:46.871718+0000","last_undegraded":"2024-03-28T16:50:46.871718+0000","last_fullsized":"2024-03-28T16:50:46.871718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'19","reported_seq":78,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:28.872837+0000","last_change":"2024-03-28T16:49:04.742579+0000","last_active":"2024-03-28T16:50:28.872837+0000","last_peered":"2024-03-28T16:50:28.872837+0000","last_clean":"2024-03-28T16:50:28.872837+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:50:28.872837+0000","last_undegraded":"2024-03-28T16:50:28.872837+0000","last_fullsized":"2024-03-28T16:50:28.872837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872098+0000","last_change":"2024-03-28T16:48:56.683934+0000","last_active":"2024-03-28T16:50:46.872098+0000","last_peered":"2024-03-28T16:50:46.872098+0000","last_clean":"2024-03-28T16:50:46.872098+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:50:46.872098+0000","last_undegraded":"2024-03-28T16:50:46.872098+0000","last_fullsized":"2024-03-28T16:50:46.872098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871328+0000","last_change":"2024-03-28T16:48:54.670023+0000","last_active":"2024-03-28T16:50:46.871328+0000","last_peered":"2024-03-28T16:50:46.871328+0000","last_clean":"2024-03-28T16:50:46.871328+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:50:46.871328+0000","last_undegraded":"2024-03-28T16:50:46.871328+0000","last_fullsized":"2024-03-28T16:50:46.871328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.137952+0000","last_change":"2024-03-28T16:50:50.138003+0000","last_active":"2024-03-28T16:50:50.137952+0000","last_peered":"2024-03-28T16:50:50.137952+0000","last_clean":"2024-03-28T16:50:50.137952+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:50:50.137952+0000","last_undegraded":"2024-03-28T16:50:50.137952+0000","last_fullsized":"2024-03-28T16:50:50.137952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.137765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:11.908122+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'34","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.226584+0000","last_change":"2024-03-28T16:51:10.226657+0000","last_active":"2024-03-28T16:51:10.226584+0000","last_peered":"2024-03-28T16:51:10.226584+0000","last_clean":"2024-03-28T16:51:10.226584+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:51:10.226584+0000","last_undegraded":"2024-03-28T16:51:10.226584+0000","last_fullsized":"2024-03-28T16:51:10.226584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.226407+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:01.054784+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.251316+0000","last_change":"2024-03-28T16:50:57.251369+0000","last_active":"2024-03-28T16:50:57.251316+0000","last_peered":"2024-03-28T16:50:57.251316+0000","last_clean":"2024-03-28T16:50:57.251316+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:50:57.251316+0000","last_undegraded":"2024-03-28T16:50:57.251316+0000","last_fullsized":"2024-03-28T16:50:57.251316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.251130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:45.232439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'6","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:22.940810+0000","last_change":"2024-03-28T16:49:04.742984+0000","last_active":"2024-03-28T16:50:22.940810+0000","last_peered":"2024-03-28T16:50:22.940810+0000","last_clean":"2024-03-28T16:50:22.940810+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:50:22.940810+0000","last_undegraded":"2024-03-28T16:50:22.940810+0000","last_fullsized":"2024-03-28T16:50:22.940810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":67,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872310+0000","last_change":"2024-03-28T16:48:56.671488+0000","last_active":"2024-03-28T16:50:46.872310+0000","last_peered":"2024-03-28T16:50:46.872310+0000","last_clean":"2024-03-28T16:50:46.872310+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:50:46.872310+0000","last_undegraded":"2024-03-28T16:50:46.872310+0000","last_fullsized":"2024-03-28T16:50:46.872310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.411953+0000","last_change":"2024-03-28T16:48:54.655701+0000","last_active":"2024-03-28T16:50:46.411953+0000","last_peered":"2024-03-28T16:50:46.411953+0000","last_clean":"2024-03-28T16:50:46.411953+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:50:46.411953+0000","last_undegraded":"2024-03-28T16:50:46.411953+0000","last_fullsized":"2024-03-28T16:50:46.411953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.328962+0000","last_change":"2024-03-28T16:51:04.328962+0000","last_active":"2024-03-28T16:51:04.328962+0000","last_peered":"2024-03-28T16:51:04.328962+0000","last_clean":"2024-03-28T16:51:04.328962+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:51:04.328962+0000","last_undegraded":"2024-03-28T16:51:04.328962+0000","last_fullsized":"2024-03-28T16:51:04.328962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.328732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:30.892979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"31'10","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.284276+0000","last_change":"2024-03-28T16:50:55.284276+0000","last_active":"2024-03-28T16:50:55.284276+0000","last_peered":"2024-03-28T16:50:55.284276+0000","last_clean":"2024-03-28T16:50:55.284276+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:50:55.284276+0000","last_undegraded":"2024-03-28T16:50:55.284276+0000","last_fullsized":"2024-03-28T16:50:55.284276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.283889+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:07.952223+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.156551+0000","last_change":"2024-03-28T16:50:49.156607+0000","last_active":"2024-03-28T16:50:49.156551+0000","last_peered":"2024-03-28T16:50:49.156551+0000","last_clean":"2024-03-28T16:50:49.156551+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:50:49.156551+0000","last_undegraded":"2024-03-28T16:50:49.156551+0000","last_fullsized":"2024-03-28T16:50:49.156551+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.156374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:19.754691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.584478+0000","last_change":"2024-03-28T16:49:04.743161+0000","last_active":"2024-03-28T16:49:37.584478+0000","last_peered":"2024-03-28T16:49:37.584478+0000","last_clean":"2024-03-28T16:49:37.584478+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:49:37.584478+0000","last_undegraded":"2024-03-28T16:49:37.584478+0000","last_fullsized":"2024-03-28T16:49:37.584478+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872362+0000","last_change":"2024-03-28T16:48:56.676331+0000","last_active":"2024-03-28T16:50:46.872362+0000","last_peered":"2024-03-28T16:50:46.872362+0000","last_clean":"2024-03-28T16:50:46.872362+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:50:46.872362+0000","last_undegraded":"2024-03-28T16:50:46.872362+0000","last_fullsized":"2024-03-28T16:50:46.872362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871281+0000","last_change":"2024-03-28T16:48:54.659330+0000","last_active":"2024-03-28T16:50:46.871281+0000","last_peered":"2024-03-28T16:50:46.871281+0000","last_clean":"2024-03-28T16:50:46.871281+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:50:46.871281+0000","last_undegraded":"2024-03-28T16:50:46.871281+0000","last_fullsized":"2024-03-28T16:50:46.871281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'6","reported_seq":215,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.337648+0000","last_change":"2024-03-28T16:51:08.337837+0000","last_active":"2024-03-28T16:51:08.337648+0000","last_peered":"2024-03-28T16:51:08.337648+0000","last_clean":"2024-03-28T16:51:08.337648+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:51:08.337648+0000","last_undegraded":"2024-03-28T16:51:08.337648+0000","last_fullsized":"2024-03-28T16:51:08.337648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.337418+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:50.313341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":270,"num_read_kb":269,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'16","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.357686+0000","last_change":"2024-03-28T16:50:59.357686+0000","last_active":"2024-03-28T16:50:59.357686+0000","last_peered":"2024-03-28T16:50:59.357686+0000","last_clean":"2024-03-28T16:50:59.357686+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:50:59.357686+0000","last_undegraded":"2024-03-28T16:50:59.357686+0000","last_fullsized":"2024-03-28T16:50:59.357686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.357393+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:54.820495+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'249","reported_seq":341,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.231778+0000","last_change":"2024-03-28T16:51:08.231850+0000","last_active":"2024-03-28T16:51:08.231778+0000","last_peered":"2024-03-28T16:51:08.231778+0000","last_clean":"2024-03-28T16:51:08.231778+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:51:08.231778+0000","last_undegraded":"2024-03-28T16:51:08.231778+0000","last_fullsized":"2024-03-28T16:51:08.231778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.231583+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:26.966593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.350859+0000","last_change":"2024-03-28T16:51:04.350955+0000","last_active":"2024-03-28T16:51:04.350859+0000","last_peered":"2024-03-28T16:51:04.350859+0000","last_clean":"2024-03-28T16:51:04.350859+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:51:04.350859+0000","last_undegraded":"2024-03-28T16:51:04.350859+0000","last_fullsized":"2024-03-28T16:51:04.350859+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.350588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:52.331897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.382333+0000","last_change":"2024-03-28T16:51:14.382333+0000","last_active":"2024-03-28T16:51:14.382333+0000","last_peered":"2024-03-28T16:51:14.382333+0000","last_clean":"2024-03-28T16:51:14.382333+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:51:14.382333+0000","last_undegraded":"2024-03-28T16:51:14.382333+0000","last_fullsized":"2024-03-28T16:51:14.382333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.382083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:28.408578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984581+0000","last_change":"2024-03-28T16:48:54.659035+0000","last_active":"2024-03-28T16:50:45.984581+0000","last_peered":"2024-03-28T16:50:45.984581+0000","last_clean":"2024-03-28T16:50:45.984581+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:50:45.984581+0000","last_undegraded":"2024-03-28T16:50:45.984581+0000","last_fullsized":"2024-03-28T16:50:45.984581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.124205+0000","last_change":"2024-03-28T16:50:49.124265+0000","last_active":"2024-03-28T16:50:49.124205+0000","last_peered":"2024-03-28T16:50:49.124205+0000","last_clean":"2024-03-28T16:50:49.124205+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:50:49.124205+0000","last_undegraded":"2024-03-28T16:50:49.124205+0000","last_fullsized":"2024-03-28T16:50:49.124205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.124014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.365150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.425199+0000","last_change":"2024-03-28T16:50:57.425199+0000","last_active":"2024-03-28T16:50:57.425199+0000","last_peered":"2024-03-28T16:50:57.425199+0000","last_clean":"2024-03-28T16:50:57.425199+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:57.425199+0000","last_undegraded":"2024-03-28T16:50:57.425199+0000","last_fullsized":"2024-03-28T16:50:57.425199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.424946+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:19.309481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'249","reported_seq":356,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.296438+0000","last_change":"2024-03-28T16:51:09.296501+0000","last_active":"2024-03-28T16:51:09.296438+0000","last_peered":"2024-03-28T16:51:09.296438+0000","last_clean":"2024-03-28T16:51:09.296438+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:51:09.296438+0000","last_undegraded":"2024-03-28T16:51:09.296438+0000","last_fullsized":"2024-03-28T16:51:09.296438+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.296245+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:57.394089+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.285771+0000","last_change":"2024-03-28T16:51:06.285771+0000","last_active":"2024-03-28T16:51:06.285771+0000","last_peered":"2024-03-28T16:51:06.285771+0000","last_clean":"2024-03-28T16:51:06.285771+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:51:06.285771+0000","last_undegraded":"2024-03-28T16:51:06.285771+0000","last_fullsized":"2024-03-28T16:51:06.285771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.285504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:11.465640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.389203+0000","last_change":"2024-03-28T16:51:17.389264+0000","last_active":"2024-03-28T16:51:17.389203+0000","last_peered":"2024-03-28T16:51:17.389203+0000","last_clean":"2024-03-28T16:51:17.389203+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:51:17.389203+0000","last_undegraded":"2024-03-28T16:51:17.389203+0000","last_fullsized":"2024-03-28T16:51:17.389203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.389016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:59.364938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.302414+0000","last_change":"2024-03-28T16:51:00.302414+0000","last_active":"2024-03-28T16:51:00.302414+0000","last_peered":"2024-03-28T16:51:00.302414+0000","last_clean":"2024-03-28T16:51:00.302414+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:51:00.302414+0000","last_undegraded":"2024-03-28T16:51:00.302414+0000","last_fullsized":"2024-03-28T16:51:00.302414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.302183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:46.487988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'49","reported_seq":109,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.444852+0000","last_change":"2024-03-28T16:50:55.444852+0000","last_active":"2024-03-28T16:50:55.444852+0000","last_peered":"2024-03-28T16:50:55.444852+0000","last_clean":"2024-03-28T16:50:55.444852+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:55.444852+0000","last_undegraded":"2024-03-28T16:50:55.444852+0000","last_fullsized":"2024-03-28T16:50:55.444852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'49","last_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_deep_scrub":"31'49","last_deep_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.444545+0000","objects_scrubbed":9,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:56.707599+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.195172+0000","last_change":"2024-03-28T16:50:53.195172+0000","last_active":"2024-03-28T16:50:53.195172+0000","last_peered":"2024-03-28T16:50:53.195172+0000","last_clean":"2024-03-28T16:50:53.195172+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:50:53.195172+0000","last_undegraded":"2024-03-28T16:50:53.195172+0000","last_fullsized":"2024-03-28T16:50:53.195172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.194894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:05.863228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.397864+0000","last_change":"2024-03-28T16:50:56.397864+0000","last_active":"2024-03-28T16:50:56.397864+0000","last_peered":"2024-03-28T16:50:56.397864+0000","last_clean":"2024-03-28T16:50:56.397864+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:56.397864+0000","last_undegraded":"2024-03-28T16:50:56.397864+0000","last_fullsized":"2024-03-28T16:50:56.397864+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.397465+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:26.995782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'24","reported_seq":112,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.267353+0000","last_change":"2024-03-28T16:51:17.267421+0000","last_active":"2024-03-28T16:51:17.267353+0000","last_peered":"2024-03-28T16:51:17.267353+0000","last_clean":"2024-03-28T16:51:17.267353+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:51:17.267353+0000","last_undegraded":"2024-03-28T16:51:17.267353+0000","last_fullsized":"2024-03-28T16:51:17.267353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'24","last_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_deep_scrub":"31'24","last_deep_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.267112+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:31.293607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.358051+0000","last_change":"2024-03-28T16:51:15.358174+0000","last_active":"2024-03-28T16:51:15.358051+0000","last_peered":"2024-03-28T16:51:15.358051+0000","last_clean":"2024-03-28T16:51:15.358051+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:51:15.358051+0000","last_undegraded":"2024-03-28T16:51:15.358051+0000","last_fullsized":"2024-03-28T16:51:15.358051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.357818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:41.892555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.371282+0000","last_change":"2024-03-28T16:51:02.371282+0000","last_active":"2024-03-28T16:51:02.371282+0000","last_peered":"2024-03-28T16:51:02.371282+0000","last_clean":"2024-03-28T16:51:02.371282+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:51:02.371282+0000","last_undegraded":"2024-03-28T16:51:02.371282+0000","last_fullsized":"2024-03-28T16:51:02.371282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.371035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:20.148230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.191604+0000","last_change":"2024-03-28T16:51:11.191604+0000","last_active":"2024-03-28T16:51:11.191604+0000","last_peered":"2024-03-28T16:51:11.191604+0000","last_clean":"2024-03-28T16:51:11.191604+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:51:11.191604+0000","last_undegraded":"2024-03-28T16:51:11.191604+0000","last_fullsized":"2024-03-28T16:51:11.191604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.191343+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:47:25.401107+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.183048+0000","last_change":"2024-03-28T16:50:51.183048+0000","last_active":"2024-03-28T16:50:51.183048+0000","last_peered":"2024-03-28T16:50:51.183048+0000","last_clean":"2024-03-28T16:50:51.183048+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:50:51.183048+0000","last_undegraded":"2024-03-28T16:50:51.183048+0000","last_fullsized":"2024-03-28T16:50:51.183048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.182678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:48.071769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'4","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.320355+0000","last_change":"2024-03-28T16:51:05.320355+0000","last_active":"2024-03-28T16:51:05.320355+0000","last_peered":"2024-03-28T16:51:05.320355+0000","last_clean":"2024-03-28T16:51:05.320355+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:51:05.320355+0000","last_undegraded":"2024-03-28T16:51:05.320355+0000","last_fullsized":"2024-03-28T16:51:05.320355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.320076+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:34.995213+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.314961+0000","last_change":"2024-03-28T16:51:14.315025+0000","last_active":"2024-03-28T16:51:14.314961+0000","last_peered":"2024-03-28T16:51:14.314961+0000","last_clean":"2024-03-28T16:51:14.314961+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:51:14.314961+0000","last_undegraded":"2024-03-28T16:51:14.314961+0000","last_fullsized":"2024-03-28T16:51:14.314961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.314772+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:49.109332+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871271+0000","last_change":"2024-03-28T16:48:54.676553+0000","last_active":"2024-03-28T16:50:46.871271+0000","last_peered":"2024-03-28T16:50:46.871271+0000","last_clean":"2024-03-28T16:50:46.871271+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:50:46.871271+0000","last_undegraded":"2024-03-28T16:50:46.871271+0000","last_fullsized":"2024-03-28T16:50:46.871271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.352892+0000","last_change":"2024-03-28T16:51:11.352892+0000","last_active":"2024-03-28T16:51:11.352892+0000","last_peered":"2024-03-28T16:51:11.352892+0000","last_clean":"2024-03-28T16:51:11.352892+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:51:11.352892+0000","last_undegraded":"2024-03-28T16:51:11.352892+0000","last_fullsized":"2024-03-28T16:51:11.352892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.352650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:16.271950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.096269+0000","last_change":"2024-03-28T16:51:18.096365+0000","last_active":"2024-03-28T16:51:18.096269+0000","last_peered":"2024-03-28T16:51:18.096269+0000","last_clean":"2024-03-28T16:51:18.096269+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:51:18.096269+0000","last_undegraded":"2024-03-28T16:51:18.096269+0000","last_fullsized":"2024-03-28T16:51:18.096269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.096048+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:10.105136+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'249","reported_seq":352,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.171801+0000","last_change":"2024-03-28T16:50:52.171801+0000","last_active":"2024-03-28T16:50:52.171801+0000","last_peered":"2024-03-28T16:50:52.171801+0000","last_clean":"2024-03-28T16:50:52.171801+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:52.171801+0000","last_undegraded":"2024-03-28T16:50:52.171801+0000","last_fullsized":"2024-03-28T16:50:52.171801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.171465+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:19.659817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.533038+0000","last_change":"2024-03-28T16:49:04.743207+0000","last_active":"2024-03-28T16:50:45.533038+0000","last_peered":"2024-03-28T16:50:45.533038+0000","last_clean":"2024-03-28T16:50:45.533038+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:50:45.533038+0000","last_undegraded":"2024-03-28T16:50:45.533038+0000","last_fullsized":"2024-03-28T16:50:45.533038+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.394496+0000","last_change":"2024-03-28T16:51:18.394559+0000","last_active":"2024-03-28T16:51:18.394496+0000","last_peered":"2024-03-28T16:51:18.394496+0000","last_clean":"2024-03-28T16:51:18.394496+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:51:18.394496+0000","last_undegraded":"2024-03-28T16:51:18.394496+0000","last_fullsized":"2024-03-28T16:51:18.394496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.394309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:06.492152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871317+0000","last_change":"2024-03-28T16:48:54.658093+0000","last_active":"2024-03-28T16:50:46.871317+0000","last_peered":"2024-03-28T16:50:46.871317+0000","last_clean":"2024-03-28T16:50:46.871317+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:50:46.871317+0000","last_undegraded":"2024-03-28T16:50:46.871317+0000","last_fullsized":"2024-03-28T16:50:46.871317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.386277+0000","last_change":"2024-03-28T16:51:12.386390+0000","last_active":"2024-03-28T16:51:12.386277+0000","last_peered":"2024-03-28T16:51:12.386277+0000","last_clean":"2024-03-28T16:51:12.386277+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:51:12.386277+0000","last_undegraded":"2024-03-28T16:51:12.386277+0000","last_fullsized":"2024-03-28T16:51:12.386277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.386042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:21.605218+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'61","reported_seq":134,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.103088+0000","last_change":"2024-03-28T16:51:14.103172+0000","last_active":"2024-03-28T16:51:14.103088+0000","last_peered":"2024-03-28T16:51:14.103088+0000","last_clean":"2024-03-28T16:51:14.103088+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:51:14.103088+0000","last_undegraded":"2024-03-28T16:51:14.103088+0000","last_fullsized":"2024-03-28T16:51:14.103088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'61","last_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_deep_scrub":"31'61","last_deep_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.102888+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:13:35.860974+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.181042+0000","last_change":"2024-03-28T16:50:54.181042+0000","last_active":"2024-03-28T16:50:54.181042+0000","last_peered":"2024-03-28T16:50:54.181042+0000","last_clean":"2024-03-28T16:50:54.181042+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:50:54.181042+0000","last_undegraded":"2024-03-28T16:50:54.181042+0000","last_fullsized":"2024-03-28T16:50:54.181042+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.180810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:17.080003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.393450+0000","last_change":"2024-03-28T16:51:13.393450+0000","last_active":"2024-03-28T16:51:13.393450+0000","last_peered":"2024-03-28T16:51:13.393450+0000","last_clean":"2024-03-28T16:51:13.393450+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:51:13.393450+0000","last_undegraded":"2024-03-28T16:51:13.393450+0000","last_fullsized":"2024-03-28T16:51:13.393450+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.393212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:22.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985236+0000","last_change":"2024-03-28T16:48:56.683578+0000","last_active":"2024-03-28T16:50:45.985236+0000","last_peered":"2024-03-28T16:50:45.985236+0000","last_clean":"2024-03-28T16:50:45.985236+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:50:45.985236+0000","last_undegraded":"2024-03-28T16:50:45.985236+0000","last_fullsized":"2024-03-28T16:50:45.985236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531868+0000","last_change":"2024-03-28T16:48:54.664924+0000","last_active":"2024-03-28T16:50:45.531868+0000","last_peered":"2024-03-28T16:50:45.531868+0000","last_clean":"2024-03-28T16:50:45.531868+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:50:45.531868+0000","last_undegraded":"2024-03-28T16:50:45.531868+0000","last_fullsized":"2024-03-28T16:50:45.531868+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.365035+0000","last_change":"2024-03-28T16:51:05.365035+0000","last_active":"2024-03-28T16:51:05.365035+0000","last_peered":"2024-03-28T16:51:05.365035+0000","last_clean":"2024-03-28T16:51:05.365035+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:51:05.365035+0000","last_undegraded":"2024-03-28T16:51:05.365035+0000","last_fullsized":"2024-03-28T16:51:05.365035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.364765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:50.121796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'37","reported_seq":93,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.221214+0000","last_change":"2024-03-28T16:50:51.221214+0000","last_active":"2024-03-28T16:50:51.221214+0000","last_peered":"2024-03-28T16:50:51.221214+0000","last_clean":"2024-03-28T16:50:51.221214+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:51.221214+0000","last_undegraded":"2024-03-28T16:50:51.221214+0000","last_fullsized":"2024-03-28T16:50:51.221214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'37","last_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_deep_scrub":"31'37","last_deep_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.220960+0000","objects_scrubbed":5,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:47.323997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.275109+0000","last_change":"2024-03-28T16:51:07.275109+0000","last_active":"2024-03-28T16:51:07.275109+0000","last_peered":"2024-03-28T16:51:07.275109+0000","last_clean":"2024-03-28T16:51:07.275109+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:51:07.275109+0000","last_undegraded":"2024-03-28T16:51:07.275109+0000","last_fullsized":"2024-03-28T16:51:07.275109+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.274868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:03.242566+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874117+0000","last_change":"2024-03-28T16:49:04.748645+0000","last_active":"2024-03-28T16:50:46.874117+0000","last_peered":"2024-03-28T16:50:46.874117+0000","last_clean":"2024-03-28T16:50:46.874117+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:50:46.874117+0000","last_undegraded":"2024-03-28T16:50:46.874117+0000","last_fullsized":"2024-03-28T16:50:46.874117+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.193692+0000","last_change":"2024-03-28T16:51:00.193692+0000","last_active":"2024-03-28T16:51:00.193692+0000","last_peered":"2024-03-28T16:51:00.193692+0000","last_clean":"2024-03-28T16:51:00.193692+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:51:00.193692+0000","last_undegraded":"2024-03-28T16:51:00.193692+0000","last_fullsized":"2024-03-28T16:51:00.193692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.193439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:35.074661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871223+0000","last_change":"2024-03-28T16:48:54.673299+0000","last_active":"2024-03-28T16:50:46.871223+0000","last_peered":"2024-03-28T16:50:46.871223+0000","last_clean":"2024-03-28T16:50:46.871223+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:50:46.871223+0000","last_undegraded":"2024-03-28T16:50:46.871223+0000","last_fullsized":"2024-03-28T16:50:46.871223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.345768+0000","last_change":"2024-03-28T16:51:10.345768+0000","last_active":"2024-03-28T16:51:10.345768+0000","last_peered":"2024-03-28T16:51:10.345768+0000","last_clean":"2024-03-28T16:51:10.345768+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:51:10.345768+0000","last_undegraded":"2024-03-28T16:51:10.345768+0000","last_fullsized":"2024-03-28T16:51:10.345768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.345522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:45.930075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.389452+0000","last_change":"2024-03-28T16:51:01.389452+0000","last_active":"2024-03-28T16:51:01.389452+0000","last_peered":"2024-03-28T16:51:01.389452+0000","last_clean":"2024-03-28T16:51:01.389452+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:51:01.389452+0000","last_undegraded":"2024-03-28T16:51:01.389452+0000","last_fullsized":"2024-03-28T16:51:01.389452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.389173+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:28.877525+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'249","reported_seq":335,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.428834+0000","last_change":"2024-03-28T16:50:47.428834+0000","last_active":"2024-03-28T16:50:47.428834+0000","last_peered":"2024-03-28T16:50:47.428834+0000","last_clean":"2024-03-28T16:50:47.428834+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:47.428834+0000","last_undegraded":"2024-03-28T16:50:47.428834+0000","last_fullsized":"2024-03-28T16:50:47.428834+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.428456+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:01.404074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'13","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532760+0000","last_change":"2024-03-28T16:49:04.743257+0000","last_active":"2024-03-28T16:50:45.532760+0000","last_peered":"2024-03-28T16:50:45.532760+0000","last_clean":"2024-03-28T16:50:45.532760+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:50:45.532760+0000","last_undegraded":"2024-03-28T16:50:45.532760+0000","last_fullsized":"2024-03-28T16:50:45.532760+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.337241+0000","last_change":"2024-03-28T16:51:22.337362+0000","last_active":"2024-03-28T16:51:22.337241+0000","last_peered":"2024-03-28T16:51:22.337241+0000","last_clean":"2024-03-28T16:51:22.337241+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:51:22.337241+0000","last_undegraded":"2024-03-28T16:51:22.337241+0000","last_fullsized":"2024-03-28T16:51:22.337241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.336956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:31.109615+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871187+0000","last_change":"2024-03-28T16:48:54.664571+0000","last_active":"2024-03-28T16:50:46.871187+0000","last_peered":"2024-03-28T16:50:46.871187+0000","last_clean":"2024-03-28T16:50:46.871187+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:50:46.871187+0000","last_undegraded":"2024-03-28T16:50:46.871187+0000","last_fullsized":"2024-03-28T16:50:46.871187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'3","reported_seq":384,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.986155+0000","last_change":"2024-03-28T16:49:37.622013+0000","last_active":"2024-03-28T16:50:45.986155+0000","last_peered":"2024-03-28T16:50:45.986155+0000","last_clean":"2024-03-28T16:50:45.986155+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:50:45.986155+0000","last_undegraded":"2024-03-28T16:50:45.986155+0000","last_fullsized":"2024-03-28T16:50:45.986155+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":340,"num_read_kb":340,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.144955+0000","last_change":"2024-03-28T16:51:12.144955+0000","last_active":"2024-03-28T16:51:12.144955+0000","last_peered":"2024-03-28T16:51:12.144955+0000","last_clean":"2024-03-28T16:51:12.144955+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:51:12.144955+0000","last_undegraded":"2024-03-28T16:51:12.144955+0000","last_fullsized":"2024-03-28T16:51:12.144955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.144489+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:36.042267+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'249","reported_seq":358,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.109690+0000","last_change":"2024-03-28T16:50:48.109690+0000","last_active":"2024-03-28T16:50:48.109690+0000","last_peered":"2024-03-28T16:50:48.109690+0000","last_clean":"2024-03-28T16:50:48.109690+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:48.109690+0000","last_undegraded":"2024-03-28T16:50:48.109690+0000","last_fullsized":"2024-03-28T16:50:48.109690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.109433+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:49.372487+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.412356+0000","last_change":"2024-03-28T16:51:08.412356+0000","last_active":"2024-03-28T16:51:08.412356+0000","last_peered":"2024-03-28T16:51:08.412356+0000","last_clean":"2024-03-28T16:51:08.412356+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:51:08.412356+0000","last_undegraded":"2024-03-28T16:51:08.412356+0000","last_fullsized":"2024-03-28T16:51:08.412356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.412104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:37.251504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.269854+0000","last_change":"2024-03-28T16:51:25.269969+0000","last_active":"2024-03-28T16:51:25.269854+0000","last_peered":"2024-03-28T16:51:25.269854+0000","last_clean":"2024-03-28T16:51:25.269854+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:51:25.269854+0000","last_undegraded":"2024-03-28T16:51:25.269854+0000","last_fullsized":"2024-03-28T16:51:25.269854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.269642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:53:12.659384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870991+0000","last_change":"2024-03-28T16:48:54.676487+0000","last_active":"2024-03-28T16:50:46.870991+0000","last_peered":"2024-03-28T16:50:46.870991+0000","last_clean":"2024-03-28T16:50:46.870991+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:50:46.870991+0000","last_undegraded":"2024-03-28T16:50:46.870991+0000","last_fullsized":"2024-03-28T16:50:46.870991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.165481+0000","last_change":"2024-03-28T16:50:48.165481+0000","last_active":"2024-03-28T16:50:48.165481+0000","last_peered":"2024-03-28T16:50:48.165481+0000","last_clean":"2024-03-28T16:50:48.165481+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:50:48.165481+0000","last_undegraded":"2024-03-28T16:50:48.165481+0000","last_fullsized":"2024-03-28T16:50:48.165481+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.165239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:40:14.898173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'46","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.272646+0000","last_change":"2024-03-28T16:50:56.272646+0000","last_active":"2024-03-28T16:50:56.272646+0000","last_peered":"2024-03-28T16:50:56.272646+0000","last_clean":"2024-03-28T16:50:56.272646+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:56.272646+0000","last_undegraded":"2024-03-28T16:50:56.272646+0000","last_fullsized":"2024-03-28T16:50:56.272646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'46","last_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_deep_scrub":"31'46","last_deep_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.272353+0000","objects_scrubbed":8,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:19.171546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.431537+0000","last_change":"2024-03-28T16:50:53.431537+0000","last_active":"2024-03-28T16:50:53.431537+0000","last_peered":"2024-03-28T16:50:53.431537+0000","last_clean":"2024-03-28T16:50:53.431537+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:50:53.431537+0000","last_undegraded":"2024-03-28T16:50:53.431537+0000","last_fullsized":"2024-03-28T16:50:53.431537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.431293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:35.681957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.348998+0000","last_change":"2024-03-28T16:51:01.348998+0000","last_active":"2024-03-28T16:51:01.348998+0000","last_peered":"2024-03-28T16:51:01.348998+0000","last_clean":"2024-03-28T16:51:01.348998+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:51:01.348998+0000","last_undegraded":"2024-03-28T16:51:01.348998+0000","last_fullsized":"2024-03-28T16:51:01.348998+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.348719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:30.188118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.197537+0000","last_change":"2024-03-28T16:51:03.197596+0000","last_active":"2024-03-28T16:51:03.197537+0000","last_peered":"2024-03-28T16:51:03.197537+0000","last_clean":"2024-03-28T16:51:03.197537+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:51:03.197537+0000","last_undegraded":"2024-03-28T16:51:03.197537+0000","last_fullsized":"2024-03-28T16:51:03.197537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.197346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:04:49.417548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'117","reported_seq":200,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.592395+0000","last_change":"2024-03-28T16:48:44.464991+0000","last_active":"2024-03-28T16:50:45.592395+0000","last_peered":"2024-03-28T16:50:45.592395+0000","last_clean":"2024-03-28T16:50:45.592395+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:50:45.592395+0000","last_undegraded":"2024-03-28T16:50:45.592395+0000","last_fullsized":"2024-03-28T16:50:45.592395+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:42.575180+0000","last_clean_scrub_stamp":"2024-03-28T16:48:42.575180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870781+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:50:46.870781+0000","last_peered":"2024-03-28T16:50:46.870781+0000","last_clean":"2024-03-28T16:50:46.870781+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:50:46.870781+0000","last_undegraded":"2024-03-28T16:50:46.870781+0000","last_fullsized":"2024-03-28T16:50:46.870781+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870947+0000","last_change":"2024-03-28T16:48:54.666099+0000","last_active":"2024-03-28T16:50:46.870947+0000","last_peered":"2024-03-28T16:50:46.870947+0000","last_clean":"2024-03-28T16:50:46.870947+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:50:46.870947+0000","last_undegraded":"2024-03-28T16:50:46.870947+0000","last_fullsized":"2024-03-28T16:50:46.870947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.295412+0000","last_change":"2024-03-28T16:51:03.295412+0000","last_active":"2024-03-28T16:51:03.295412+0000","last_peered":"2024-03-28T16:51:03.295412+0000","last_clean":"2024-03-28T16:51:03.295412+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:51:03.295412+0000","last_undegraded":"2024-03-28T16:51:03.295412+0000","last_fullsized":"2024-03-28T16:51:03.295412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.295139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:08.548996+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'81","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.160163+0000","last_change":"2024-03-28T16:50:53.160163+0000","last_active":"2024-03-28T16:50:53.160163+0000","last_peered":"2024-03-28T16:50:53.160163+0000","last_clean":"2024-03-28T16:50:53.160163+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:53.160163+0000","last_undegraded":"2024-03-28T16:50:53.160163+0000","last_fullsized":"2024-03-28T16:50:53.160163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'81","last_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_deep_scrub":"31'81","last_deep_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.159787+0000","objects_scrubbed":5,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:32.543735+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.290671+0000","last_change":"2024-03-28T16:50:57.290671+0000","last_active":"2024-03-28T16:50:57.290671+0000","last_peered":"2024-03-28T16:50:57.290671+0000","last_clean":"2024-03-28T16:50:57.290671+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:50:57.290671+0000","last_undegraded":"2024-03-28T16:50:57.290671+0000","last_fullsized":"2024-03-28T16:50:57.290671+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.290416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:32.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'52","reported_seq":116,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.414096+0000","last_change":"2024-03-28T16:49:04.752277+0000","last_active":"2024-03-28T16:50:46.414096+0000","last_peered":"2024-03-28T16:50:46.414096+0000","last_clean":"2024-03-28T16:50:46.414096+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:50:46.414096+0000","last_undegraded":"2024-03-28T16:50:46.414096+0000","last_fullsized":"2024-03-28T16:50:46.414096+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'11","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469786+0000","last_change":"2024-03-28T16:49:37.614693+0000","last_active":"2024-03-28T16:50:35.469786+0000","last_peered":"2024-03-28T16:50:35.469786+0000","last_clean":"2024-03-28T16:50:35.469786+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:50:35.469786+0000","last_undegraded":"2024-03-28T16:50:35.469786+0000","last_fullsized":"2024-03-28T16:50:35.469786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870684+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:50:46.870684+0000","last_peered":"2024-03-28T16:50:46.870684+0000","last_clean":"2024-03-28T16:50:46.870684+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:50:46.870684+0000","last_undegraded":"2024-03-28T16:50:46.870684+0000","last_fullsized":"2024-03-28T16:50:46.870684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532168+0000","last_change":"2024-03-28T16:48:54.662603+0000","last_active":"2024-03-28T16:50:45.532168+0000","last_peered":"2024-03-28T16:50:45.532168+0000","last_clean":"2024-03-28T16:50:45.532168+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:50:45.532168+0000","last_undegraded":"2024-03-28T16:50:45.532168+0000","last_fullsized":"2024-03-28T16:50:45.532168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.191091+0000","last_change":"2024-03-28T16:50:47.191091+0000","last_active":"2024-03-28T16:50:47.191091+0000","last_peered":"2024-03-28T16:50:47.191091+0000","last_clean":"2024-03-28T16:50:47.191091+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:50:47.191091+0000","last_undegraded":"2024-03-28T16:50:47.191091+0000","last_fullsized":"2024-03-28T16:50:47.191091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.190844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:09.946965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'260","reported_seq":455,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.259946+0000","last_change":"2024-03-28T16:50:54.259946+0000","last_active":"2024-03-28T16:50:54.259946+0000","last_peered":"2024-03-28T16:50:54.259946+0000","last_clean":"2024-03-28T16:50:54.259946+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:54.259946+0000","last_undegraded":"2024-03-28T16:50:54.259946+0000","last_fullsized":"2024-03-28T16:50:54.259946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'260","last_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_deep_scrub":"31'260","last_deep_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.259667+0000","objects_scrubbed":10,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:21.748018+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":342,"num_read_kb":327,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.065783+0000","last_change":"2024-03-28T16:51:21.065839+0000","last_active":"2024-03-28T16:51:21.065783+0000","last_peered":"2024-03-28T16:51:21.065783+0000","last_clean":"2024-03-28T16:51:21.065783+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:51:21.065783+0000","last_undegraded":"2024-03-28T16:51:21.065783+0000","last_fullsized":"2024-03-28T16:51:21.065783+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.065552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:21.706323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'4","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.987139+0000","last_change":"2024-03-28T16:49:04.744802+0000","last_active":"2024-03-28T16:50:45.987139+0000","last_peered":"2024-03-28T16:50:45.987139+0000","last_clean":"2024-03-28T16:50:45.987139+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:50:45.987139+0000","last_undegraded":"2024-03-28T16:50:45.987139+0000","last_fullsized":"2024-03-28T16:50:45.987139+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532252+0000","last_change":"2024-03-28T16:48:56.689836+0000","last_active":"2024-03-28T16:50:45.532252+0000","last_peered":"2024-03-28T16:50:45.532252+0000","last_clean":"2024-03-28T16:50:45.532252+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:50:45.532252+0000","last_undegraded":"2024-03-28T16:50:45.532252+0000","last_fullsized":"2024-03-28T16:50:45.532252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.411785+0000","last_change":"2024-03-28T16:48:48.620790+0000","last_active":"2024-03-28T16:50:46.411785+0000","last_peered":"2024-03-28T16:50:46.411785+0000","last_clean":"2024-03-28T16:50:46.411785+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:50:46.411785+0000","last_undegraded":"2024-03-28T16:50:46.411785+0000","last_fullsized":"2024-03-28T16:50:46.411785+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531810+0000","last_change":"2024-03-28T16:48:54.663058+0000","last_active":"2024-03-28T16:50:45.531810+0000","last_peered":"2024-03-28T16:50:45.531810+0000","last_clean":"2024-03-28T16:50:45.531810+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:50:45.531810+0000","last_undegraded":"2024-03-28T16:50:45.531810+0000","last_fullsized":"2024-03-28T16:50:45.531810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.414766+0000","last_change":"2024-03-28T16:51:07.414766+0000","last_active":"2024-03-28T16:51:07.414766+0000","last_peered":"2024-03-28T16:51:07.414766+0000","last_clean":"2024-03-28T16:51:07.414766+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:51:07.414766+0000","last_undegraded":"2024-03-28T16:51:07.414766+0000","last_fullsized":"2024-03-28T16:51:07.414766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.414513+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:53.600318+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'23","reported_seq":82,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.409006+0000","last_change":"2024-03-28T16:50:54.409006+0000","last_active":"2024-03-28T16:50:54.409006+0000","last_peered":"2024-03-28T16:50:54.409006+0000","last_clean":"2024-03-28T16:50:54.409006+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:54.409006+0000","last_undegraded":"2024-03-28T16:50:54.409006+0000","last_fullsized":"2024-03-28T16:50:54.409006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'23","last_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_deep_scrub":"31'23","last_deep_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.408718+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:56.786363+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.341170+0000","last_change":"2024-03-28T16:51:01.341170+0000","last_active":"2024-03-28T16:51:01.341170+0000","last_peered":"2024-03-28T16:51:01.341170+0000","last_clean":"2024-03-28T16:51:01.341170+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:51:01.341170+0000","last_undegraded":"2024-03-28T16:51:01.341170+0000","last_fullsized":"2024-03-28T16:51:01.341170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.340925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:06.594781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874154+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:50:46.874154+0000","last_peered":"2024-03-28T16:50:46.874154+0000","last_clean":"2024-03-28T16:50:46.874154+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:50:46.874154+0000","last_undegraded":"2024-03-28T16:50:46.874154+0000","last_fullsized":"2024-03-28T16:50:46.874154+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532146+0000","last_change":"2024-03-28T16:48:56.683769+0000","last_active":"2024-03-28T16:50:45.532146+0000","last_peered":"2024-03-28T16:50:45.532146+0000","last_clean":"2024-03-28T16:50:45.532146+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:50:45.532146+0000","last_undegraded":"2024-03-28T16:50:45.532146+0000","last_fullsized":"2024-03-28T16:50:45.532146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870800+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:50:46.870800+0000","last_peered":"2024-03-28T16:50:46.870800+0000","last_clean":"2024-03-28T16:50:46.870800+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:50:46.870800+0000","last_undegraded":"2024-03-28T16:50:46.870800+0000","last_fullsized":"2024-03-28T16:50:46.870800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.385902+0000","last_change":"2024-03-28T16:51:19.386018+0000","last_active":"2024-03-28T16:51:19.385902+0000","last_peered":"2024-03-28T16:51:19.385902+0000","last_clean":"2024-03-28T16:51:19.385902+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:51:19.385902+0000","last_undegraded":"2024-03-28T16:51:19.385902+0000","last_fullsized":"2024-03-28T16:51:19.385902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.385614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:54.970167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.375959+0000","last_change":"2024-03-28T16:51:06.375959+0000","last_active":"2024-03-28T16:51:06.375959+0000","last_peered":"2024-03-28T16:51:06.375959+0000","last_clean":"2024-03-28T16:51:06.375959+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:51:06.375959+0000","last_undegraded":"2024-03-28T16:51:06.375959+0000","last_fullsized":"2024-03-28T16:51:06.375959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.375714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:32.910452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.195780+0000","last_change":"2024-03-28T16:50:51.195780+0000","last_active":"2024-03-28T16:50:51.195780+0000","last_peered":"2024-03-28T16:50:51.195780+0000","last_clean":"2024-03-28T16:50:51.195780+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:50:51.195780+0000","last_undegraded":"2024-03-28T16:50:51.195780+0000","last_fullsized":"2024-03-28T16:50:51.195780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'19","last_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_deep_scrub":"31'19","last_deep_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.195522+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:16.598840+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.873749+0000","last_change":"2024-03-28T16:49:02.741194+0000","last_active":"2024-03-28T16:50:46.873749+0000","last_peered":"2024-03-28T16:50:46.873749+0000","last_clean":"2024-03-28T16:50:46.873749+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:50:46.873749+0000","last_undegraded":"2024-03-28T16:50:46.873749+0000","last_fullsized":"2024-03-28T16:50:46.873749+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:01.680227+0000","last_clean_scrub_stamp":"2024-03-28T16:49:01.680227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.302529+0000","last_change":"2024-03-28T16:51:13.302529+0000","last_active":"2024-03-28T16:51:13.302529+0000","last_peered":"2024-03-28T16:51:13.302529+0000","last_clean":"2024-03-28T16:51:13.302529+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:51:13.302529+0000","last_undegraded":"2024-03-28T16:51:13.302529+0000","last_fullsized":"2024-03-28T16:51:13.302529+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.302200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:35.072558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":68,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.328005+0000","last_change":"2024-03-28T16:51:12.328075+0000","last_active":"2024-03-28T16:51:12.328005+0000","last_peered":"2024-03-28T16:51:12.328005+0000","last_clean":"2024-03-28T16:51:12.328005+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:51:12.328005+0000","last_undegraded":"2024-03-28T16:51:12.328005+0000","last_fullsized":"2024-03-28T16:51:12.328005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.327822+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:47.912375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.411817+0000","last_change":"2024-03-28T16:48:54.657105+0000","last_active":"2024-03-28T16:50:46.411817+0000","last_peered":"2024-03-28T16:50:46.411817+0000","last_clean":"2024-03-28T16:50:46.411817+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:50:46.411817+0000","last_undegraded":"2024-03-28T16:50:46.411817+0000","last_fullsized":"2024-03-28T16:50:46.411817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531684+0000","last_change":"2024-03-28T16:48:48.623086+0000","last_active":"2024-03-28T16:50:45.531684+0000","last_peered":"2024-03-28T16:50:45.531684+0000","last_clean":"2024-03-28T16:50:45.531684+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:50:45.531684+0000","last_undegraded":"2024-03-28T16:50:45.531684+0000","last_fullsized":"2024-03-28T16:50:45.531684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.355429+0000","last_change":"2024-03-28T16:51:03.355429+0000","last_active":"2024-03-28T16:51:03.355429+0000","last_peered":"2024-03-28T16:51:03.355429+0000","last_clean":"2024-03-28T16:51:03.355429+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:51:03.355429+0000","last_undegraded":"2024-03-28T16:51:03.355429+0000","last_fullsized":"2024-03-28T16:51:03.355429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.355165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:33.030302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'69","reported_seq":145,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.306941+0000","last_change":"2024-03-28T16:50:46.306941+0000","last_active":"2024-03-28T16:50:46.306941+0000","last_peered":"2024-03-28T16:50:46.306941+0000","last_clean":"2024-03-28T16:50:46.306941+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:46.306941+0000","last_undegraded":"2024-03-28T16:50:46.306941+0000","last_fullsized":"2024-03-28T16:50:46.306941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'69","last_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_deep_scrub":"31'69","last_deep_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.306710+0000","objects_scrubbed":8,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:28.604401+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.441291+0000","last_change":"2024-03-28T16:50:51.441291+0000","last_active":"2024-03-28T16:50:51.441291+0000","last_peered":"2024-03-28T16:50:51.441291+0000","last_clean":"2024-03-28T16:50:51.441291+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:50:51.441291+0000","last_undegraded":"2024-03-28T16:50:51.441291+0000","last_fullsized":"2024-03-28T16:50:51.441291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.441051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:13:01.864029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'9","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.533082+0000","last_change":"2024-03-28T16:49:04.741551+0000","last_active":"2024-03-28T16:50:45.533082+0000","last_peered":"2024-03-28T16:50:45.533082+0000","last_clean":"2024-03-28T16:50:45.533082+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:50:45.533082+0000","last_undegraded":"2024-03-28T16:50:45.533082+0000","last_fullsized":"2024-03-28T16:50:45.533082+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985239+0000","last_change":"2024-03-28T16:48:56.683849+0000","last_active":"2024-03-28T16:50:45.985239+0000","last_peered":"2024-03-28T16:50:45.985239+0000","last_clean":"2024-03-28T16:50:45.985239+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:50:45.985239+0000","last_undegraded":"2024-03-28T16:50:45.985239+0000","last_fullsized":"2024-03-28T16:50:45.985239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531753+0000","last_change":"2024-03-28T16:48:54.663076+0000","last_active":"2024-03-28T16:50:45.531753+0000","last_peered":"2024-03-28T16:50:45.531753+0000","last_clean":"2024-03-28T16:50:45.531753+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:50:45.531753+0000","last_undegraded":"2024-03-28T16:50:45.531753+0000","last_fullsized":"2024-03-28T16:50:45.531753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870790+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:50:46.870790+0000","last_peered":"2024-03-28T16:50:46.870790+0000","last_clean":"2024-03-28T16:50:46.870790+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:50:46.870790+0000","last_undegraded":"2024-03-28T16:50:46.870790+0000","last_fullsized":"2024-03-28T16:50:46.870790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.159039+0000","last_change":"2024-03-28T16:50:55.159039+0000","last_active":"2024-03-28T16:50:55.159039+0000","last_peered":"2024-03-28T16:50:55.159039+0000","last_clean":"2024-03-28T16:50:55.159039+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:50:55.159039+0000","last_undegraded":"2024-03-28T16:50:55.159039+0000","last_fullsized":"2024-03-28T16:50:55.159039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.158776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:53:10.542165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"31'21","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.252537+0000","last_change":"2024-03-28T16:50:52.252537+0000","last_active":"2024-03-28T16:50:52.252537+0000","last_peered":"2024-03-28T16:50:52.252537+0000","last_clean":"2024-03-28T16:50:52.252537+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:50:52.252537+0000","last_undegraded":"2024-03-28T16:50:52.252537+0000","last_fullsized":"2024-03-28T16:50:52.252537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'21","last_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_deep_scrub":"31'21","last_deep_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.252240+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:47.715342+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'249","reported_seq":304,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.104945+0000","last_change":"2024-03-28T16:50:46.104945+0000","last_active":"2024-03-28T16:50:46.104945+0000","last_peered":"2024-03-28T16:50:46.104945+0000","last_clean":"2024-03-28T16:50:46.104945+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:46.104945+0000","last_undegraded":"2024-03-28T16:50:46.104945+0000","last_fullsized":"2024-03-28T16:50:46.104945+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.104655+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:39.533970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874227+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:46.874227+0000","last_peered":"2024-03-28T16:50:46.874227+0000","last_clean":"2024-03-28T16:50:46.874227+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:46.874227+0000","last_undegraded":"2024-03-28T16:50:46.874227+0000","last_fullsized":"2024-03-28T16:50:46.874227+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.222398+0000","last_change":"2024-03-28T16:51:04.222527+0000","last_active":"2024-03-28T16:51:04.222398+0000","last_peered":"2024-03-28T16:51:04.222398+0000","last_clean":"2024-03-28T16:51:04.222398+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:51:04.222398+0000","last_undegraded":"2024-03-28T16:51:04.222398+0000","last_fullsized":"2024-03-28T16:51:04.222398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.222151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:13.382250+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870972+0000","last_change":"2024-03-28T16:48:54.662217+0000","last_active":"2024-03-28T16:50:46.870972+0000","last_peered":"2024-03-28T16:50:46.870972+0000","last_clean":"2024-03-28T16:50:46.870972+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:50:46.870972+0000","last_undegraded":"2024-03-28T16:50:46.870972+0000","last_fullsized":"2024-03-28T16:50:46.870972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531684+0000","last_change":"2024-03-28T16:48:48.622190+0000","last_active":"2024-03-28T16:50:45.531684+0000","last_peered":"2024-03-28T16:50:45.531684+0000","last_clean":"2024-03-28T16:50:45.531684+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:50:45.531684+0000","last_undegraded":"2024-03-28T16:50:45.531684+0000","last_fullsized":"2024-03-28T16:50:45.531684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.163965+0000","last_change":"2024-03-28T16:50:54.163965+0000","last_active":"2024-03-28T16:50:54.163965+0000","last_peered":"2024-03-28T16:50:54.163965+0000","last_clean":"2024-03-28T16:50:54.163965+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:50:54.163965+0000","last_undegraded":"2024-03-28T16:50:54.163965+0000","last_fullsized":"2024-03-28T16:50:54.163965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.163592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:27.253015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.038093+0000","last_change":"2024-03-28T16:51:16.038179+0000","last_active":"2024-03-28T16:51:16.038093+0000","last_peered":"2024-03-28T16:51:16.038093+0000","last_clean":"2024-03-28T16:51:16.038093+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:51:16.038093+0000","last_undegraded":"2024-03-28T16:51:16.038093+0000","last_fullsized":"2024-03-28T16:51:16.038093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.037881+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:44.064566+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'249","reported_seq":337,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.233041+0000","last_change":"2024-03-28T16:50:49.233041+0000","last_active":"2024-03-28T16:50:49.233041+0000","last_peered":"2024-03-28T16:50:49.233041+0000","last_clean":"2024-03-28T16:50:49.233041+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:49.233041+0000","last_undegraded":"2024-03-28T16:50:49.233041+0000","last_fullsized":"2024-03-28T16:50:49.233041+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.232798+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:57.227518+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.267309+0000","last_change":"2024-03-28T16:51:09.267309+0000","last_active":"2024-03-28T16:51:09.267309+0000","last_peered":"2024-03-28T16:51:09.267309+0000","last_clean":"2024-03-28T16:51:09.267309+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:51:09.267309+0000","last_undegraded":"2024-03-28T16:51:09.267309+0000","last_fullsized":"2024-03-28T16:51:09.267309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.267056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:58.701007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.363436+0000","last_change":"2024-03-28T16:51:16.363501+0000","last_active":"2024-03-28T16:51:16.363436+0000","last_peered":"2024-03-28T16:51:16.363436+0000","last_clean":"2024-03-28T16:51:16.363436+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:51:16.363436+0000","last_undegraded":"2024-03-28T16:51:16.363436+0000","last_fullsized":"2024-03-28T16:51:16.363436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.363254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:07:05.797204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.287919+0000","last_change":"2024-03-28T16:51:20.287986+0000","last_active":"2024-03-28T16:51:20.287919+0000","last_peered":"2024-03-28T16:51:20.287919+0000","last_clean":"2024-03-28T16:51:20.287919+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:51:20.287919+0000","last_undegraded":"2024-03-28T16:51:20.287919+0000","last_fullsized":"2024-03-28T16:51:20.287919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.287729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:53.368108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870766+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:50:46.870766+0000","last_peered":"2024-03-28T16:50:46.870766+0000","last_clean":"2024-03-28T16:50:46.870766+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:50:46.870766+0000","last_undegraded":"2024-03-28T16:50:46.870766+0000","last_fullsized":"2024-03-28T16:50:46.870766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.166685+0000","last_change":"2024-03-28T16:50:56.166685+0000","last_active":"2024-03-28T16:50:56.166685+0000","last_peered":"2024-03-28T16:50:56.166685+0000","last_clean":"2024-03-28T16:50:56.166685+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:50:56.166685+0000","last_undegraded":"2024-03-28T16:50:56.166685+0000","last_fullsized":"2024-03-28T16:50:56.166685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.166428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:06.567829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'90","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.384285+0000","last_change":"2024-03-28T16:50:58.384285+0000","last_active":"2024-03-28T16:50:58.384285+0000","last_peered":"2024-03-28T16:50:58.384285+0000","last_clean":"2024-03-28T16:50:58.384285+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:58.384285+0000","last_undegraded":"2024-03-28T16:50:58.384285+0000","last_fullsized":"2024-03-28T16:50:58.384285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'90","last_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_deep_scrub":"31'90","last_deep_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.383995+0000","objects_scrubbed":8,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:54.487031+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.285325+0000","last_change":"2024-03-28T16:50:47.285325+0000","last_active":"2024-03-28T16:50:47.285325+0000","last_peered":"2024-03-28T16:50:47.285325+0000","last_clean":"2024-03-28T16:50:47.285325+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:50:47.285325+0000","last_undegraded":"2024-03-28T16:50:47.285325+0000","last_fullsized":"2024-03-28T16:50:47.285325+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.285202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:46:10.170506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.414097+0000","last_change":"2024-03-28T16:49:04.754190+0000","last_active":"2024-03-28T16:50:46.414097+0000","last_peered":"2024-03-28T16:50:46.414097+0000","last_clean":"2024-03-28T16:50:46.414097+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:50:46.414097+0000","last_undegraded":"2024-03-28T16:50:46.414097+0000","last_fullsized":"2024-03-28T16:50:46.414097+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.344614+0000","last_change":"2024-03-28T16:51:00.344614+0000","last_active":"2024-03-28T16:51:00.344614+0000","last_peered":"2024-03-28T16:51:00.344614+0000","last_clean":"2024-03-28T16:51:00.344614+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:51:00.344614+0000","last_undegraded":"2024-03-28T16:51:00.344614+0000","last_fullsized":"2024-03-28T16:51:00.344614+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.344363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:57.233454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.364821+0000","last_change":"2024-03-28T16:51:10.364821+0000","last_active":"2024-03-28T16:51:10.364821+0000","last_peered":"2024-03-28T16:51:10.364821+0000","last_clean":"2024-03-28T16:51:10.364821+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:51:10.364821+0000","last_undegraded":"2024-03-28T16:51:10.364821+0000","last_fullsized":"2024-03-28T16:51:10.364821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.364556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:15.618413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871399+0000","last_change":"2024-03-28T16:48:54.663088+0000","last_active":"2024-03-28T16:50:46.871399+0000","last_peered":"2024-03-28T16:50:46.871399+0000","last_clean":"2024-03-28T16:50:46.871399+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:50:46.871399+0000","last_undegraded":"2024-03-28T16:50:46.871399+0000","last_fullsized":"2024-03-28T16:50:46.871399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.986181+0000","last_change":"2024-03-28T16:49:37.606876+0000","last_active":"2024-03-28T16:50:45.986181+0000","last_peered":"2024-03-28T16:50:45.986181+0000","last_clean":"2024-03-28T16:50:45.986181+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:50:45.986181+0000","last_undegraded":"2024-03-28T16:50:45.986181+0000","last_fullsized":"2024-03-28T16:50:45.986181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.084532+0000","last_change":"2024-03-28T16:51:15.084532+0000","last_active":"2024-03-28T16:51:15.084532+0000","last_peered":"2024-03-28T16:51:15.084532+0000","last_clean":"2024-03-28T16:51:15.084532+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:51:15.084532+0000","last_undegraded":"2024-03-28T16:51:15.084532+0000","last_fullsized":"2024-03-28T16:51:15.084532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.084235+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:01:39.553948+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.246294+0000","last_change":"2024-03-28T16:51:09.246355+0000","last_active":"2024-03-28T16:51:09.246294+0000","last_peered":"2024-03-28T16:51:09.246294+0000","last_clean":"2024-03-28T16:51:09.246294+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:51:09.246294+0000","last_undegraded":"2024-03-28T16:51:09.246294+0000","last_fullsized":"2024-03-28T16:51:09.246294+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.246098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:55:30.745667+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.442625+0000","last_change":"2024-03-28T16:51:17.442696+0000","last_active":"2024-03-28T16:51:17.442625+0000","last_peered":"2024-03-28T16:51:17.442625+0000","last_clean":"2024-03-28T16:51:17.442625+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:51:17.442625+0000","last_undegraded":"2024-03-28T16:51:17.442625+0000","last_fullsized":"2024-03-28T16:51:17.442625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.442424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:34.861019+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985435+0000","last_change":"2024-03-28T16:49:37.603811+0000","last_active":"2024-03-28T16:50:45.985435+0000","last_peered":"2024-03-28T16:50:45.985435+0000","last_clean":"2024-03-28T16:50:45.985435+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:50:45.985435+0000","last_undegraded":"2024-03-28T16:50:45.985435+0000","last_fullsized":"2024-03-28T16:50:45.985435+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.411962+0000","last_change":"2024-03-28T16:48:54.657586+0000","last_active":"2024-03-28T16:50:46.411962+0000","last_peered":"2024-03-28T16:50:46.411962+0000","last_clean":"2024-03-28T16:50:46.411962+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:50:46.411962+0000","last_undegraded":"2024-03-28T16:50:46.411962+0000","last_fullsized":"2024-03-28T16:50:46.411962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.178814+0000","last_change":"2024-03-28T16:50:52.178814+0000","last_active":"2024-03-28T16:50:52.178814+0000","last_peered":"2024-03-28T16:50:52.178814+0000","last_clean":"2024-03-28T16:50:52.178814+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:50:52.178814+0000","last_undegraded":"2024-03-28T16:50:52.178814+0000","last_fullsized":"2024-03-28T16:50:52.178814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.178561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:00.951220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.353835+0000","last_change":"2024-03-28T16:51:00.353835+0000","last_active":"2024-03-28T16:51:00.353835+0000","last_peered":"2024-03-28T16:51:00.353835+0000","last_clean":"2024-03-28T16:51:00.353835+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:51:00.353835+0000","last_undegraded":"2024-03-28T16:51:00.353835+0000","last_fullsized":"2024-03-28T16:51:00.353835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.353557+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:18.400713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.443394+0000","last_change":"2024-03-28T16:50:49.443394+0000","last_active":"2024-03-28T16:50:49.443394+0000","last_peered":"2024-03-28T16:50:49.443394+0000","last_clean":"2024-03-28T16:50:49.443394+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:50:49.443394+0000","last_undegraded":"2024-03-28T16:50:49.443394+0000","last_fullsized":"2024-03-28T16:50:49.443394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.443035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:33:21.896718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.586273+0000","last_change":"2024-03-28T16:49:04.744040+0000","last_active":"2024-03-28T16:49:37.586273+0000","last_peered":"2024-03-28T16:49:37.586273+0000","last_clean":"2024-03-28T16:49:37.586273+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:49:37.586273+0000","last_undegraded":"2024-03-28T16:49:37.586273+0000","last_fullsized":"2024-03-28T16:49:37.586273+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.486418+0000","last_change":"2024-03-28T16:51:19.486556+0000","last_active":"2024-03-28T16:51:19.486418+0000","last_peered":"2024-03-28T16:51:19.486418+0000","last_clean":"2024-03-28T16:51:19.486418+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:51:19.486418+0000","last_undegraded":"2024-03-28T16:51:19.486418+0000","last_fullsized":"2024-03-28T16:51:19.486418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.486207+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:42.867614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412016+0000","last_change":"2024-03-28T16:48:54.659093+0000","last_active":"2024-03-28T16:50:46.412016+0000","last_peered":"2024-03-28T16:50:46.412016+0000","last_clean":"2024-03-28T16:50:46.412016+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:50:46.412016+0000","last_undegraded":"2024-03-28T16:50:46.412016+0000","last_fullsized":"2024-03-28T16:50:46.412016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":209,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.379662+0000","last_change":"2024-03-28T16:51:06.379662+0000","last_active":"2024-03-28T16:51:06.379662+0000","last_peered":"2024-03-28T16:51:06.379662+0000","last_clean":"2024-03-28T16:51:06.379662+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:51:06.379662+0000","last_undegraded":"2024-03-28T16:51:06.379662+0000","last_fullsized":"2024-03-28T16:51:06.379662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.379283+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:41.522575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":256,"num_read_kb":255,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'42","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.233580+0000","last_change":"2024-03-28T16:50:55.233580+0000","last_active":"2024-03-28T16:50:55.233580+0000","last_peered":"2024-03-28T16:50:55.233580+0000","last_clean":"2024-03-28T16:50:55.233580+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:55.233580+0000","last_undegraded":"2024-03-28T16:50:55.233580+0000","last_fullsized":"2024-03-28T16:50:55.233580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'42","last_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_deep_scrub":"31'42","last_deep_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.233264+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:43.214573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":56,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.129943+0000","last_change":"2024-03-28T16:50:47.129943+0000","last_active":"2024-03-28T16:50:47.129943+0000","last_peered":"2024-03-28T16:50:47.129943+0000","last_clean":"2024-03-28T16:50:47.129943+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:50:47.129943+0000","last_undegraded":"2024-03-28T16:50:47.129943+0000","last_fullsized":"2024-03-28T16:50:47.129943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.129698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:49.507343+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582135+0000","last_change":"2024-03-28T16:49:04.752570+0000","last_active":"2024-03-28T16:49:37.582135+0000","last_peered":"2024-03-28T16:49:37.582135+0000","last_clean":"2024-03-28T16:49:37.582135+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:49:37.582135+0000","last_undegraded":"2024-03-28T16:49:37.582135+0000","last_fullsized":"2024-03-28T16:49:37.582135+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872113+0000","last_change":"2024-03-28T16:48:56.683249+0000","last_active":"2024-03-28T16:50:46.872113+0000","last_peered":"2024-03-28T16:50:46.872113+0000","last_clean":"2024-03-28T16:50:46.872113+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:50:46.872113+0000","last_undegraded":"2024-03-28T16:50:46.872113+0000","last_fullsized":"2024-03-28T16:50:46.872113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984599+0000","last_change":"2024-03-28T16:48:54.674928+0000","last_active":"2024-03-28T16:50:45.984599+0000","last_peered":"2024-03-28T16:50:45.984599+0000","last_clean":"2024-03-28T16:50:45.984599+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:50:45.984599+0000","last_undegraded":"2024-03-28T16:50:45.984599+0000","last_fullsized":"2024-03-28T16:50:45.984599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.166465+0000","last_change":"2024-03-28T16:50:58.166465+0000","last_active":"2024-03-28T16:50:58.166465+0000","last_peered":"2024-03-28T16:50:58.166465+0000","last_clean":"2024-03-28T16:50:58.166465+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:50:58.166465+0000","last_undegraded":"2024-03-28T16:50:58.166465+0000","last_fullsized":"2024-03-28T16:50:58.166465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.166221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:21.547629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.370703+0000","last_change":"2024-03-28T16:51:02.370703+0000","last_active":"2024-03-28T16:51:02.370703+0000","last_peered":"2024-03-28T16:51:02.370703+0000","last_clean":"2024-03-28T16:51:02.370703+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:51:02.370703+0000","last_undegraded":"2024-03-28T16:51:02.370703+0000","last_fullsized":"2024-03-28T16:51:02.370703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.370450+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:15.038783+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.448158+0000","last_change":"2024-03-28T16:50:50.448158+0000","last_active":"2024-03-28T16:50:50.448158+0000","last_peered":"2024-03-28T16:50:50.448158+0000","last_clean":"2024-03-28T16:50:50.448158+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:50:50.448158+0000","last_undegraded":"2024-03-28T16:50:50.448158+0000","last_fullsized":"2024-03-28T16:50:50.448158+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.447911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:45.521899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872459+0000","last_change":"2024-03-28T16:48:56.675359+0000","last_active":"2024-03-28T16:50:46.872459+0000","last_peered":"2024-03-28T16:50:46.872459+0000","last_clean":"2024-03-28T16:50:46.872459+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:50:46.872459+0000","last_undegraded":"2024-03-28T16:50:46.872459+0000","last_fullsized":"2024-03-28T16:50:46.872459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:41.566962+0000","last_change":"2024-03-28T16:49:37.600567+0000","last_active":"2024-03-28T16:49:41.566962+0000","last_peered":"2024-03-28T16:49:41.566962+0000","last_clean":"2024-03-28T16:49:41.566962+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:49:41.566962+0000","last_undegraded":"2024-03-28T16:49:41.566962+0000","last_fullsized":"2024-03-28T16:49:41.566962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.369809+0000","last_change":"2024-03-28T16:51:07.369837+0000","last_active":"2024-03-28T16:51:07.369809+0000","last_peered":"2024-03-28T16:51:07.369809+0000","last_clean":"2024-03-28T16:51:07.369809+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:51:07.369809+0000","last_undegraded":"2024-03-28T16:51:07.369809+0000","last_fullsized":"2024-03-28T16:51:07.369809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.369707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:56.803658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'60","reported_seq":132,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.252481+0000","last_change":"2024-03-28T16:50:56.252559+0000","last_active":"2024-03-28T16:50:56.252481+0000","last_peered":"2024-03-28T16:50:56.252481+0000","last_clean":"2024-03-28T16:50:56.252481+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:56.252481+0000","last_undegraded":"2024-03-28T16:50:56.252481+0000","last_fullsized":"2024-03-28T16:50:56.252481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'60","last_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_deep_scrub":"31'60","last_deep_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.252305+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:41.009336+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.444508+0000","last_change":"2024-03-28T16:50:48.444559+0000","last_active":"2024-03-28T16:50:48.444508+0000","last_peered":"2024-03-28T16:50:48.444508+0000","last_clean":"2024-03-28T16:50:48.444508+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:50:48.444508+0000","last_undegraded":"2024-03-28T16:50:48.444508+0000","last_fullsized":"2024-03-28T16:50:48.444508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.444297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:24:42.861356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'11","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:30.216050+0000","last_change":"2024-03-28T16:49:04.752525+0000","last_active":"2024-03-28T16:50:30.216050+0000","last_peered":"2024-03-28T16:50:30.216050+0000","last_clean":"2024-03-28T16:50:30.216050+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:50:30.216050+0000","last_undegraded":"2024-03-28T16:50:30.216050+0000","last_fullsized":"2024-03-28T16:50:30.216050+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.399302+0000","last_change":"2024-03-28T16:51:15.399354+0000","last_active":"2024-03-28T16:51:15.399302+0000","last_peered":"2024-03-28T16:51:15.399302+0000","last_clean":"2024-03-28T16:51:15.399302+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:51:15.399302+0000","last_undegraded":"2024-03-28T16:51:15.399302+0000","last_fullsized":"2024-03-28T16:51:15.399302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.399113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:12.560237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984911+0000","last_change":"2024-03-28T16:48:54.668040+0000","last_active":"2024-03-28T16:50:45.984911+0000","last_peered":"2024-03-28T16:50:45.984911+0000","last_clean":"2024-03-28T16:50:45.984911+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:50:45.984911+0000","last_undegraded":"2024-03-28T16:50:45.984911+0000","last_fullsized":"2024-03-28T16:50:45.984911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.409622+0000","last_change":"2024-03-28T16:51:09.409674+0000","last_active":"2024-03-28T16:51:09.409622+0000","last_peered":"2024-03-28T16:51:09.409622+0000","last_clean":"2024-03-28T16:51:09.409622+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:51:09.409622+0000","last_undegraded":"2024-03-28T16:51:09.409622+0000","last_fullsized":"2024-03-28T16:51:09.409622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.409427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:27.456583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'43","reported_seq":100,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.182779+0000","last_change":"2024-03-28T16:50:57.182861+0000","last_active":"2024-03-28T16:50:57.182779+0000","last_peered":"2024-03-28T16:50:57.182779+0000","last_clean":"2024-03-28T16:50:57.182779+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:57.182779+0000","last_undegraded":"2024-03-28T16:50:57.182779+0000","last_fullsized":"2024-03-28T16:50:57.182779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.182585+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.262964+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.238718+0000","last_change":"2024-03-28T16:50:53.238776+0000","last_active":"2024-03-28T16:50:53.238718+0000","last_peered":"2024-03-28T16:50:53.238718+0000","last_clean":"2024-03-28T16:50:53.238718+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:50:53.238718+0000","last_undegraded":"2024-03-28T16:50:53.238718+0000","last_fullsized":"2024-03-28T16:50:53.238718+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.238528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:50.127619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.567266+0000","last_change":"2024-03-28T16:49:04.754981+0000","last_active":"2024-03-28T16:49:40.567266+0000","last_peered":"2024-03-28T16:49:40.567266+0000","last_clean":"2024-03-28T16:49:40.567266+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:49:40.567266+0000","last_undegraded":"2024-03-28T16:49:40.567266+0000","last_fullsized":"2024-03-28T16:49:40.567266+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.175450+0000","last_change":"2024-03-28T16:51:02.175511+0000","last_active":"2024-03-28T16:51:02.175450+0000","last_peered":"2024-03-28T16:51:02.175450+0000","last_clean":"2024-03-28T16:51:02.175450+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:51:02.175450+0000","last_undegraded":"2024-03-28T16:51:02.175450+0000","last_fullsized":"2024-03-28T16:51:02.175450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.175255+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:28.260113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984751+0000","last_change":"2024-03-28T16:48:54.654527+0000","last_active":"2024-03-28T16:50:45.984751+0000","last_peered":"2024-03-28T16:50:45.984751+0000","last_clean":"2024-03-28T16:50:45.984751+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:50:45.984751+0000","last_undegraded":"2024-03-28T16:50:45.984751+0000","last_fullsized":"2024-03-28T16:50:45.984751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.612252+0000","last_change":"2024-03-28T16:49:37.610755+0000","last_active":"2024-03-28T16:49:37.612252+0000","last_peered":"2024-03-28T16:49:37.612252+0000","last_clean":"2024-03-28T16:49:37.612252+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:49:37.612252+0000","last_undegraded":"2024-03-28T16:49:37.612252+0000","last_fullsized":"2024-03-28T16:49:37.612252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.342055+0000","last_change":"2024-03-28T16:50:58.342055+0000","last_active":"2024-03-28T16:50:58.342055+0000","last_peered":"2024-03-28T16:50:58.342055+0000","last_clean":"2024-03-28T16:50:58.342055+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:58.342055+0000","last_undegraded":"2024-03-28T16:50:58.342055+0000","last_fullsized":"2024-03-28T16:50:58.342055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.341815+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:35.862752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.375363+0000","last_change":"2024-03-28T16:51:05.375363+0000","last_active":"2024-03-28T16:51:05.375363+0000","last_peered":"2024-03-28T16:51:05.375363+0000","last_clean":"2024-03-28T16:51:05.375363+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:51:05.375363+0000","last_undegraded":"2024-03-28T16:51:05.375363+0000","last_fullsized":"2024-03-28T16:51:05.375363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.375113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:40.169672+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:39.874641+0000","last_change":"2024-03-28T16:49:04.748086+0000","last_active":"2024-03-28T16:49:39.874641+0000","last_peered":"2024-03-28T16:49:39.874641+0000","last_clean":"2024-03-28T16:49:39.874641+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:49:39.874641+0000","last_undegraded":"2024-03-28T16:49:39.874641+0000","last_fullsized":"2024-03-28T16:49:39.874641+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'3","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:35.469109+0000","last_change":"2024-03-28T16:48:56.682850+0000","last_active":"2024-03-28T16:50:35.469109+0000","last_peered":"2024-03-28T16:50:35.469109+0000","last_clean":"2024-03-28T16:50:35.469109+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:50:35.469109+0000","last_undegraded":"2024-03-28T16:50:35.469109+0000","last_fullsized":"2024-03-28T16:50:35.469109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.592742+0000","last_change":"2024-03-28T16:48:54.674069+0000","last_active":"2024-03-28T16:49:37.592742+0000","last_peered":"2024-03-28T16:49:37.592742+0000","last_clean":"2024-03-28T16:49:37.592742+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:49:37.592742+0000","last_undegraded":"2024-03-28T16:49:37.592742+0000","last_fullsized":"2024-03-28T16:49:37.592742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.303927+0000","last_change":"2024-03-28T16:51:04.303980+0000","last_active":"2024-03-28T16:51:04.303927+0000","last_peered":"2024-03-28T16:51:04.303927+0000","last_clean":"2024-03-28T16:51:04.303927+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:51:04.303927+0000","last_undegraded":"2024-03-28T16:51:04.303927+0000","last_fullsized":"2024-03-28T16:51:04.303927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.303737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:22.080932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.229100+0000","last_change":"2024-03-28T16:51:01.229100+0000","last_active":"2024-03-28T16:51:01.229100+0000","last_peered":"2024-03-28T16:51:01.229100+0000","last_clean":"2024-03-28T16:51:01.229100+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:51:01.229100+0000","last_undegraded":"2024-03-28T16:51:01.229100+0000","last_fullsized":"2024-03-28T16:51:01.229100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.228821+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:13.034030+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'249","reported_seq":310,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.440393+0000","last_change":"2024-03-28T16:50:52.440393+0000","last_active":"2024-03-28T16:50:52.440393+0000","last_peered":"2024-03-28T16:50:52.440393+0000","last_clean":"2024-03-28T16:50:52.440393+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:52.440393+0000","last_undegraded":"2024-03-28T16:50:52.440393+0000","last_fullsized":"2024-03-28T16:50:52.440393+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.440130+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:54:49.215852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.580073+0000","last_change":"2024-03-28T16:49:04.748224+0000","last_active":"2024-03-28T16:49:37.580073+0000","last_peered":"2024-03-28T16:49:37.580073+0000","last_clean":"2024-03-28T16:49:37.580073+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:49:37.580073+0000","last_undegraded":"2024-03-28T16:49:37.580073+0000","last_fullsized":"2024-03-28T16:49:37.580073+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'6","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.351983+0000","last_change":"2024-03-28T16:51:20.352050+0000","last_active":"2024-03-28T16:51:20.351983+0000","last_peered":"2024-03-28T16:51:20.351983+0000","last_clean":"2024-03-28T16:51:20.351983+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:51:20.351983+0000","last_undegraded":"2024-03-28T16:51:20.351983+0000","last_fullsized":"2024-03-28T16:51:20.351983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.351803+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:25.271104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984815+0000","last_change":"2024-03-28T16:48:54.667648+0000","last_active":"2024-03-28T16:50:45.984815+0000","last_peered":"2024-03-28T16:50:45.984815+0000","last_clean":"2024-03-28T16:50:45.984815+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:50:45.984815+0000","last_undegraded":"2024-03-28T16:50:45.984815+0000","last_fullsized":"2024-03-28T16:50:45.984815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.331599+0000","last_change":"2024-03-28T16:51:02.331654+0000","last_active":"2024-03-28T16:51:02.331599+0000","last_peered":"2024-03-28T16:51:02.331599+0000","last_clean":"2024-03-28T16:51:02.331599+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:51:02.331599+0000","last_undegraded":"2024-03-28T16:51:02.331599+0000","last_fullsized":"2024-03-28T16:51:02.331599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.331416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:20.378572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'43","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.202390+0000","last_change":"2024-03-28T16:50:50.202390+0000","last_active":"2024-03-28T16:50:50.202390+0000","last_peered":"2024-03-28T16:50:50.202390+0000","last_clean":"2024-03-28T16:50:50.202390+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:50.202390+0000","last_undegraded":"2024-03-28T16:50:50.202390+0000","last_fullsized":"2024-03-28T16:50:50.202390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.202141+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.086676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.276885+0000","last_change":"2024-03-28T16:50:48.276938+0000","last_active":"2024-03-28T16:50:48.276885+0000","last_peered":"2024-03-28T16:50:48.276885+0000","last_clean":"2024-03-28T16:50:48.276885+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:50:48.276885+0000","last_undegraded":"2024-03-28T16:50:48.276885+0000","last_fullsized":"2024-03-28T16:50:48.276885+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.276698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:29.219774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.601299+0000","last_change":"2024-03-28T16:49:04.745332+0000","last_active":"2024-03-28T16:49:37.601299+0000","last_peered":"2024-03-28T16:49:37.601299+0000","last_clean":"2024-03-28T16:49:37.601299+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:49:37.601299+0000","last_undegraded":"2024-03-28T16:49:37.601299+0000","last_fullsized":"2024-03-28T16:49:37.601299+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'4","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.310199+0000","last_change":"2024-03-28T16:51:24.310265+0000","last_active":"2024-03-28T16:51:24.310199+0000","last_peered":"2024-03-28T16:51:24.310199+0000","last_clean":"2024-03-28T16:51:24.310199+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:51:24.310199+0000","last_undegraded":"2024-03-28T16:51:24.310199+0000","last_fullsized":"2024-03-28T16:51:24.310199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.310025+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:21.471148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.588778+0000","last_change":"2024-03-28T16:48:54.669871+0000","last_active":"2024-03-28T16:49:37.588778+0000","last_peered":"2024-03-28T16:49:37.588778+0000","last_clean":"2024-03-28T16:49:37.588778+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:49:37.588778+0000","last_undegraded":"2024-03-28T16:49:37.588778+0000","last_fullsized":"2024-03-28T16:49:37.588778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.396440+0000","last_change":"2024-03-28T16:51:03.396495+0000","last_active":"2024-03-28T16:51:03.396440+0000","last_peered":"2024-03-28T16:51:03.396440+0000","last_clean":"2024-03-28T16:51:03.396440+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:51:03.396440+0000","last_undegraded":"2024-03-28T16:51:03.396440+0000","last_fullsized":"2024-03-28T16:51:03.396440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.396253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:26.295446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'34","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.307684+0000","last_change":"2024-03-28T16:50:59.307759+0000","last_active":"2024-03-28T16:50:59.307684+0000","last_peered":"2024-03-28T16:50:59.307684+0000","last_clean":"2024-03-28T16:50:59.307684+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:59.307684+0000","last_undegraded":"2024-03-28T16:50:59.307684+0000","last_fullsized":"2024-03-28T16:50:59.307684+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.307494+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:28.146893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.090758+0000","last_change":"2024-03-28T16:51:13.090891+0000","last_active":"2024-03-28T16:51:13.090758+0000","last_peered":"2024-03-28T16:51:13.090758+0000","last_clean":"2024-03-28T16:51:13.090758+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:51:13.090758+0000","last_undegraded":"2024-03-28T16:51:13.090758+0000","last_fullsized":"2024-03-28T16:51:13.090758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.090537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:44:11.869557+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582903+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:49:37.582903+0000","last_peered":"2024-03-28T16:49:37.582903+0000","last_clean":"2024-03-28T16:49:37.582903+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:49:37.582903+0000","last_undegraded":"2024-03-28T16:49:37.582903+0000","last_fullsized":"2024-03-28T16:49:37.582903+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'1","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872384+0000","last_change":"2024-03-28T16:48:56.666104+0000","last_active":"2024-03-28T16:50:46.872384+0000","last_peered":"2024-03-28T16:50:46.872384+0000","last_clean":"2024-03-28T16:50:46.872384+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:50:46.872384+0000","last_undegraded":"2024-03-28T16:50:46.872384+0000","last_fullsized":"2024-03-28T16:50:46.872384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531809+0000","last_change":"2024-03-28T16:48:54.665975+0000","last_active":"2024-03-28T16:50:45.531809+0000","last_peered":"2024-03-28T16:50:45.531809+0000","last_clean":"2024-03-28T16:50:45.531809+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:50:45.531809+0000","last_undegraded":"2024-03-28T16:50:45.531809+0000","last_fullsized":"2024-03-28T16:50:45.531809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.256599+0000","last_change":"2024-03-28T16:50:59.256599+0000","last_active":"2024-03-28T16:50:59.256599+0000","last_peered":"2024-03-28T16:50:59.256599+0000","last_clean":"2024-03-28T16:50:59.256599+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:50:59.256599+0000","last_undegraded":"2024-03-28T16:50:59.256599+0000","last_fullsized":"2024-03-28T16:50:59.256599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.256359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:34.399651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.213370+0000","last_change":"2024-03-28T16:50:59.213439+0000","last_active":"2024-03-28T16:50:59.213370+0000","last_peered":"2024-03-28T16:50:59.213370+0000","last_clean":"2024-03-28T16:50:59.213370+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:59.213370+0000","last_undegraded":"2024-03-28T16:50:59.213370+0000","last_fullsized":"2024-03-28T16:50:59.213370+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'28","last_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_deep_scrub":"31'28","last_deep_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.213187+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:38.889215+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.080247+0000","last_change":"2024-03-28T16:51:17.080305+0000","last_active":"2024-03-28T16:51:17.080247+0000","last_peered":"2024-03-28T16:51:17.080247+0000","last_clean":"2024-03-28T16:51:17.080247+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:51:17.080247+0000","last_undegraded":"2024-03-28T16:51:17.080247+0000","last_fullsized":"2024-03-28T16:51:17.080247+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.080049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:12.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.587298+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:49:37.587298+0000","last_peered":"2024-03-28T16:49:37.587298+0000","last_clean":"2024-03-28T16:49:37.587298+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:49:37.587298+0000","last_undegraded":"2024-03-28T16:49:37.587298+0000","last_fullsized":"2024-03-28T16:49:37.587298+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'12","reported_seq":66,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.347902+0000","last_change":"2024-03-28T16:51:13.347902+0000","last_active":"2024-03-28T16:51:13.347902+0000","last_peered":"2024-03-28T16:51:13.347902+0000","last_clean":"2024-03-28T16:51:13.347902+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:51:13.347902+0000","last_undegraded":"2024-03-28T16:51:13.347902+0000","last_fullsized":"2024-03-28T16:51:13.347902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.347663+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:39.911911+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984783+0000","last_change":"2024-03-28T16:48:54.655009+0000","last_active":"2024-03-28T16:50:45.984783+0000","last_peered":"2024-03-28T16:50:45.984783+0000","last_clean":"2024-03-28T16:50:45.984783+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:50:45.984783+0000","last_undegraded":"2024-03-28T16:50:45.984783+0000","last_fullsized":"2024-03-28T16:50:45.984783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.215895+0000","last_change":"2024-03-28T16:50:58.215895+0000","last_active":"2024-03-28T16:50:58.215895+0000","last_peered":"2024-03-28T16:50:58.215895+0000","last_clean":"2024-03-28T16:50:58.215895+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:50:58.215895+0000","last_undegraded":"2024-03-28T16:50:58.215895+0000","last_fullsized":"2024-03-28T16:50:58.215895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.215651+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:42.972682+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'35","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.271980+0000","last_change":"2024-03-28T16:51:05.272061+0000","last_active":"2024-03-28T16:51:05.271980+0000","last_peered":"2024-03-28T16:51:05.271980+0000","last_clean":"2024-03-28T16:51:05.271980+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:51:05.271980+0000","last_undegraded":"2024-03-28T16:51:05.271980+0000","last_fullsized":"2024-03-28T16:51:05.271980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'35","last_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_deep_scrub":"31'35","last_deep_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.271785+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:01.679261+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1150,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.209217+0000","last_change":"2024-03-28T16:50:50.209342+0000","last_active":"2024-03-28T16:50:50.209217+0000","last_peered":"2024-03-28T16:50:50.209217+0000","last_clean":"2024-03-28T16:50:50.209217+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:50:50.209217+0000","last_undegraded":"2024-03-28T16:50:50.209217+0000","last_fullsized":"2024-03-28T16:50:50.209217+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.209010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:45.672112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:37.582486+0000","last_change":"2024-03-28T16:49:04.753745+0000","last_active":"2024-03-28T16:49:37.582486+0000","last_peered":"2024-03-28T16:49:37.582486+0000","last_clean":"2024-03-28T16:49:37.582486+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:49:37.582486+0000","last_undegraded":"2024-03-28T16:49:37.582486+0000","last_fullsized":"2024-03-28T16:49:37.582486+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:40.575349+0000","last_change":"2024-03-28T16:49:37.618664+0000","last_active":"2024-03-28T16:49:40.575349+0000","last_peered":"2024-03-28T16:49:40.575349+0000","last_clean":"2024-03-28T16:49:40.575349+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:49:40.575349+0000","last_undegraded":"2024-03-28T16:49:40.575349+0000","last_fullsized":"2024-03-28T16:49:40.575349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.412173+0000","last_change":"2024-03-28T16:48:54.658051+0000","last_active":"2024-03-28T16:50:46.412173+0000","last_peered":"2024-03-28T16:50:46.412173+0000","last_clean":"2024-03-28T16:50:46.412173+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:50:46.412173+0000","last_undegraded":"2024-03-28T16:50:46.412173+0000","last_fullsized":"2024-03-28T16:50:46.412173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532370+0000","last_change":"2024-03-28T16:48:56.682367+0000","last_active":"2024-03-28T16:50:45.532370+0000","last_peered":"2024-03-28T16:50:45.532370+0000","last_clean":"2024-03-28T16:50:45.532370+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:50:45.532370+0000","last_undegraded":"2024-03-28T16:50:45.532370+0000","last_fullsized":"2024-03-28T16:50:45.532370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532018+0000","last_change":"2024-03-28T16:48:54.655298+0000","last_active":"2024-03-28T16:50:45.532018+0000","last_peered":"2024-03-28T16:50:45.532018+0000","last_clean":"2024-03-28T16:50:45.532018+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:50:45.532018+0000","last_undegraded":"2024-03-28T16:50:45.532018+0000","last_fullsized":"2024-03-28T16:50:45.532018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'3","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872476+0000","last_change":"2024-03-28T16:48:56.671798+0000","last_active":"2024-03-28T16:50:46.872476+0000","last_peered":"2024-03-28T16:50:46.872476+0000","last_clean":"2024-03-28T16:50:46.872476+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:50:46.872476+0000","last_undegraded":"2024-03-28T16:50:46.872476+0000","last_fullsized":"2024-03-28T16:50:46.872476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871513+0000","last_change":"2024-03-28T16:48:54.657491+0000","last_active":"2024-03-28T16:50:46.871513+0000","last_peered":"2024-03-28T16:50:46.871513+0000","last_clean":"2024-03-28T16:50:46.871513+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:50:46.871513+0000","last_undegraded":"2024-03-28T16:50:46.871513+0000","last_fullsized":"2024-03-28T16:50:46.871513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532324+0000","last_change":"2024-03-28T16:48:56.683483+0000","last_active":"2024-03-28T16:50:45.532324+0000","last_peered":"2024-03-28T16:50:45.532324+0000","last_clean":"2024-03-28T16:50:45.532324+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:50:45.532324+0000","last_undegraded":"2024-03-28T16:50:45.532324+0000","last_fullsized":"2024-03-28T16:50:45.532324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531959+0000","last_change":"2024-03-28T16:48:54.665811+0000","last_active":"2024-03-28T16:50:45.531959+0000","last_peered":"2024-03-28T16:50:45.531959+0000","last_clean":"2024-03-28T16:50:45.531959+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:50:45.531959+0000","last_undegraded":"2024-03-28T16:50:45.531959+0000","last_fullsized":"2024-03-28T16:50:45.531959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.523436+0000","last_change":"2024-03-28T16:51:21.523575+0000","last_active":"2024-03-28T16:51:21.523436+0000","last_peered":"2024-03-28T16:51:21.523436+0000","last_clean":"2024-03-28T16:51:21.523436+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:51:21.523436+0000","last_undegraded":"2024-03-28T16:51:21.523436+0000","last_fullsized":"2024-03-28T16:51:21.523436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.523213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:56.404435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531872+0000","last_change":"2024-03-28T16:48:54.655391+0000","last_active":"2024-03-28T16:50:45.531872+0000","last_peered":"2024-03-28T16:50:45.531872+0000","last_clean":"2024-03-28T16:50:45.531872+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:50:45.531872+0000","last_undegraded":"2024-03-28T16:50:45.531872+0000","last_fullsized":"2024-03-28T16:50:45.531872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":71,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.278285+0000","last_change":"2024-03-28T16:51:19.278345+0000","last_active":"2024-03-28T16:51:19.278285+0000","last_peered":"2024-03-28T16:51:19.278285+0000","last_clean":"2024-03-28T16:51:19.278285+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:51:19.278285+0000","last_undegraded":"2024-03-28T16:51:19.278285+0000","last_fullsized":"2024-03-28T16:51:19.278285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.278071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:29.679472+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.984798+0000","last_change":"2024-03-28T16:48:54.669640+0000","last_active":"2024-03-28T16:50:45.984798+0000","last_peered":"2024-03-28T16:50:45.984798+0000","last_clean":"2024-03-28T16:50:45.984798+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:50:45.984798+0000","last_undegraded":"2024-03-28T16:50:45.984798+0000","last_fullsized":"2024-03-28T16:50:45.984798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.446697+0000","last_change":"2024-03-28T16:51:18.446757+0000","last_active":"2024-03-28T16:51:18.446697+0000","last_peered":"2024-03-28T16:51:18.446697+0000","last_clean":"2024-03-28T16:51:18.446697+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:51:18.446697+0000","last_undegraded":"2024-03-28T16:51:18.446697+0000","last_fullsized":"2024-03-28T16:51:18.446697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.446501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:51.526879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.283525+0000","last_change":"2024-03-28T16:51:25.283581+0000","last_active":"2024-03-28T16:51:25.283525+0000","last_peered":"2024-03-28T16:51:25.283525+0000","last_clean":"2024-03-28T16:51:25.283525+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:51:25.283525+0000","last_undegraded":"2024-03-28T16:51:25.283525+0000","last_fullsized":"2024-03-28T16:51:25.283525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.283348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:28.047908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.316649+0000","last_change":"2024-03-28T16:51:21.316736+0000","last_active":"2024-03-28T16:51:21.316649+0000","last_peered":"2024-03-28T16:51:21.316649+0000","last_clean":"2024-03-28T16:51:21.316649+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:51:21.316649+0000","last_undegraded":"2024-03-28T16:51:21.316649+0000","last_fullsized":"2024-03-28T16:51:21.316649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.316476+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:10:20.931612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531947+0000","last_change":"2024-03-28T16:48:54.659870+0000","last_active":"2024-03-28T16:50:45.531947+0000","last_peered":"2024-03-28T16:50:45.531947+0000","last_clean":"2024-03-28T16:50:45.531947+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:50:45.531947+0000","last_undegraded":"2024-03-28T16:50:45.531947+0000","last_fullsized":"2024-03-28T16:50:45.531947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.289210+0000","last_change":"2024-03-28T16:51:10.289210+0000","last_active":"2024-03-28T16:51:10.289210+0000","last_peered":"2024-03-28T16:51:10.289210+0000","last_clean":"2024-03-28T16:51:10.289210+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:51:10.289210+0000","last_undegraded":"2024-03-28T16:51:10.289210+0000","last_fullsized":"2024-03-28T16:51:10.289210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.288939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:52.264862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871590+0000","last_change":"2024-03-28T16:48:54.657662+0000","last_active":"2024-03-28T16:50:46.871590+0000","last_peered":"2024-03-28T16:50:46.871590+0000","last_clean":"2024-03-28T16:50:46.871590+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:50:46.871590+0000","last_undegraded":"2024-03-28T16:50:46.871590+0000","last_fullsized":"2024-03-28T16:50:46.871590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'1","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872522+0000","last_change":"2024-03-28T16:48:56.682872+0000","last_active":"2024-03-28T16:50:46.872522+0000","last_peered":"2024-03-28T16:50:46.872522+0000","last_clean":"2024-03-28T16:50:46.872522+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:50:46.872522+0000","last_undegraded":"2024-03-28T16:50:46.872522+0000","last_fullsized":"2024-03-28T16:50:46.872522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532074+0000","last_change":"2024-03-28T16:48:54.663496+0000","last_active":"2024-03-28T16:50:45.532074+0000","last_peered":"2024-03-28T16:50:45.532074+0000","last_clean":"2024-03-28T16:50:45.532074+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:50:45.532074+0000","last_undegraded":"2024-03-28T16:50:45.532074+0000","last_fullsized":"2024-03-28T16:50:45.532074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'1","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532298+0000","last_change":"2024-03-28T16:48:56.678342+0000","last_active":"2024-03-28T16:50:45.532298+0000","last_peered":"2024-03-28T16:50:45.532298+0000","last_clean":"2024-03-28T16:50:45.532298+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:50:45.532298+0000","last_undegraded":"2024-03-28T16:50:45.532298+0000","last_fullsized":"2024-03-28T16:50:45.532298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.301018+0000","last_change":"2024-03-28T16:51:22.301084+0000","last_active":"2024-03-28T16:51:22.301018+0000","last_peered":"2024-03-28T16:51:22.301018+0000","last_clean":"2024-03-28T16:51:22.301018+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:51:22.301018+0000","last_undegraded":"2024-03-28T16:51:22.301018+0000","last_fullsized":"2024-03-28T16:51:22.301018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.300834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:39.676855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.261019+0000","last_change":"2024-03-28T16:51:18.261152+0000","last_active":"2024-03-28T16:51:18.261019+0000","last_peered":"2024-03-28T16:51:18.261019+0000","last_clean":"2024-03-28T16:51:18.261019+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:51:18.261019+0000","last_undegraded":"2024-03-28T16:51:18.261019+0000","last_fullsized":"2024-03-28T16:51:18.261019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.260778+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:15.421901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.256951+0000","last_change":"2024-03-28T16:51:24.257006+0000","last_active":"2024-03-28T16:51:24.256951+0000","last_peered":"2024-03-28T16:51:24.256951+0000","last_clean":"2024-03-28T16:51:24.256951+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:51:24.256951+0000","last_undegraded":"2024-03-28T16:51:24.256951+0000","last_fullsized":"2024-03-28T16:51:24.256951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.256765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:47.235830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.319730+0000","last_change":"2024-03-28T16:51:14.319730+0000","last_active":"2024-03-28T16:51:14.319730+0000","last_peered":"2024-03-28T16:51:14.319730+0000","last_clean":"2024-03-28T16:51:14.319730+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:51:14.319730+0000","last_undegraded":"2024-03-28T16:51:14.319730+0000","last_fullsized":"2024-03-28T16:51:14.319730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.319444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:39.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871574+0000","last_change":"2024-03-28T16:48:54.676815+0000","last_active":"2024-03-28T16:50:46.871574+0000","last_peered":"2024-03-28T16:50:46.871574+0000","last_clean":"2024-03-28T16:50:46.871574+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:50:46.871574+0000","last_undegraded":"2024-03-28T16:50:46.871574+0000","last_fullsized":"2024-03-28T16:50:46.871574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.281665+0000","last_change":"2024-03-28T16:51:06.281665+0000","last_active":"2024-03-28T16:51:06.281665+0000","last_peered":"2024-03-28T16:51:06.281665+0000","last_clean":"2024-03-28T16:51:06.281665+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:51:06.281665+0000","last_undegraded":"2024-03-28T16:51:06.281665+0000","last_fullsized":"2024-03-28T16:51:06.281665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.281413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:32.845661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871650+0000","last_change":"2024-03-28T16:48:54.663003+0000","last_active":"2024-03-28T16:50:46.871650+0000","last_peered":"2024-03-28T16:50:46.871650+0000","last_clean":"2024-03-28T16:50:46.871650+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:50:46.871650+0000","last_undegraded":"2024-03-28T16:50:46.871650+0000","last_fullsized":"2024-03-28T16:50:46.871650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":66,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.872591+0000","last_change":"2024-03-28T16:48:56.685753+0000","last_active":"2024-03-28T16:50:46.872591+0000","last_peered":"2024-03-28T16:50:46.872591+0000","last_clean":"2024-03-28T16:50:46.872591+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:50:46.872591+0000","last_undegraded":"2024-03-28T16:50:46.872591+0000","last_fullsized":"2024-03-28T16:50:46.872591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.338591+0000","last_change":"2024-03-28T16:51:23.338657+0000","last_active":"2024-03-28T16:51:23.338591+0000","last_peered":"2024-03-28T16:51:23.338591+0000","last_clean":"2024-03-28T16:51:23.338591+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:51:23.338591+0000","last_undegraded":"2024-03-28T16:51:23.338591+0000","last_fullsized":"2024-03-28T16:51:23.338591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:37.364884+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'10","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.275847+0000","last_change":"2024-03-28T16:51:23.275918+0000","last_active":"2024-03-28T16:51:23.275847+0000","last_peered":"2024-03-28T16:51:23.275847+0000","last_clean":"2024-03-28T16:51:23.275847+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:51:23.275847+0000","last_undegraded":"2024-03-28T16:51:23.275847+0000","last_fullsized":"2024-03-28T16:51:23.275847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.275672+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:58.156894+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871661+0000","last_change":"2024-03-28T16:48:54.665812+0000","last_active":"2024-03-28T16:50:46.871661+0000","last_peered":"2024-03-28T16:50:46.871661+0000","last_clean":"2024-03-28T16:50:46.871661+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:50:46.871661+0000","last_undegraded":"2024-03-28T16:50:46.871661+0000","last_fullsized":"2024-03-28T16:50:46.871661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871739+0000","last_change":"2024-03-28T16:48:54.665260+0000","last_active":"2024-03-28T16:50:46.871739+0000","last_peered":"2024-03-28T16:50:46.871739+0000","last_clean":"2024-03-28T16:50:46.871739+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:50:46.871739+0000","last_undegraded":"2024-03-28T16:50:46.871739+0000","last_fullsized":"2024-03-28T16:50:46.871739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.440088+0000","last_change":"2024-03-28T16:51:16.440088+0000","last_active":"2024-03-28T16:51:16.440088+0000","last_peered":"2024-03-28T16:51:16.440088+0000","last_clean":"2024-03-28T16:51:16.440088+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:51:16.440088+0000","last_undegraded":"2024-03-28T16:51:16.440088+0000","last_fullsized":"2024-03-28T16:51:16.440088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.439837+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:19.204397+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532148+0000","last_change":"2024-03-28T16:48:54.663735+0000","last_active":"2024-03-28T16:50:45.532148+0000","last_peered":"2024-03-28T16:50:45.532148+0000","last_clean":"2024-03-28T16:50:45.532148+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:50:45.532148+0000","last_undegraded":"2024-03-28T16:50:45.532148+0000","last_fullsized":"2024-03-28T16:50:45.532148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'1","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.985833+0000","last_change":"2024-03-28T16:48:56.683832+0000","last_active":"2024-03-28T16:50:45.985833+0000","last_peered":"2024-03-28T16:50:45.985833+0000","last_clean":"2024-03-28T16:50:45.985833+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:50:45.985833+0000","last_undegraded":"2024-03-28T16:50:45.985833+0000","last_fullsized":"2024-03-28T16:50:45.985833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1992,"ondisk_log_size":1992,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1539,"num_read_kb":1325,"num_write":581,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1750,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1422,"ondisk_log_size":1422,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":866,"num_read_kb":864,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":291,"ondisk_log_size":291,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705699,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39916,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331924,"statfs":{"total":96636764160,"available":96595890176,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":1.246,"15min":1.246},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81899999999999995,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.26700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.79000000000000004,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.63},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.76600000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.69299999999999995}]},{"osd":2,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77400000000000002,"5min":1.671,"15min":1.671},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76400000000000001,"5min":1.331,"15min":1.331},"last":0.53500000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705700,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.26900000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.56100000000000005,"15min":0.54000000000000004},"min":{"1min":0.28599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.30399999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.32200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.64200000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705700,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.45300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79600000000000004,"5min":0.878,"15min":0.878},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69399999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.83799999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.501},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.27100000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.86299999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78700000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705699,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.40600000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90000000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.73399999999999999}]},{"osd":2,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90800000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.117},"last":0.82499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:51:28.097 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.097 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.098 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.098 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.098 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.098 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.098 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.099 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.099 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.099 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.099 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.099 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.099 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.100 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.100 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.100 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.100 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.100 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.100 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.101 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.101 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.101 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.101 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.101 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.101 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.102 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.102 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.102 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.102 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.103 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.103 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.103 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.103 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.103 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.104 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.104 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.104 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.104 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.104 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-28T16:48:42.575180+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=42, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.104 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.105 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.105 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.105 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.105 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.105 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.106 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.106 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.106 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.106 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.106 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.106 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.107 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.107 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-03-28T16:49:01.680227+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=1, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.107 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.107 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.107 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.107 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.108 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.108 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.108 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.108 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.108 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.109 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.109 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.109 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.109 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.109 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.109 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.110 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.110 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.110 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.110 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.110 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.111 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.111 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.111 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.111 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.111 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.111 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.112 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.112 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.112 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.112 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.112 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.112 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.113 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.113 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.113 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.113 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.113 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.113 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.113 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.114 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.114 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.114 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.114 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.114 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.114 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.115 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.115 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.115 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.115 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.115 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.115 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.116 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.116 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.116 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.116 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.116 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.116 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.116 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.117 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.117 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.117 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.117 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:28.117 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-28T16:51:48.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:51:48.251 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.248+0000 7f95838e8640 1 Processor -- start 2024-03-28T16:51:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.248+0000 7f95838e8640 1 -- start start 2024-03-28T16:51:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.248+0000 7f95838e8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f957c098910 0x7f957c098cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.248+0000 7f95838e8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c099d80 con 0x7f957c09d7e0 2024-03-28T16:51:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.248+0000 7f95838e8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957c09a340 con 0x7f957c098910 2024-03-28T16:51:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9581684640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f957c098910 0x7f957c098cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9580e83640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f957c09d7e0 0x7f957c096590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51164/0 (socket says 172.21.15.119:51164) 2024-03-28T16:51:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9580e83640 1 -- 172.21.15.119:0/1466459048 learned_addr learned my addr 172.21.15.119:0/1466459048 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:51:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9573fff640 1 -- 172.21.15.119:0/1466459048 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527678400 0 0) 0x7f957c099d80 con 0x7f957c09d7e0 2024-03-28T16:51:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9573fff640 1 -- 172.21.15.119:0/1466459048 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9568003530 con 0x7f957c09d7e0 2024-03-28T16:51:48.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9581684640 1 -- 172.21.15.119:0/1466459048 >> v1:172.21.15.119:6789/0 conn(0x7f957c09d7e0 legacy=0x7f957c096590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:48.254 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9581684640 1 -- 172.21.15.119:0/1466459048 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c09a5e0 con 0x7f957c098910 2024-03-28T16:51:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9581684640 1 --2- 172.21.15.119:0/1466459048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c098cf0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9578009740 tx=0x7f957801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578306fd58751e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9573fff640 1 -- 172.21.15.119:0/1466459048 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95780ac570 con 0x7f957c098910 2024-03-28T16:51:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9573fff640 1 -- 172.21.15.119:0/1466459048 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95780ac6d0 con 0x7f957c098910 2024-03-28T16:51:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f9573fff640 1 -- 172.21.15.119:0/1466459048 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95780ac970 con 0x7f957c098910 2024-03-28T16:51:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f95838e8640 1 -- 172.21.15.119:0/1466459048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 msgr2=0x7f957c098cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f95838e8640 1 --2- 172.21.15.119:0/1466459048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c098cf0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9578009740 tx=0x7f957801c3c0 comp rx=0 tx=0).stop 2024-03-28T16:51:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f95838e8640 1 -- 172.21.15.119:0/1466459048 shutdown_connections 2024-03-28T16:51:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f95838e8640 1 --2- 172.21.15.119:0/1466459048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c098cf0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f95838e8640 1 -- 172.21.15.119:0/1466459048 >> 172.21.15.119:0/1466459048 conn(0x7f957c0a0a00 msgr2=0x7f957c09cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:51:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f95838e8640 1 -- 172.21.15.119:0/1466459048 shutdown_connections 2024-03-28T16:51:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.252+0000 7f95838e8640 1 -- 172.21.15.119:0/1466459048 wait complete. 2024-03-28T16:51:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f95838e8640 1 Processor -- start 2024-03-28T16:51:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f95838e8640 1 -- start start 2024-03-28T16:51:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f95838e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c1c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f95838e8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c133570 con 0x7f957c098910 2024-03-28T16:51:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9581684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c1c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9581684640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c1c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33874/0 (socket says 172.21.15.119:33874) 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9581684640 1 -- 172.21.15.119:0/2680644628 learned_addr learned my addr 172.21.15.119:0/2680644628 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9581684640 1 -- 172.21.15.119:0/2680644628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009420 con 0x7f957c098910 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9581684640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c1c5bc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9578019ff0 tx=0x7f957801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9571ffb640 1 -- 172.21.15.119:0/2680644628 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95780b8450 con 0x7f957c098910 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9571ffb640 1 -- 172.21.15.119:0/2680644628 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95780b85b0 con 0x7f957c098910 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c132ee0 con 0x7f957c098910 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9571ffb640 1 -- 172.21.15.119:0/2680644628 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95780b8870 con 0x7f957c098910 2024-03-28T16:51:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c133d20 con 0x7f957c098910 2024-03-28T16:51:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9571ffb640 1 -- 172.21.15.119:0/2680644628 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f95780b89d0 con 0x7f957c098910 2024-03-28T16:51:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c062860 con 0x7f957c098910 2024-03-28T16:51:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9571ffb640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9560035a10 0x7f9560037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:51:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9571ffb640 1 -- 172.21.15.119:0/2680644628 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f95780e31e0 con 0x7f957c098910 2024-03-28T16:51:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9580e83640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9560035a10 0x7f9560037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:51:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.256+0000 7f9580e83640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9560035a10 0x7f9560037ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f956c007fd0 tx=0x7f956c007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:51:48.262 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.260+0000 7f9571ffb640 1 -- 172.21.15.119:0/2680644628 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95780ab760 con 0x7f957c098910 2024-03-28T16:51:48.417 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.416+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f957c133260 con 0x7f9560035a10 2024-03-28T16:51:48.463 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.460+0000 7f9571ffb640 1 -- 172.21.15.119:0/2680644628 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+672154 (secure 0 0 0) 0x7f957c133260 con 0x7f9560035a10 2024-03-28T16:51:48.469 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:51:48.475 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:51:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9560035a10 msgr2=0x7f9560037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9560035a10 0x7f9560037ed0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f956c007fd0 tx=0x7f956c007980 comp rx=0 tx=0).stop 2024-03-28T16:51:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 msgr2=0x7f957c1c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:51:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c1c5bc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9578019ff0 tx=0x7f957801ac00 comp rx=0 tx=0).stop 2024-03-28T16:51:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 shutdown_connections 2024-03-28T16:51:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f9560035a10 0x7f9560037ed0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 --2- 172.21.15.119:0/2680644628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f957c098910 0x7f957c1c5bc0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:51:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 >> 172.21.15.119:0/2680644628 conn(0x7f957c0a0a00 msgr2=0x7f957c09b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:51:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 shutdown_connections 2024-03-28T16:51:48.479 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:51:48.468+0000 7f95838e8640 1 -- 172.21.15.119:0/2680644628 wait complete. 2024-03-28T16:51:48.485 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-03-28T16:51:46.566762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":4029,"num_read_kb":3679,"num_write":1543,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4051,"ondisk_log_size":4051,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":164928,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":162105,"kb_avail":377322432,"statfs":{"total":386547056640,"available":386378170368,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":165996310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005680"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.238075+0000","last_change":"2024-03-28T16:51:28.238133+0000","last_active":"2024-03-28T16:51:28.238075+0000","last_peered":"2024-03-28T16:51:28.238075+0000","last_clean":"2024-03-28T16:51:28.238075+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:51:28.238075+0000","last_undegraded":"2024-03-28T16:51:28.238075+0000","last_fullsized":"2024-03-28T16:51:28.238075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:28.237880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:28.237880+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.237880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:51.619287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.303646+0000","last_change":"2024-03-28T16:51:11.303646+0000","last_active":"2024-03-28T16:51:11.303646+0000","last_peered":"2024-03-28T16:51:11.303646+0000","last_clean":"2024-03-28T16:51:11.303646+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:51:11.303646+0000","last_undegraded":"2024-03-28T16:51:11.303646+0000","last_fullsized":"2024-03-28T16:51:11.303646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.303263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:59.401107+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.569902+0000","last_change":"2024-03-28T16:51:22.569959+0000","last_active":"2024-03-28T16:51:22.569902+0000","last_peered":"2024-03-28T16:51:22.569902+0000","last_clean":"2024-03-28T16:51:22.569902+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:51:22.569902+0000","last_undegraded":"2024-03-28T16:51:22.569902+0000","last_fullsized":"2024-03-28T16:51:22.569902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:22.569729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:22.569729+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.569729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:34.374938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.098164+0000","last_change":"2024-03-28T16:51:22.098164+0000","last_active":"2024-03-28T16:51:22.098164+0000","last_peered":"2024-03-28T16:51:22.098164+0000","last_clean":"2024-03-28T16:51:22.098164+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:51:22.098164+0000","last_undegraded":"2024-03-28T16:51:22.098164+0000","last_fullsized":"2024-03-28T16:51:22.098164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:22.097907+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:22.097907+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.097907+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:26.268880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.105158+0000","last_change":"2024-03-28T16:51:39.105220+0000","last_active":"2024-03-28T16:51:39.105158+0000","last_peered":"2024-03-28T16:51:39.105158+0000","last_clean":"2024-03-28T16:51:39.105158+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:51:39.105158+0000","last_undegraded":"2024-03-28T16:51:39.105158+0000","last_fullsized":"2024-03-28T16:51:39.105158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.104968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.104968+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.104968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:12:14.485249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'4","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.105540+0000","last_change":"2024-03-28T16:51:19.105540+0000","last_active":"2024-03-28T16:51:19.105540+0000","last_peered":"2024-03-28T16:51:19.105540+0000","last_clean":"2024-03-28T16:51:19.105540+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:51:19.105540+0000","last_undegraded":"2024-03-28T16:51:19.105540+0000","last_fullsized":"2024-03-28T16:51:19.105540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.105244+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:35:56.759918+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'3","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.232984+0000","last_change":"2024-03-28T16:51:29.233046+0000","last_active":"2024-03-28T16:51:29.232984+0000","last_peered":"2024-03-28T16:51:29.232984+0000","last_clean":"2024-03-28T16:51:29.232984+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:51:29.232984+0000","last_undegraded":"2024-03-28T16:51:29.232984+0000","last_fullsized":"2024-03-28T16:51:29.232984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:51:29.232791+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:51:29.232791+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.232791+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:46.608812+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.197100+0000","last_change":"2024-03-28T16:51:37.197158+0000","last_active":"2024-03-28T16:51:37.197100+0000","last_peered":"2024-03-28T16:51:37.197100+0000","last_clean":"2024-03-28T16:51:37.197100+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:51:37.197100+0000","last_undegraded":"2024-03-28T16:51:37.197100+0000","last_fullsized":"2024-03-28T16:51:37.197100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:37.196925+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:37.196925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.196925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:33.164623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871870+0000","last_change":"2024-03-28T16:48:54.661971+0000","last_active":"2024-03-28T16:50:46.871870+0000","last_peered":"2024-03-28T16:50:46.871870+0000","last_clean":"2024-03-28T16:50:46.871870+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:50:46.871870+0000","last_undegraded":"2024-03-28T16:50:46.871870+0000","last_fullsized":"2024-03-28T16:50:46.871870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.305502+0000","last_change":"2024-03-28T16:51:16.305559+0000","last_active":"2024-03-28T16:51:16.305502+0000","last_peered":"2024-03-28T16:51:16.305502+0000","last_clean":"2024-03-28T16:51:16.305502+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:51:16.305502+0000","last_undegraded":"2024-03-28T16:51:16.305502+0000","last_fullsized":"2024-03-28T16:51:16.305502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.305318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:43.571649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.321896+0000","last_change":"2024-03-28T16:51:21.321896+0000","last_active":"2024-03-28T16:51:21.321896+0000","last_peered":"2024-03-28T16:51:21.321896+0000","last_clean":"2024-03-28T16:51:21.321896+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:51:21.321896+0000","last_undegraded":"2024-03-28T16:51:21.321896+0000","last_fullsized":"2024-03-28T16:51:21.321896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.321658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:30.540834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.057665+0000","last_change":"2024-03-28T16:51:20.057727+0000","last_active":"2024-03-28T16:51:20.057665+0000","last_peered":"2024-03-28T16:51:20.057665+0000","last_clean":"2024-03-28T16:51:20.057665+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:51:20.057665+0000","last_undegraded":"2024-03-28T16:51:20.057665+0000","last_fullsized":"2024-03-28T16:51:20.057665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.057477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:08.878405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871840+0000","last_change":"2024-03-28T16:48:54.672203+0000","last_active":"2024-03-28T16:50:46.871840+0000","last_peered":"2024-03-28T16:50:46.871840+0000","last_clean":"2024-03-28T16:50:46.871840+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:50:46.871840+0000","last_undegraded":"2024-03-28T16:50:46.871840+0000","last_fullsized":"2024-03-28T16:50:46.871840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.206567+0000","last_change":"2024-03-28T16:51:34.206624+0000","last_active":"2024-03-28T16:51:34.206567+0000","last_peered":"2024-03-28T16:51:34.206567+0000","last_clean":"2024-03-28T16:51:34.206567+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:51:34.206567+0000","last_undegraded":"2024-03-28T16:51:34.206567+0000","last_fullsized":"2024-03-28T16:51:34.206567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:51:34.206377+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:51:34.206377+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.206377+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:21.188653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.467047+0000","last_change":"2024-03-28T16:51:39.467111+0000","last_active":"2024-03-28T16:51:39.467047+0000","last_peered":"2024-03-28T16:51:39.467047+0000","last_clean":"2024-03-28T16:51:39.467047+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:51:39.467047+0000","last_undegraded":"2024-03-28T16:51:39.467047+0000","last_fullsized":"2024-03-28T16:51:39.467047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.466858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.466858+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.466858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:31.475945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.353532+0000","last_change":"2024-03-28T16:51:12.353604+0000","last_active":"2024-03-28T16:51:12.353532+0000","last_peered":"2024-03-28T16:51:12.353532+0000","last_clean":"2024-03-28T16:51:12.353532+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:51:12.353532+0000","last_undegraded":"2024-03-28T16:51:12.353532+0000","last_fullsized":"2024-03-28T16:51:12.353532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.353309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:42.028446+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.145909+0000","last_change":"2024-03-28T16:51:28.145970+0000","last_active":"2024-03-28T16:51:28.145909+0000","last_peered":"2024-03-28T16:51:28.145909+0000","last_clean":"2024-03-28T16:51:28.145909+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:51:28.145909+0000","last_undegraded":"2024-03-28T16:51:28.145909+0000","last_fullsized":"2024-03-28T16:51:28.145909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:28.145685+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:28.145685+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.145685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:12.221861+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.234222+0000","last_change":"2024-03-28T16:51:28.234290+0000","last_active":"2024-03-28T16:51:28.234222+0000","last_peered":"2024-03-28T16:51:28.234222+0000","last_clean":"2024-03-28T16:51:28.234222+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:51:28.234222+0000","last_undegraded":"2024-03-28T16:51:28.234222+0000","last_fullsized":"2024-03-28T16:51:28.234222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:28.234045+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:28.234045+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.234045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:10:44.817774+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'10","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.246752+0000","last_change":"2024-03-28T16:51:07.246752+0000","last_active":"2024-03-28T16:51:07.246752+0000","last_peered":"2024-03-28T16:51:07.246752+0000","last_clean":"2024-03-28T16:51:07.246752+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:51:07.246752+0000","last_undegraded":"2024-03-28T16:51:07.246752+0000","last_fullsized":"2024-03-28T16:51:07.246752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.246384+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:42.040943+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.241054+0000","last_change":"2024-03-28T16:51:39.241109+0000","last_active":"2024-03-28T16:51:39.241054+0000","last_peered":"2024-03-28T16:51:39.241054+0000","last_clean":"2024-03-28T16:51:39.241054+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:51:39.241054+0000","last_undegraded":"2024-03-28T16:51:39.241054+0000","last_fullsized":"2024-03-28T16:51:39.241054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.240890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.240890+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.240890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:00.740460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.251563+0000","last_change":"2024-03-28T16:51:08.251563+0000","last_active":"2024-03-28T16:51:08.251563+0000","last_peered":"2024-03-28T16:51:08.251563+0000","last_clean":"2024-03-28T16:51:08.251563+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:51:08.251563+0000","last_undegraded":"2024-03-28T16:51:08.251563+0000","last_fullsized":"2024-03-28T16:51:08.251563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.251179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:34.785916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.213499+0000","last_change":"2024-03-28T16:51:30.213568+0000","last_active":"2024-03-28T16:51:30.213499+0000","last_peered":"2024-03-28T16:51:30.213499+0000","last_clean":"2024-03-28T16:51:30.213499+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:51:30.213499+0000","last_undegraded":"2024-03-28T16:51:30.213499+0000","last_fullsized":"2024-03-28T16:51:30.213499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.213281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.213281+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.213281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:48.948290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.549877+0000","last_change":"2024-03-28T16:51:25.549934+0000","last_active":"2024-03-28T16:51:25.549877+0000","last_peered":"2024-03-28T16:51:25.549877+0000","last_clean":"2024-03-28T16:51:25.549877+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:51:25.549877+0000","last_undegraded":"2024-03-28T16:51:25.549877+0000","last_fullsized":"2024-03-28T16:51:25.549877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:25.549686+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:25.549686+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.549686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:12.531962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.161305+0000","last_change":"2024-03-28T16:51:33.161357+0000","last_active":"2024-03-28T16:51:33.161305+0000","last_peered":"2024-03-28T16:51:33.161305+0000","last_clean":"2024-03-28T16:51:33.161305+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:51:33.161305+0000","last_undegraded":"2024-03-28T16:51:33.161305+0000","last_fullsized":"2024-03-28T16:51:33.161305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.161112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.161112+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.161112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:23.989489+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.320871+0000","last_change":"2024-03-28T16:51:11.320871+0000","last_active":"2024-03-28T16:51:11.320871+0000","last_peered":"2024-03-28T16:51:11.320871+0000","last_clean":"2024-03-28T16:51:11.320871+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:51:11.320871+0000","last_undegraded":"2024-03-28T16:51:11.320871+0000","last_fullsized":"2024-03-28T16:51:11.320871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.320440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:29.097635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.480322+0000","last_change":"2024-03-28T16:51:20.480322+0000","last_active":"2024-03-28T16:51:20.480322+0000","last_peered":"2024-03-28T16:51:20.480322+0000","last_clean":"2024-03-28T16:51:20.480322+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:51:20.480322+0000","last_undegraded":"2024-03-28T16:51:20.480322+0000","last_fullsized":"2024-03-28T16:51:20.480322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.479971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:37.855992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.332059+0000","last_change":"2024-03-28T16:51:15.332059+0000","last_active":"2024-03-28T16:51:15.332059+0000","last_peered":"2024-03-28T16:51:15.332059+0000","last_clean":"2024-03-28T16:51:15.332059+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:51:15.332059+0000","last_undegraded":"2024-03-28T16:51:15.332059+0000","last_fullsized":"2024-03-28T16:51:15.332059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.331796+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:37:44.933269+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871718+0000","last_change":"2024-03-28T16:48:54.673730+0000","last_active":"2024-03-28T16:50:46.871718+0000","last_peered":"2024-03-28T16:50:46.871718+0000","last_clean":"2024-03-28T16:50:46.871718+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:50:46.871718+0000","last_undegraded":"2024-03-28T16:50:46.871718+0000","last_fullsized":"2024-03-28T16:50:46.871718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'20","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.046926+0000","last_change":"2024-03-28T16:51:40.046995+0000","last_active":"2024-03-28T16:51:40.046926+0000","last_peered":"2024-03-28T16:51:40.046926+0000","last_clean":"2024-03-28T16:51:40.046926+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:51:40.046926+0000","last_undegraded":"2024-03-28T16:51:40.046926+0000","last_fullsized":"2024-03-28T16:51:40.046926+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'20","last_scrub_stamp":"2024-03-28T16:51:40.046736+0000","last_deep_scrub":"31'20","last_deep_scrub_stamp":"2024-03-28T16:51:40.046736+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.046736+0000","objects_scrubbed":1,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:55:13.676549+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.132778+0000","last_change":"2024-03-28T16:51:24.132846+0000","last_active":"2024-03-28T16:51:24.132778+0000","last_peered":"2024-03-28T16:51:24.132778+0000","last_clean":"2024-03-28T16:51:24.132778+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:51:24.132778+0000","last_undegraded":"2024-03-28T16:51:24.132778+0000","last_fullsized":"2024-03-28T16:51:24.132778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:24.132547+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:24.132547+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.132547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:23.773840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871328+0000","last_change":"2024-03-28T16:48:54.670023+0000","last_active":"2024-03-28T16:50:46.871328+0000","last_peered":"2024-03-28T16:50:46.871328+0000","last_clean":"2024-03-28T16:50:46.871328+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:50:46.871328+0000","last_undegraded":"2024-03-28T16:50:46.871328+0000","last_fullsized":"2024-03-28T16:50:46.871328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.138003+0000","last_change":"2024-03-28T16:50:50.138003+0000","last_active":"2024-03-28T16:50:50.138003+0000","last_peered":"2024-03-28T16:50:50.138003+0000","last_clean":"2024-03-28T16:50:50.138003+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:50:50.138003+0000","last_undegraded":"2024-03-28T16:50:50.138003+0000","last_fullsized":"2024-03-28T16:50:50.138003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.137765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:11.908122+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.226657+0000","last_change":"2024-03-28T16:51:10.226657+0000","last_active":"2024-03-28T16:51:10.226657+0000","last_peered":"2024-03-28T16:51:10.226657+0000","last_clean":"2024-03-28T16:51:10.226657+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:51:10.226657+0000","last_undegraded":"2024-03-28T16:51:10.226657+0000","last_fullsized":"2024-03-28T16:51:10.226657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.226407+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:01.054784+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.251369+0000","last_change":"2024-03-28T16:50:57.251369+0000","last_active":"2024-03-28T16:50:57.251369+0000","last_peered":"2024-03-28T16:50:57.251369+0000","last_clean":"2024-03-28T16:50:57.251369+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:50:57.251369+0000","last_undegraded":"2024-03-28T16:50:57.251369+0000","last_fullsized":"2024-03-28T16:50:57.251369+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.251130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:45.232439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'7","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.077837+0000","last_change":"2024-03-28T16:51:41.077908+0000","last_active":"2024-03-28T16:51:41.077837+0000","last_peered":"2024-03-28T16:51:41.077837+0000","last_clean":"2024-03-28T16:51:41.077837+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:51:41.077837+0000","last_undegraded":"2024-03-28T16:51:41.077837+0000","last_fullsized":"2024-03-28T16:51:41.077837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'7","last_scrub_stamp":"2024-03-28T16:51:41.077657+0000","last_deep_scrub":"31'7","last_deep_scrub_stamp":"2024-03-28T16:51:41.077657+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.077657+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:33.086744+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.082932+0000","last_change":"2024-03-28T16:51:34.082995+0000","last_active":"2024-03-28T16:51:34.082932+0000","last_peered":"2024-03-28T16:51:34.082932+0000","last_clean":"2024-03-28T16:51:34.082932+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:51:34.082932+0000","last_undegraded":"2024-03-28T16:51:34.082932+0000","last_fullsized":"2024-03-28T16:51:34.082932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:34.082730+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:34.082730+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.082730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:12:07.452135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.286192+0000","last_change":"2024-03-28T16:51:27.286244+0000","last_active":"2024-03-28T16:51:27.286192+0000","last_peered":"2024-03-28T16:51:27.286192+0000","last_clean":"2024-03-28T16:51:27.286192+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:51:27.286192+0000","last_undegraded":"2024-03-28T16:51:27.286192+0000","last_fullsized":"2024-03-28T16:51:27.286192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:27.286005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:27.286005+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.286005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:00.366383+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.328962+0000","last_change":"2024-03-28T16:51:04.328962+0000","last_active":"2024-03-28T16:51:04.328962+0000","last_peered":"2024-03-28T16:51:04.328962+0000","last_clean":"2024-03-28T16:51:04.328962+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:51:04.328962+0000","last_undegraded":"2024-03-28T16:51:04.328962+0000","last_fullsized":"2024-03-28T16:51:04.328962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.328732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:30.892979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"31'10","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.284276+0000","last_change":"2024-03-28T16:50:55.284276+0000","last_active":"2024-03-28T16:50:55.284276+0000","last_peered":"2024-03-28T16:50:55.284276+0000","last_clean":"2024-03-28T16:50:55.284276+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:50:55.284276+0000","last_undegraded":"2024-03-28T16:50:55.284276+0000","last_fullsized":"2024-03-28T16:50:55.284276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.283889+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:07.952223+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.156607+0000","last_change":"2024-03-28T16:50:49.156607+0000","last_active":"2024-03-28T16:50:49.156607+0000","last_peered":"2024-03-28T16:50:49.156607+0000","last_clean":"2024-03-28T16:50:49.156607+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:50:49.156607+0000","last_undegraded":"2024-03-28T16:50:49.156607+0000","last_fullsized":"2024-03-28T16:50:49.156607+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.156374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:19.754691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.533011+0000","last_change":"2024-03-28T16:49:04.743161+0000","last_active":"2024-03-28T16:50:45.533011+0000","last_peered":"2024-03-28T16:50:45.533011+0000","last_clean":"2024-03-28T16:50:45.533011+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:50:45.533011+0000","last_undegraded":"2024-03-28T16:50:45.533011+0000","last_fullsized":"2024-03-28T16:50:45.533011+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":68,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.147922+0000","last_change":"2024-03-28T16:51:30.147995+0000","last_active":"2024-03-28T16:51:30.147922+0000","last_peered":"2024-03-28T16:51:30.147922+0000","last_clean":"2024-03-28T16:51:30.147922+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:51:30.147922+0000","last_undegraded":"2024-03-28T16:51:30.147922+0000","last_fullsized":"2024-03-28T16:51:30.147922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:30.147724+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:30.147724+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.147724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:42:47.609297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871281+0000","last_change":"2024-03-28T16:48:54.659330+0000","last_active":"2024-03-28T16:50:46.871281+0000","last_peered":"2024-03-28T16:50:46.871281+0000","last_clean":"2024-03-28T16:50:46.871281+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:50:46.871281+0000","last_undegraded":"2024-03-28T16:50:46.871281+0000","last_fullsized":"2024-03-28T16:50:46.871281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'6","reported_seq":216,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.337837+0000","last_change":"2024-03-28T16:51:08.337837+0000","last_active":"2024-03-28T16:51:08.337837+0000","last_peered":"2024-03-28T16:51:08.337837+0000","last_clean":"2024-03-28T16:51:08.337837+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:51:08.337837+0000","last_undegraded":"2024-03-28T16:51:08.337837+0000","last_fullsized":"2024-03-28T16:51:08.337837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.337418+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:50.313341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":270,"num_read_kb":269,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'16","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.357686+0000","last_change":"2024-03-28T16:50:59.357686+0000","last_active":"2024-03-28T16:50:59.357686+0000","last_peered":"2024-03-28T16:50:59.357686+0000","last_clean":"2024-03-28T16:50:59.357686+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:50:59.357686+0000","last_undegraded":"2024-03-28T16:50:59.357686+0000","last_fullsized":"2024-03-28T16:50:59.357686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.357393+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:54.820495+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'249","reported_seq":342,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.231850+0000","last_change":"2024-03-28T16:51:08.231850+0000","last_active":"2024-03-28T16:51:08.231850+0000","last_peered":"2024-03-28T16:51:08.231850+0000","last_clean":"2024-03-28T16:51:08.231850+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:51:08.231850+0000","last_undegraded":"2024-03-28T16:51:08.231850+0000","last_fullsized":"2024-03-28T16:51:08.231850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.231583+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:26.966593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.350955+0000","last_change":"2024-03-28T16:51:04.350955+0000","last_active":"2024-03-28T16:51:04.350955+0000","last_peered":"2024-03-28T16:51:04.350955+0000","last_clean":"2024-03-28T16:51:04.350955+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:51:04.350955+0000","last_undegraded":"2024-03-28T16:51:04.350955+0000","last_fullsized":"2024-03-28T16:51:04.350955+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.350588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:52.331897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.382333+0000","last_change":"2024-03-28T16:51:14.382333+0000","last_active":"2024-03-28T16:51:14.382333+0000","last_peered":"2024-03-28T16:51:14.382333+0000","last_clean":"2024-03-28T16:51:14.382333+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:51:14.382333+0000","last_undegraded":"2024-03-28T16:51:14.382333+0000","last_fullsized":"2024-03-28T16:51:14.382333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.382083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:28.408578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.082705+0000","last_change":"2024-03-28T16:51:35.082806+0000","last_active":"2024-03-28T16:51:35.082705+0000","last_peered":"2024-03-28T16:51:35.082705+0000","last_clean":"2024-03-28T16:51:35.082705+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:51:35.082705+0000","last_undegraded":"2024-03-28T16:51:35.082705+0000","last_fullsized":"2024-03-28T16:51:35.082705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:35.082491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:35.082491+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.082491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:11.308208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.124265+0000","last_change":"2024-03-28T16:50:49.124265+0000","last_active":"2024-03-28T16:50:49.124265+0000","last_peered":"2024-03-28T16:50:49.124265+0000","last_clean":"2024-03-28T16:50:49.124265+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:50:49.124265+0000","last_undegraded":"2024-03-28T16:50:49.124265+0000","last_fullsized":"2024-03-28T16:50:49.124265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.124014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.365150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.425199+0000","last_change":"2024-03-28T16:50:57.425199+0000","last_active":"2024-03-28T16:50:57.425199+0000","last_peered":"2024-03-28T16:50:57.425199+0000","last_clean":"2024-03-28T16:50:57.425199+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:57.425199+0000","last_undegraded":"2024-03-28T16:50:57.425199+0000","last_fullsized":"2024-03-28T16:50:57.425199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.424946+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:19.309481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'249","reported_seq":357,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.296501+0000","last_change":"2024-03-28T16:51:09.296501+0000","last_active":"2024-03-28T16:51:09.296501+0000","last_peered":"2024-03-28T16:51:09.296501+0000","last_clean":"2024-03-28T16:51:09.296501+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:51:09.296501+0000","last_undegraded":"2024-03-28T16:51:09.296501+0000","last_fullsized":"2024-03-28T16:51:09.296501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.296245+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:57.394089+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.285771+0000","last_change":"2024-03-28T16:51:06.285771+0000","last_active":"2024-03-28T16:51:06.285771+0000","last_peered":"2024-03-28T16:51:06.285771+0000","last_clean":"2024-03-28T16:51:06.285771+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:51:06.285771+0000","last_undegraded":"2024-03-28T16:51:06.285771+0000","last_fullsized":"2024-03-28T16:51:06.285771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.285504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:11.465640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.389203+0000","last_change":"2024-03-28T16:51:17.389264+0000","last_active":"2024-03-28T16:51:17.389203+0000","last_peered":"2024-03-28T16:51:17.389203+0000","last_clean":"2024-03-28T16:51:17.389203+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:51:17.389203+0000","last_undegraded":"2024-03-28T16:51:17.389203+0000","last_fullsized":"2024-03-28T16:51:17.389203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.389016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:59.364938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.302414+0000","last_change":"2024-03-28T16:51:00.302414+0000","last_active":"2024-03-28T16:51:00.302414+0000","last_peered":"2024-03-28T16:51:00.302414+0000","last_clean":"2024-03-28T16:51:00.302414+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:51:00.302414+0000","last_undegraded":"2024-03-28T16:51:00.302414+0000","last_fullsized":"2024-03-28T16:51:00.302414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.302183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:46.487988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'49","reported_seq":109,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.444852+0000","last_change":"2024-03-28T16:50:55.444852+0000","last_active":"2024-03-28T16:50:55.444852+0000","last_peered":"2024-03-28T16:50:55.444852+0000","last_clean":"2024-03-28T16:50:55.444852+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:55.444852+0000","last_undegraded":"2024-03-28T16:50:55.444852+0000","last_fullsized":"2024-03-28T16:50:55.444852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'49","last_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_deep_scrub":"31'49","last_deep_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.444545+0000","objects_scrubbed":9,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:56.707599+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.195172+0000","last_change":"2024-03-28T16:50:53.195172+0000","last_active":"2024-03-28T16:50:53.195172+0000","last_peered":"2024-03-28T16:50:53.195172+0000","last_clean":"2024-03-28T16:50:53.195172+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:50:53.195172+0000","last_undegraded":"2024-03-28T16:50:53.195172+0000","last_fullsized":"2024-03-28T16:50:53.195172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.194894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:05.863228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.397864+0000","last_change":"2024-03-28T16:50:56.397864+0000","last_active":"2024-03-28T16:50:56.397864+0000","last_peered":"2024-03-28T16:50:56.397864+0000","last_clean":"2024-03-28T16:50:56.397864+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:56.397864+0000","last_undegraded":"2024-03-28T16:50:56.397864+0000","last_fullsized":"2024-03-28T16:50:56.397864+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.397465+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:26.995782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'24","reported_seq":112,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.267353+0000","last_change":"2024-03-28T16:51:17.267421+0000","last_active":"2024-03-28T16:51:17.267353+0000","last_peered":"2024-03-28T16:51:17.267353+0000","last_clean":"2024-03-28T16:51:17.267353+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:51:17.267353+0000","last_undegraded":"2024-03-28T16:51:17.267353+0000","last_fullsized":"2024-03-28T16:51:17.267353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'24","last_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_deep_scrub":"31'24","last_deep_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.267112+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:31.293607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.358051+0000","last_change":"2024-03-28T16:51:15.358174+0000","last_active":"2024-03-28T16:51:15.358051+0000","last_peered":"2024-03-28T16:51:15.358051+0000","last_clean":"2024-03-28T16:51:15.358051+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:51:15.358051+0000","last_undegraded":"2024-03-28T16:51:15.358051+0000","last_fullsized":"2024-03-28T16:51:15.358051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.357818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:41.892555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.371282+0000","last_change":"2024-03-28T16:51:02.371282+0000","last_active":"2024-03-28T16:51:02.371282+0000","last_peered":"2024-03-28T16:51:02.371282+0000","last_clean":"2024-03-28T16:51:02.371282+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:51:02.371282+0000","last_undegraded":"2024-03-28T16:51:02.371282+0000","last_fullsized":"2024-03-28T16:51:02.371282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.371035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:20.148230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.191604+0000","last_change":"2024-03-28T16:51:11.191604+0000","last_active":"2024-03-28T16:51:11.191604+0000","last_peered":"2024-03-28T16:51:11.191604+0000","last_clean":"2024-03-28T16:51:11.191604+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:51:11.191604+0000","last_undegraded":"2024-03-28T16:51:11.191604+0000","last_fullsized":"2024-03-28T16:51:11.191604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.191343+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:47:25.401107+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.183048+0000","last_change":"2024-03-28T16:50:51.183048+0000","last_active":"2024-03-28T16:50:51.183048+0000","last_peered":"2024-03-28T16:50:51.183048+0000","last_clean":"2024-03-28T16:50:51.183048+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:50:51.183048+0000","last_undegraded":"2024-03-28T16:50:51.183048+0000","last_fullsized":"2024-03-28T16:50:51.183048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.182678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:48.071769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'4","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.320355+0000","last_change":"2024-03-28T16:51:05.320355+0000","last_active":"2024-03-28T16:51:05.320355+0000","last_peered":"2024-03-28T16:51:05.320355+0000","last_clean":"2024-03-28T16:51:05.320355+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:51:05.320355+0000","last_undegraded":"2024-03-28T16:51:05.320355+0000","last_fullsized":"2024-03-28T16:51:05.320355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.320076+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:34.995213+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.314961+0000","last_change":"2024-03-28T16:51:14.315025+0000","last_active":"2024-03-28T16:51:14.314961+0000","last_peered":"2024-03-28T16:51:14.314961+0000","last_clean":"2024-03-28T16:51:14.314961+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:51:14.314961+0000","last_undegraded":"2024-03-28T16:51:14.314961+0000","last_fullsized":"2024-03-28T16:51:14.314961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.314772+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:49.109332+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.126840+0000","last_change":"2024-03-28T16:51:38.126897+0000","last_active":"2024-03-28T16:51:38.126840+0000","last_peered":"2024-03-28T16:51:38.126840+0000","last_clean":"2024-03-28T16:51:38.126840+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:51:38.126840+0000","last_undegraded":"2024-03-28T16:51:38.126840+0000","last_fullsized":"2024-03-28T16:51:38.126840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.126656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.126656+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.126656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:49.967866+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.352892+0000","last_change":"2024-03-28T16:51:11.352892+0000","last_active":"2024-03-28T16:51:11.352892+0000","last_peered":"2024-03-28T16:51:11.352892+0000","last_clean":"2024-03-28T16:51:11.352892+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:51:11.352892+0000","last_undegraded":"2024-03-28T16:51:11.352892+0000","last_fullsized":"2024-03-28T16:51:11.352892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.352650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:16.271950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.096269+0000","last_change":"2024-03-28T16:51:18.096365+0000","last_active":"2024-03-28T16:51:18.096269+0000","last_peered":"2024-03-28T16:51:18.096269+0000","last_clean":"2024-03-28T16:51:18.096269+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:51:18.096269+0000","last_undegraded":"2024-03-28T16:51:18.096269+0000","last_fullsized":"2024-03-28T16:51:18.096269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.096048+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:10.105136+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'249","reported_seq":352,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.171801+0000","last_change":"2024-03-28T16:50:52.171801+0000","last_active":"2024-03-28T16:50:52.171801+0000","last_peered":"2024-03-28T16:50:52.171801+0000","last_clean":"2024-03-28T16:50:52.171801+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:52.171801+0000","last_undegraded":"2024-03-28T16:50:52.171801+0000","last_fullsized":"2024-03-28T16:50:52.171801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.171465+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:19.659817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.533038+0000","last_change":"2024-03-28T16:49:04.743207+0000","last_active":"2024-03-28T16:50:45.533038+0000","last_peered":"2024-03-28T16:50:45.533038+0000","last_clean":"2024-03-28T16:50:45.533038+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:50:45.533038+0000","last_undegraded":"2024-03-28T16:50:45.533038+0000","last_fullsized":"2024-03-28T16:50:45.533038+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.394496+0000","last_change":"2024-03-28T16:51:18.394559+0000","last_active":"2024-03-28T16:51:18.394496+0000","last_peered":"2024-03-28T16:51:18.394496+0000","last_clean":"2024-03-28T16:51:18.394496+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:51:18.394496+0000","last_undegraded":"2024-03-28T16:51:18.394496+0000","last_fullsized":"2024-03-28T16:51:18.394496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.394309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:06.492152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871317+0000","last_change":"2024-03-28T16:48:54.658093+0000","last_active":"2024-03-28T16:50:46.871317+0000","last_peered":"2024-03-28T16:50:46.871317+0000","last_clean":"2024-03-28T16:50:46.871317+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:50:46.871317+0000","last_undegraded":"2024-03-28T16:50:46.871317+0000","last_fullsized":"2024-03-28T16:50:46.871317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.386277+0000","last_change":"2024-03-28T16:51:12.386390+0000","last_active":"2024-03-28T16:51:12.386277+0000","last_peered":"2024-03-28T16:51:12.386277+0000","last_clean":"2024-03-28T16:51:12.386277+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:51:12.386277+0000","last_undegraded":"2024-03-28T16:51:12.386277+0000","last_fullsized":"2024-03-28T16:51:12.386277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.386042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:21.605218+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'61","reported_seq":134,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.103088+0000","last_change":"2024-03-28T16:51:14.103172+0000","last_active":"2024-03-28T16:51:14.103088+0000","last_peered":"2024-03-28T16:51:14.103088+0000","last_clean":"2024-03-28T16:51:14.103088+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:51:14.103088+0000","last_undegraded":"2024-03-28T16:51:14.103088+0000","last_fullsized":"2024-03-28T16:51:14.103088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'61","last_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_deep_scrub":"31'61","last_deep_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.102888+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:13:35.860974+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.181042+0000","last_change":"2024-03-28T16:50:54.181042+0000","last_active":"2024-03-28T16:50:54.181042+0000","last_peered":"2024-03-28T16:50:54.181042+0000","last_clean":"2024-03-28T16:50:54.181042+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:50:54.181042+0000","last_undegraded":"2024-03-28T16:50:54.181042+0000","last_fullsized":"2024-03-28T16:50:54.181042+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.180810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:17.080003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.393450+0000","last_change":"2024-03-28T16:51:13.393450+0000","last_active":"2024-03-28T16:51:13.393450+0000","last_peered":"2024-03-28T16:51:13.393450+0000","last_clean":"2024-03-28T16:51:13.393450+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:51:13.393450+0000","last_undegraded":"2024-03-28T16:51:13.393450+0000","last_fullsized":"2024-03-28T16:51:13.393450+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.393212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:22.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.203924+0000","last_change":"2024-03-28T16:51:27.203985+0000","last_active":"2024-03-28T16:51:27.203924+0000","last_peered":"2024-03-28T16:51:27.203924+0000","last_clean":"2024-03-28T16:51:27.203924+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:51:27.203924+0000","last_undegraded":"2024-03-28T16:51:27.203924+0000","last_fullsized":"2024-03-28T16:51:27.203924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:27.203737+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:27.203737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.203737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:14.186013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.551323+0000","last_change":"2024-03-28T16:51:28.551377+0000","last_active":"2024-03-28T16:51:28.551323+0000","last_peered":"2024-03-28T16:51:28.551323+0000","last_clean":"2024-03-28T16:51:28.551323+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:51:28.551323+0000","last_undegraded":"2024-03-28T16:51:28.551323+0000","last_fullsized":"2024-03-28T16:51:28.551323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:28.551113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:28.551113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.551113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:24.518811+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.365035+0000","last_change":"2024-03-28T16:51:05.365035+0000","last_active":"2024-03-28T16:51:05.365035+0000","last_peered":"2024-03-28T16:51:05.365035+0000","last_clean":"2024-03-28T16:51:05.365035+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:51:05.365035+0000","last_undegraded":"2024-03-28T16:51:05.365035+0000","last_fullsized":"2024-03-28T16:51:05.365035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.364765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:50.121796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'37","reported_seq":93,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.221214+0000","last_change":"2024-03-28T16:50:51.221214+0000","last_active":"2024-03-28T16:50:51.221214+0000","last_peered":"2024-03-28T16:50:51.221214+0000","last_clean":"2024-03-28T16:50:51.221214+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:51.221214+0000","last_undegraded":"2024-03-28T16:50:51.221214+0000","last_fullsized":"2024-03-28T16:50:51.221214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'37","last_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_deep_scrub":"31'37","last_deep_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.220960+0000","objects_scrubbed":5,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:47.323997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.275109+0000","last_change":"2024-03-28T16:51:07.275109+0000","last_active":"2024-03-28T16:51:07.275109+0000","last_peered":"2024-03-28T16:51:07.275109+0000","last_clean":"2024-03-28T16:51:07.275109+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:51:07.275109+0000","last_undegraded":"2024-03-28T16:51:07.275109+0000","last_fullsized":"2024-03-28T16:51:07.275109+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.274868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:03.242566+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.076599+0000","last_change":"2024-03-28T16:51:37.076651+0000","last_active":"2024-03-28T16:51:37.076599+0000","last_peered":"2024-03-28T16:51:37.076599+0000","last_clean":"2024-03-28T16:51:37.076599+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:51:37.076599+0000","last_undegraded":"2024-03-28T16:51:37.076599+0000","last_fullsized":"2024-03-28T16:51:37.076599+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.076411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.076411+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.076411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:47.198210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.193692+0000","last_change":"2024-03-28T16:51:00.193692+0000","last_active":"2024-03-28T16:51:00.193692+0000","last_peered":"2024-03-28T16:51:00.193692+0000","last_clean":"2024-03-28T16:51:00.193692+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:51:00.193692+0000","last_undegraded":"2024-03-28T16:51:00.193692+0000","last_fullsized":"2024-03-28T16:51:00.193692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.193439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:35.074661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871223+0000","last_change":"2024-03-28T16:48:54.673299+0000","last_active":"2024-03-28T16:50:46.871223+0000","last_peered":"2024-03-28T16:50:46.871223+0000","last_clean":"2024-03-28T16:50:46.871223+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:50:46.871223+0000","last_undegraded":"2024-03-28T16:50:46.871223+0000","last_fullsized":"2024-03-28T16:50:46.871223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.345768+0000","last_change":"2024-03-28T16:51:10.345768+0000","last_active":"2024-03-28T16:51:10.345768+0000","last_peered":"2024-03-28T16:51:10.345768+0000","last_clean":"2024-03-28T16:51:10.345768+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:51:10.345768+0000","last_undegraded":"2024-03-28T16:51:10.345768+0000","last_fullsized":"2024-03-28T16:51:10.345768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.345522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:45.930075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.389452+0000","last_change":"2024-03-28T16:51:01.389452+0000","last_active":"2024-03-28T16:51:01.389452+0000","last_peered":"2024-03-28T16:51:01.389452+0000","last_clean":"2024-03-28T16:51:01.389452+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:51:01.389452+0000","last_undegraded":"2024-03-28T16:51:01.389452+0000","last_fullsized":"2024-03-28T16:51:01.389452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.389173+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:28.877525+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'249","reported_seq":335,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.428834+0000","last_change":"2024-03-28T16:50:47.428834+0000","last_active":"2024-03-28T16:50:47.428834+0000","last_peered":"2024-03-28T16:50:47.428834+0000","last_clean":"2024-03-28T16:50:47.428834+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:47.428834+0000","last_undegraded":"2024-03-28T16:50:47.428834+0000","last_fullsized":"2024-03-28T16:50:47.428834+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.428456+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:01.404074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'13","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532760+0000","last_change":"2024-03-28T16:49:04.743257+0000","last_active":"2024-03-28T16:50:45.532760+0000","last_peered":"2024-03-28T16:50:45.532760+0000","last_clean":"2024-03-28T16:50:45.532760+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:50:45.532760+0000","last_undegraded":"2024-03-28T16:50:45.532760+0000","last_fullsized":"2024-03-28T16:50:45.532760+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.337241+0000","last_change":"2024-03-28T16:51:22.337362+0000","last_active":"2024-03-28T16:51:22.337241+0000","last_peered":"2024-03-28T16:51:22.337241+0000","last_clean":"2024-03-28T16:51:22.337241+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:51:22.337241+0000","last_undegraded":"2024-03-28T16:51:22.337241+0000","last_fullsized":"2024-03-28T16:51:22.337241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.336956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:31.109615+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871187+0000","last_change":"2024-03-28T16:48:54.664571+0000","last_active":"2024-03-28T16:50:46.871187+0000","last_peered":"2024-03-28T16:50:46.871187+0000","last_clean":"2024-03-28T16:50:46.871187+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:50:46.871187+0000","last_undegraded":"2024-03-28T16:50:46.871187+0000","last_fullsized":"2024-03-28T16:50:46.871187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'3","reported_seq":384,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.986155+0000","last_change":"2024-03-28T16:49:37.622013+0000","last_active":"2024-03-28T16:50:45.986155+0000","last_peered":"2024-03-28T16:50:45.986155+0000","last_clean":"2024-03-28T16:50:45.986155+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:50:45.986155+0000","last_undegraded":"2024-03-28T16:50:45.986155+0000","last_fullsized":"2024-03-28T16:50:45.986155+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:57.649745+0000","last_clean_scrub_stamp":"2024-03-28T16:48:57.649745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":340,"num_read_kb":340,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.144955+0000","last_change":"2024-03-28T16:51:12.144955+0000","last_active":"2024-03-28T16:51:12.144955+0000","last_peered":"2024-03-28T16:51:12.144955+0000","last_clean":"2024-03-28T16:51:12.144955+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:51:12.144955+0000","last_undegraded":"2024-03-28T16:51:12.144955+0000","last_fullsized":"2024-03-28T16:51:12.144955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.144489+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:36.042267+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'249","reported_seq":358,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.109690+0000","last_change":"2024-03-28T16:50:48.109690+0000","last_active":"2024-03-28T16:50:48.109690+0000","last_peered":"2024-03-28T16:50:48.109690+0000","last_clean":"2024-03-28T16:50:48.109690+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:48.109690+0000","last_undegraded":"2024-03-28T16:50:48.109690+0000","last_fullsized":"2024-03-28T16:50:48.109690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.109433+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:49.372487+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.412356+0000","last_change":"2024-03-28T16:51:08.412356+0000","last_active":"2024-03-28T16:51:08.412356+0000","last_peered":"2024-03-28T16:51:08.412356+0000","last_clean":"2024-03-28T16:51:08.412356+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:51:08.412356+0000","last_undegraded":"2024-03-28T16:51:08.412356+0000","last_fullsized":"2024-03-28T16:51:08.412356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.412104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:37.251504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.269854+0000","last_change":"2024-03-28T16:51:25.269969+0000","last_active":"2024-03-28T16:51:25.269854+0000","last_peered":"2024-03-28T16:51:25.269854+0000","last_clean":"2024-03-28T16:51:25.269854+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:51:25.269854+0000","last_undegraded":"2024-03-28T16:51:25.269854+0000","last_fullsized":"2024-03-28T16:51:25.269854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.269642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:53:12.659384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870991+0000","last_change":"2024-03-28T16:48:54.676487+0000","last_active":"2024-03-28T16:50:46.870991+0000","last_peered":"2024-03-28T16:50:46.870991+0000","last_clean":"2024-03-28T16:50:46.870991+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:50:46.870991+0000","last_undegraded":"2024-03-28T16:50:46.870991+0000","last_fullsized":"2024-03-28T16:50:46.870991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.165481+0000","last_change":"2024-03-28T16:50:48.165481+0000","last_active":"2024-03-28T16:50:48.165481+0000","last_peered":"2024-03-28T16:50:48.165481+0000","last_clean":"2024-03-28T16:50:48.165481+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:50:48.165481+0000","last_undegraded":"2024-03-28T16:50:48.165481+0000","last_fullsized":"2024-03-28T16:50:48.165481+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.165239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:40:14.898173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'46","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.272646+0000","last_change":"2024-03-28T16:50:56.272646+0000","last_active":"2024-03-28T16:50:56.272646+0000","last_peered":"2024-03-28T16:50:56.272646+0000","last_clean":"2024-03-28T16:50:56.272646+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:56.272646+0000","last_undegraded":"2024-03-28T16:50:56.272646+0000","last_fullsized":"2024-03-28T16:50:56.272646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'46","last_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_deep_scrub":"31'46","last_deep_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.272353+0000","objects_scrubbed":8,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:19.171546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.431537+0000","last_change":"2024-03-28T16:50:53.431537+0000","last_active":"2024-03-28T16:50:53.431537+0000","last_peered":"2024-03-28T16:50:53.431537+0000","last_clean":"2024-03-28T16:50:53.431537+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:50:53.431537+0000","last_undegraded":"2024-03-28T16:50:53.431537+0000","last_fullsized":"2024-03-28T16:50:53.431537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.431293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:35.681957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.348998+0000","last_change":"2024-03-28T16:51:01.348998+0000","last_active":"2024-03-28T16:51:01.348998+0000","last_peered":"2024-03-28T16:51:01.348998+0000","last_clean":"2024-03-28T16:51:01.348998+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:51:01.348998+0000","last_undegraded":"2024-03-28T16:51:01.348998+0000","last_fullsized":"2024-03-28T16:51:01.348998+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.348719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:30.188118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.197537+0000","last_change":"2024-03-28T16:51:03.197596+0000","last_active":"2024-03-28T16:51:03.197537+0000","last_peered":"2024-03-28T16:51:03.197537+0000","last_clean":"2024-03-28T16:51:03.197537+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:51:03.197537+0000","last_undegraded":"2024-03-28T16:51:03.197537+0000","last_fullsized":"2024-03-28T16:51:03.197537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.197346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:04:49.417548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"31'119","reported_seq":210,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.205423+0000","last_change":"2024-03-28T16:51:31.205497+0000","last_active":"2024-03-28T16:51:31.205423+0000","last_peered":"2024-03-28T16:51:31.205423+0000","last_clean":"2024-03-28T16:51:31.205423+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:51:31.205423+0000","last_undegraded":"2024-03-28T16:51:31.205423+0000","last_fullsized":"2024-03-28T16:51:31.205423+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'119","last_scrub_stamp":"2024-03-28T16:51:31.205230+0000","last_deep_scrub":"31'119","last_deep_scrub_stamp":"2024-03-28T16:51:31.205230+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.205230+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:47:05.795250+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870781+0000","last_change":"2024-03-28T16:48:48.625723+0000","last_active":"2024-03-28T16:50:46.870781+0000","last_peered":"2024-03-28T16:50:46.870781+0000","last_clean":"2024-03-28T16:50:46.870781+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:50:46.870781+0000","last_undegraded":"2024-03-28T16:50:46.870781+0000","last_fullsized":"2024-03-28T16:50:46.870781+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.064738+0000","last_change":"2024-03-28T16:51:41.064875+0000","last_active":"2024-03-28T16:51:41.064738+0000","last_peered":"2024-03-28T16:51:41.064738+0000","last_clean":"2024-03-28T16:51:41.064738+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:51:41.064738+0000","last_undegraded":"2024-03-28T16:51:41.064738+0000","last_fullsized":"2024-03-28T16:51:41.064738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:41.064504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:41.064504+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.064504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:51.380462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.295412+0000","last_change":"2024-03-28T16:51:03.295412+0000","last_active":"2024-03-28T16:51:03.295412+0000","last_peered":"2024-03-28T16:51:03.295412+0000","last_clean":"2024-03-28T16:51:03.295412+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:51:03.295412+0000","last_undegraded":"2024-03-28T16:51:03.295412+0000","last_fullsized":"2024-03-28T16:51:03.295412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.295139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:08.548996+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'81","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.160163+0000","last_change":"2024-03-28T16:50:53.160163+0000","last_active":"2024-03-28T16:50:53.160163+0000","last_peered":"2024-03-28T16:50:53.160163+0000","last_clean":"2024-03-28T16:50:53.160163+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:53.160163+0000","last_undegraded":"2024-03-28T16:50:53.160163+0000","last_fullsized":"2024-03-28T16:50:53.160163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'81","last_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_deep_scrub":"31'81","last_deep_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.159787+0000","objects_scrubbed":5,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:32.543735+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.290671+0000","last_change":"2024-03-28T16:50:57.290671+0000","last_active":"2024-03-28T16:50:57.290671+0000","last_peered":"2024-03-28T16:50:57.290671+0000","last_clean":"2024-03-28T16:50:57.290671+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:50:57.290671+0000","last_undegraded":"2024-03-28T16:50:57.290671+0000","last_fullsized":"2024-03-28T16:50:57.290671+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.290416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:32.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'53","reported_seq":125,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.247075+0000","last_change":"2024-03-28T16:51:35.247207+0000","last_active":"2024-03-28T16:51:35.247075+0000","last_peered":"2024-03-28T16:51:35.247075+0000","last_clean":"2024-03-28T16:51:35.247075+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:51:35.247075+0000","last_undegraded":"2024-03-28T16:51:35.247075+0000","last_fullsized":"2024-03-28T16:51:35.247075+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'53","last_scrub_stamp":"2024-03-28T16:51:35.246864+0000","last_deep_scrub":"31'53","last_deep_scrub_stamp":"2024-03-28T16:51:35.246864+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.246864+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:53.626119+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'12","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.204552+0000","last_change":"2024-03-28T16:51:40.204617+0000","last_active":"2024-03-28T16:51:40.204552+0000","last_peered":"2024-03-28T16:51:40.204552+0000","last_clean":"2024-03-28T16:51:40.204552+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:51:40.204552+0000","last_undegraded":"2024-03-28T16:51:40.204552+0000","last_fullsized":"2024-03-28T16:51:40.204552+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:40.204359+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:40.204359+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.204359+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:32.212954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870684+0000","last_change":"2024-03-28T16:48:48.624554+0000","last_active":"2024-03-28T16:50:46.870684+0000","last_peered":"2024-03-28T16:50:46.870684+0000","last_clean":"2024-03-28T16:50:46.870684+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:50:46.870684+0000","last_undegraded":"2024-03-28T16:50:46.870684+0000","last_fullsized":"2024-03-28T16:50:46.870684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.432079+0000","last_change":"2024-03-28T16:51:41.432131+0000","last_active":"2024-03-28T16:51:41.432079+0000","last_peered":"2024-03-28T16:51:41.432079+0000","last_clean":"2024-03-28T16:51:41.432079+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:51:41.432079+0000","last_undegraded":"2024-03-28T16:51:41.432079+0000","last_fullsized":"2024-03-28T16:51:41.432079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:41.431882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:41.431882+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.431882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:48.943852+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.191091+0000","last_change":"2024-03-28T16:50:47.191091+0000","last_active":"2024-03-28T16:50:47.191091+0000","last_peered":"2024-03-28T16:50:47.191091+0000","last_clean":"2024-03-28T16:50:47.191091+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:50:47.191091+0000","last_undegraded":"2024-03-28T16:50:47.191091+0000","last_fullsized":"2024-03-28T16:50:47.191091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.190844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:09.946965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'260","reported_seq":455,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.259946+0000","last_change":"2024-03-28T16:50:54.259946+0000","last_active":"2024-03-28T16:50:54.259946+0000","last_peered":"2024-03-28T16:50:54.259946+0000","last_clean":"2024-03-28T16:50:54.259946+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:54.259946+0000","last_undegraded":"2024-03-28T16:50:54.259946+0000","last_fullsized":"2024-03-28T16:50:54.259946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'260","last_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_deep_scrub":"31'260","last_deep_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.259667+0000","objects_scrubbed":10,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:21.748018+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":342,"num_read_kb":327,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.065783+0000","last_change":"2024-03-28T16:51:21.065839+0000","last_active":"2024-03-28T16:51:21.065783+0000","last_peered":"2024-03-28T16:51:21.065783+0000","last_clean":"2024-03-28T16:51:21.065783+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:51:21.065783+0000","last_undegraded":"2024-03-28T16:51:21.065783+0000","last_fullsized":"2024-03-28T16:51:21.065783+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.065552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:21.706323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'5","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:42.068714+0000","last_change":"2024-03-28T16:51:42.068792+0000","last_active":"2024-03-28T16:51:42.068714+0000","last_peered":"2024-03-28T16:51:42.068714+0000","last_clean":"2024-03-28T16:51:42.068714+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:51:42.068714+0000","last_undegraded":"2024-03-28T16:51:42.068714+0000","last_fullsized":"2024-03-28T16:51:42.068714+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'5","last_scrub_stamp":"2024-03-28T16:51:42.068482+0000","last_deep_scrub":"31'5","last_deep_scrub_stamp":"2024-03-28T16:51:42.068482+0000","last_clean_scrub_stamp":"2024-03-28T16:51:42.068482+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:11.682779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.532252+0000","last_change":"2024-03-28T16:48:56.689836+0000","last_active":"2024-03-28T16:50:45.532252+0000","last_peered":"2024-03-28T16:50:45.532252+0000","last_clean":"2024-03-28T16:50:45.532252+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:50:45.532252+0000","last_undegraded":"2024-03-28T16:50:45.532252+0000","last_fullsized":"2024-03-28T16:50:45.532252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:55.634750+0000","last_clean_scrub_stamp":"2024-03-28T16:48:55.634750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.236189+0000","last_change":"2024-03-28T16:51:26.236274+0000","last_active":"2024-03-28T16:51:26.236189+0000","last_peered":"2024-03-28T16:51:26.236189+0000","last_clean":"2024-03-28T16:51:26.236189+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:51:26.236189+0000","last_undegraded":"2024-03-28T16:51:26.236189+0000","last_fullsized":"2024-03-28T16:51:26.236189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:26.235994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:26.235994+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.235994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:43.654590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.531810+0000","last_change":"2024-03-28T16:48:54.663058+0000","last_active":"2024-03-28T16:50:45.531810+0000","last_peered":"2024-03-28T16:50:45.531810+0000","last_clean":"2024-03-28T16:50:45.531810+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:50:45.531810+0000","last_undegraded":"2024-03-28T16:50:45.531810+0000","last_fullsized":"2024-03-28T16:50:45.531810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.414766+0000","last_change":"2024-03-28T16:51:07.414766+0000","last_active":"2024-03-28T16:51:07.414766+0000","last_peered":"2024-03-28T16:51:07.414766+0000","last_clean":"2024-03-28T16:51:07.414766+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:51:07.414766+0000","last_undegraded":"2024-03-28T16:51:07.414766+0000","last_fullsized":"2024-03-28T16:51:07.414766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.414513+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:53.600318+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'23","reported_seq":82,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.409006+0000","last_change":"2024-03-28T16:50:54.409006+0000","last_active":"2024-03-28T16:50:54.409006+0000","last_peered":"2024-03-28T16:50:54.409006+0000","last_clean":"2024-03-28T16:50:54.409006+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:54.409006+0000","last_undegraded":"2024-03-28T16:50:54.409006+0000","last_fullsized":"2024-03-28T16:50:54.409006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'23","last_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_deep_scrub":"31'23","last_deep_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.408718+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:56.786363+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.341170+0000","last_change":"2024-03-28T16:51:01.341170+0000","last_active":"2024-03-28T16:51:01.341170+0000","last_peered":"2024-03-28T16:51:01.341170+0000","last_clean":"2024-03-28T16:51:01.341170+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:51:01.341170+0000","last_undegraded":"2024-03-28T16:51:01.341170+0000","last_fullsized":"2024-03-28T16:51:01.341170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.340925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:06.594781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874154+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:50:46.874154+0000","last_peered":"2024-03-28T16:50:46.874154+0000","last_clean":"2024-03-28T16:50:46.874154+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:50:46.874154+0000","last_undegraded":"2024-03-28T16:50:46.874154+0000","last_fullsized":"2024-03-28T16:50:46.874154+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.501412+0000","last_change":"2024-03-28T16:51:33.501547+0000","last_active":"2024-03-28T16:51:33.501412+0000","last_peered":"2024-03-28T16:51:33.501412+0000","last_clean":"2024-03-28T16:51:33.501412+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:51:33.501412+0000","last_undegraded":"2024-03-28T16:51:33.501412+0000","last_fullsized":"2024-03-28T16:51:33.501412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.501165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.501165+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.501165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:09.726882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870800+0000","last_change":"2024-03-28T16:48:48.624388+0000","last_active":"2024-03-28T16:50:46.870800+0000","last_peered":"2024-03-28T16:50:46.870800+0000","last_clean":"2024-03-28T16:50:46.870800+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:50:46.870800+0000","last_undegraded":"2024-03-28T16:50:46.870800+0000","last_fullsized":"2024-03-28T16:50:46.870800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.385902+0000","last_change":"2024-03-28T16:51:19.386018+0000","last_active":"2024-03-28T16:51:19.385902+0000","last_peered":"2024-03-28T16:51:19.385902+0000","last_clean":"2024-03-28T16:51:19.385902+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:51:19.385902+0000","last_undegraded":"2024-03-28T16:51:19.385902+0000","last_fullsized":"2024-03-28T16:51:19.385902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.385614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:54.970167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.375959+0000","last_change":"2024-03-28T16:51:06.375959+0000","last_active":"2024-03-28T16:51:06.375959+0000","last_peered":"2024-03-28T16:51:06.375959+0000","last_clean":"2024-03-28T16:51:06.375959+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:51:06.375959+0000","last_undegraded":"2024-03-28T16:51:06.375959+0000","last_fullsized":"2024-03-28T16:51:06.375959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.375714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:32.910452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.195780+0000","last_change":"2024-03-28T16:50:51.195780+0000","last_active":"2024-03-28T16:50:51.195780+0000","last_peered":"2024-03-28T16:50:51.195780+0000","last_clean":"2024-03-28T16:50:51.195780+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:50:51.195780+0000","last_undegraded":"2024-03-28T16:50:51.195780+0000","last_fullsized":"2024-03-28T16:50:51.195780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'19","last_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_deep_scrub":"31'19","last_deep_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.195522+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:16.598840+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.126759+0000","last_change":"2024-03-28T16:51:25.126813+0000","last_active":"2024-03-28T16:51:25.126759+0000","last_peered":"2024-03-28T16:51:25.126759+0000","last_clean":"2024-03-28T16:51:25.126759+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:51:25.126759+0000","last_undegraded":"2024-03-28T16:51:25.126759+0000","last_fullsized":"2024-03-28T16:51:25.126759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:25.126557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:25.126557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.126557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:42.133827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.302529+0000","last_change":"2024-03-28T16:51:13.302529+0000","last_active":"2024-03-28T16:51:13.302529+0000","last_peered":"2024-03-28T16:51:13.302529+0000","last_clean":"2024-03-28T16:51:13.302529+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:51:13.302529+0000","last_undegraded":"2024-03-28T16:51:13.302529+0000","last_fullsized":"2024-03-28T16:51:13.302529+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.302200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:35.072558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":68,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.328005+0000","last_change":"2024-03-28T16:51:12.328075+0000","last_active":"2024-03-28T16:51:12.328005+0000","last_peered":"2024-03-28T16:51:12.328005+0000","last_clean":"2024-03-28T16:51:12.328005+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:51:12.328005+0000","last_undegraded":"2024-03-28T16:51:12.328005+0000","last_fullsized":"2024-03-28T16:51:12.328005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.327822+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:47.912375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.230294+0000","last_change":"2024-03-28T16:51:30.230347+0000","last_active":"2024-03-28T16:51:30.230294+0000","last_peered":"2024-03-28T16:51:30.230294+0000","last_clean":"2024-03-28T16:51:30.230294+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:51:30.230294+0000","last_undegraded":"2024-03-28T16:51:30.230294+0000","last_fullsized":"2024-03-28T16:51:30.230294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.230107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.230107+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.230107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:06:05.111329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.444458+0000","last_change":"2024-03-28T16:51:36.444515+0000","last_active":"2024-03-28T16:51:36.444458+0000","last_peered":"2024-03-28T16:51:36.444458+0000","last_clean":"2024-03-28T16:51:36.444458+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:51:36.444458+0000","last_undegraded":"2024-03-28T16:51:36.444458+0000","last_fullsized":"2024-03-28T16:51:36.444458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.444268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.444268+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.444268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:33.466795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.355429+0000","last_change":"2024-03-28T16:51:03.355429+0000","last_active":"2024-03-28T16:51:03.355429+0000","last_peered":"2024-03-28T16:51:03.355429+0000","last_clean":"2024-03-28T16:51:03.355429+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:51:03.355429+0000","last_undegraded":"2024-03-28T16:51:03.355429+0000","last_fullsized":"2024-03-28T16:51:03.355429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.355165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:33.030302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'69","reported_seq":145,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.306941+0000","last_change":"2024-03-28T16:50:46.306941+0000","last_active":"2024-03-28T16:50:46.306941+0000","last_peered":"2024-03-28T16:50:46.306941+0000","last_clean":"2024-03-28T16:50:46.306941+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:46.306941+0000","last_undegraded":"2024-03-28T16:50:46.306941+0000","last_fullsized":"2024-03-28T16:50:46.306941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'69","last_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_deep_scrub":"31'69","last_deep_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.306710+0000","objects_scrubbed":8,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:28.604401+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.441291+0000","last_change":"2024-03-28T16:50:51.441291+0000","last_active":"2024-03-28T16:50:51.441291+0000","last_peered":"2024-03-28T16:50:51.441291+0000","last_clean":"2024-03-28T16:50:51.441291+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:50:51.441291+0000","last_undegraded":"2024-03-28T16:50:51.441291+0000","last_fullsized":"2024-03-28T16:50:51.441291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.441051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:13:01.864029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'9","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.533082+0000","last_change":"2024-03-28T16:49:04.741551+0000","last_active":"2024-03-28T16:50:45.533082+0000","last_peered":"2024-03-28T16:50:45.533082+0000","last_clean":"2024-03-28T16:50:45.533082+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:50:45.533082+0000","last_undegraded":"2024-03-28T16:50:45.533082+0000","last_fullsized":"2024-03-28T16:50:45.533082+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.006930+0000","last_change":"2024-03-28T16:51:39.006995+0000","last_active":"2024-03-28T16:51:39.006930+0000","last_peered":"2024-03-28T16:51:39.006930+0000","last_clean":"2024-03-28T16:51:39.006930+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:51:39.006930+0000","last_undegraded":"2024-03-28T16:51:39.006930+0000","last_fullsized":"2024-03-28T16:51:39.006930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:39.006741+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:39.006741+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.006741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:46.985028+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.491771+0000","last_change":"2024-03-28T16:51:34.491822+0000","last_active":"2024-03-28T16:51:34.491771+0000","last_peered":"2024-03-28T16:51:34.491771+0000","last_clean":"2024-03-28T16:51:34.491771+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:51:34.491771+0000","last_undegraded":"2024-03-28T16:51:34.491771+0000","last_fullsized":"2024-03-28T16:51:34.491771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:34.491575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:34.491575+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.491575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:59.766359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870790+0000","last_change":"2024-03-28T16:48:48.625370+0000","last_active":"2024-03-28T16:50:46.870790+0000","last_peered":"2024-03-28T16:50:46.870790+0000","last_clean":"2024-03-28T16:50:46.870790+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:50:46.870790+0000","last_undegraded":"2024-03-28T16:50:46.870790+0000","last_fullsized":"2024-03-28T16:50:46.870790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.159039+0000","last_change":"2024-03-28T16:50:55.159039+0000","last_active":"2024-03-28T16:50:55.159039+0000","last_peered":"2024-03-28T16:50:55.159039+0000","last_clean":"2024-03-28T16:50:55.159039+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:50:55.159039+0000","last_undegraded":"2024-03-28T16:50:55.159039+0000","last_fullsized":"2024-03-28T16:50:55.159039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.158776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:53:10.542165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"31'21","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.252537+0000","last_change":"2024-03-28T16:50:52.252537+0000","last_active":"2024-03-28T16:50:52.252537+0000","last_peered":"2024-03-28T16:50:52.252537+0000","last_clean":"2024-03-28T16:50:52.252537+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:50:52.252537+0000","last_undegraded":"2024-03-28T16:50:52.252537+0000","last_fullsized":"2024-03-28T16:50:52.252537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'21","last_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_deep_scrub":"31'21","last_deep_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.252240+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:47.715342+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'249","reported_seq":304,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.104945+0000","last_change":"2024-03-28T16:50:46.104945+0000","last_active":"2024-03-28T16:50:46.104945+0000","last_peered":"2024-03-28T16:50:46.104945+0000","last_clean":"2024-03-28T16:50:46.104945+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:46.104945+0000","last_undegraded":"2024-03-28T16:50:46.104945+0000","last_fullsized":"2024-03-28T16:50:46.104945+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.104655+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:39.533970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874227+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:46.874227+0000","last_peered":"2024-03-28T16:50:46.874227+0000","last_clean":"2024-03-28T16:50:46.874227+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:46.874227+0000","last_undegraded":"2024-03-28T16:50:46.874227+0000","last_fullsized":"2024-03-28T16:50:46.874227+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.222398+0000","last_change":"2024-03-28T16:51:04.222527+0000","last_active":"2024-03-28T16:51:04.222398+0000","last_peered":"2024-03-28T16:51:04.222398+0000","last_clean":"2024-03-28T16:51:04.222398+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:51:04.222398+0000","last_undegraded":"2024-03-28T16:51:04.222398+0000","last_fullsized":"2024-03-28T16:51:04.222398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.222151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:13.382250+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870972+0000","last_change":"2024-03-28T16:48:54.662217+0000","last_active":"2024-03-28T16:50:46.870972+0000","last_peered":"2024-03-28T16:50:46.870972+0000","last_clean":"2024-03-28T16:50:46.870972+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:50:46.870972+0000","last_undegraded":"2024-03-28T16:50:46.870972+0000","last_fullsized":"2024-03-28T16:50:46.870972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.449142+0000","last_change":"2024-03-28T16:51:35.449195+0000","last_active":"2024-03-28T16:51:35.449142+0000","last_peered":"2024-03-28T16:51:35.449142+0000","last_clean":"2024-03-28T16:51:35.449142+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:51:35.449142+0000","last_undegraded":"2024-03-28T16:51:35.449142+0000","last_fullsized":"2024-03-28T16:51:35.449142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:35.448941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:35.448941+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.448941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:12.528913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.163965+0000","last_change":"2024-03-28T16:50:54.163965+0000","last_active":"2024-03-28T16:50:54.163965+0000","last_peered":"2024-03-28T16:50:54.163965+0000","last_clean":"2024-03-28T16:50:54.163965+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:50:54.163965+0000","last_undegraded":"2024-03-28T16:50:54.163965+0000","last_fullsized":"2024-03-28T16:50:54.163965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.163592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:27.253015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.038093+0000","last_change":"2024-03-28T16:51:16.038179+0000","last_active":"2024-03-28T16:51:16.038093+0000","last_peered":"2024-03-28T16:51:16.038093+0000","last_clean":"2024-03-28T16:51:16.038093+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:51:16.038093+0000","last_undegraded":"2024-03-28T16:51:16.038093+0000","last_fullsized":"2024-03-28T16:51:16.038093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.037881+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:44.064566+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'249","reported_seq":337,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.233041+0000","last_change":"2024-03-28T16:50:49.233041+0000","last_active":"2024-03-28T16:50:49.233041+0000","last_peered":"2024-03-28T16:50:49.233041+0000","last_clean":"2024-03-28T16:50:49.233041+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:49.233041+0000","last_undegraded":"2024-03-28T16:50:49.233041+0000","last_fullsized":"2024-03-28T16:50:49.233041+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.232798+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:57.227518+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.267309+0000","last_change":"2024-03-28T16:51:09.267309+0000","last_active":"2024-03-28T16:51:09.267309+0000","last_peered":"2024-03-28T16:51:09.267309+0000","last_clean":"2024-03-28T16:51:09.267309+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:51:09.267309+0000","last_undegraded":"2024-03-28T16:51:09.267309+0000","last_fullsized":"2024-03-28T16:51:09.267309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.267056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:58.701007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.363436+0000","last_change":"2024-03-28T16:51:16.363501+0000","last_active":"2024-03-28T16:51:16.363436+0000","last_peered":"2024-03-28T16:51:16.363436+0000","last_clean":"2024-03-28T16:51:16.363436+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:51:16.363436+0000","last_undegraded":"2024-03-28T16:51:16.363436+0000","last_fullsized":"2024-03-28T16:51:16.363436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.363254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:07:05.797204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.287919+0000","last_change":"2024-03-28T16:51:20.287986+0000","last_active":"2024-03-28T16:51:20.287919+0000","last_peered":"2024-03-28T16:51:20.287919+0000","last_clean":"2024-03-28T16:51:20.287919+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:51:20.287919+0000","last_undegraded":"2024-03-28T16:51:20.287919+0000","last_fullsized":"2024-03-28T16:51:20.287919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.287729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:53.368108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.870766+0000","last_change":"2024-03-28T16:48:49.261577+0000","last_active":"2024-03-28T16:50:46.870766+0000","last_peered":"2024-03-28T16:50:46.870766+0000","last_clean":"2024-03-28T16:50:46.870766+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:50:46.870766+0000","last_undegraded":"2024-03-28T16:50:46.870766+0000","last_fullsized":"2024-03-28T16:50:46.870766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:47.598556+0000","last_clean_scrub_stamp":"2024-03-28T16:48:47.598556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.166685+0000","last_change":"2024-03-28T16:50:56.166685+0000","last_active":"2024-03-28T16:50:56.166685+0000","last_peered":"2024-03-28T16:50:56.166685+0000","last_clean":"2024-03-28T16:50:56.166685+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:50:56.166685+0000","last_undegraded":"2024-03-28T16:50:56.166685+0000","last_fullsized":"2024-03-28T16:50:56.166685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.166428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:06.567829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'90","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.384285+0000","last_change":"2024-03-28T16:50:58.384285+0000","last_active":"2024-03-28T16:50:58.384285+0000","last_peered":"2024-03-28T16:50:58.384285+0000","last_clean":"2024-03-28T16:50:58.384285+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:58.384285+0000","last_undegraded":"2024-03-28T16:50:58.384285+0000","last_fullsized":"2024-03-28T16:50:58.384285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'90","last_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_deep_scrub":"31'90","last_deep_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.383995+0000","objects_scrubbed":8,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:54.487031+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.285325+0000","last_change":"2024-03-28T16:50:47.285325+0000","last_active":"2024-03-28T16:50:47.285325+0000","last_peered":"2024-03-28T16:50:47.285325+0000","last_clean":"2024-03-28T16:50:47.285325+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:50:47.285325+0000","last_undegraded":"2024-03-28T16:50:47.285325+0000","last_fullsized":"2024-03-28T16:50:47.285325+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.285202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:46:10.170506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.228376+0000","last_change":"2024-03-28T16:51:36.228433+0000","last_active":"2024-03-28T16:51:36.228376+0000","last_peered":"2024-03-28T16:51:36.228376+0000","last_clean":"2024-03-28T16:51:36.228376+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:51:36.228376+0000","last_undegraded":"2024-03-28T16:51:36.228376+0000","last_fullsized":"2024-03-28T16:51:36.228376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.228185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.228185+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.228185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:41.408321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.344614+0000","last_change":"2024-03-28T16:51:00.344614+0000","last_active":"2024-03-28T16:51:00.344614+0000","last_peered":"2024-03-28T16:51:00.344614+0000","last_clean":"2024-03-28T16:51:00.344614+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:51:00.344614+0000","last_undegraded":"2024-03-28T16:51:00.344614+0000","last_fullsized":"2024-03-28T16:51:00.344614+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.344363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:57.233454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.364821+0000","last_change":"2024-03-28T16:51:10.364821+0000","last_active":"2024-03-28T16:51:10.364821+0000","last_peered":"2024-03-28T16:51:10.364821+0000","last_clean":"2024-03-28T16:51:10.364821+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:51:10.364821+0000","last_undegraded":"2024-03-28T16:51:10.364821+0000","last_fullsized":"2024-03-28T16:51:10.364821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.364556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:15.618413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871399+0000","last_change":"2024-03-28T16:48:54.663088+0000","last_active":"2024-03-28T16:50:46.871399+0000","last_peered":"2024-03-28T16:50:46.871399+0000","last_clean":"2024-03-28T16:50:46.871399+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:50:46.871399+0000","last_undegraded":"2024-03-28T16:50:46.871399+0000","last_fullsized":"2024-03-28T16:50:46.871399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:45.081449+0000","last_change":"2024-03-28T16:51:45.081503+0000","last_active":"2024-03-28T16:51:45.081449+0000","last_peered":"2024-03-28T16:51:45.081449+0000","last_clean":"2024-03-28T16:51:45.081449+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:51:45.081449+0000","last_undegraded":"2024-03-28T16:51:45.081449+0000","last_fullsized":"2024-03-28T16:51:45.081449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:45.081273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:45.081273+0000","last_clean_scrub_stamp":"2024-03-28T16:51:45.081273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:49.252246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.084532+0000","last_change":"2024-03-28T16:51:15.084532+0000","last_active":"2024-03-28T16:51:15.084532+0000","last_peered":"2024-03-28T16:51:15.084532+0000","last_clean":"2024-03-28T16:51:15.084532+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:51:15.084532+0000","last_undegraded":"2024-03-28T16:51:15.084532+0000","last_fullsized":"2024-03-28T16:51:15.084532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.084235+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:01:39.553948+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.246355+0000","last_change":"2024-03-28T16:51:09.246355+0000","last_active":"2024-03-28T16:51:09.246355+0000","last_peered":"2024-03-28T16:51:09.246355+0000","last_clean":"2024-03-28T16:51:09.246355+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:51:09.246355+0000","last_undegraded":"2024-03-28T16:51:09.246355+0000","last_fullsized":"2024-03-28T16:51:09.246355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.246098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:55:30.745667+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.442625+0000","last_change":"2024-03-28T16:51:17.442696+0000","last_active":"2024-03-28T16:51:17.442625+0000","last_peered":"2024-03-28T16:51:17.442625+0000","last_clean":"2024-03-28T16:51:17.442625+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:51:17.442625+0000","last_undegraded":"2024-03-28T16:51:17.442625+0000","last_fullsized":"2024-03-28T16:51:17.442625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.442424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:34.861019+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:44.063414+0000","last_change":"2024-03-28T16:51:44.063470+0000","last_active":"2024-03-28T16:51:44.063414+0000","last_peered":"2024-03-28T16:51:44.063414+0000","last_clean":"2024-03-28T16:51:44.063414+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:51:44.063414+0000","last_undegraded":"2024-03-28T16:51:44.063414+0000","last_fullsized":"2024-03-28T16:51:44.063414+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:44.063240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:44.063240+0000","last_clean_scrub_stamp":"2024-03-28T16:51:44.063240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:57:07.474188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.199918+0000","last_change":"2024-03-28T16:51:31.200010+0000","last_active":"2024-03-28T16:51:31.199918+0000","last_peered":"2024-03-28T16:51:31.199918+0000","last_clean":"2024-03-28T16:51:31.199918+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:51:31.199918+0000","last_undegraded":"2024-03-28T16:51:31.199918+0000","last_fullsized":"2024-03-28T16:51:31.199918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:31.199694+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:31.199694+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.199694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:43.004903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.178814+0000","last_change":"2024-03-28T16:50:52.178814+0000","last_active":"2024-03-28T16:50:52.178814+0000","last_peered":"2024-03-28T16:50:52.178814+0000","last_clean":"2024-03-28T16:50:52.178814+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:50:52.178814+0000","last_undegraded":"2024-03-28T16:50:52.178814+0000","last_fullsized":"2024-03-28T16:50:52.178814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.178561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:00.951220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.353835+0000","last_change":"2024-03-28T16:51:00.353835+0000","last_active":"2024-03-28T16:51:00.353835+0000","last_peered":"2024-03-28T16:51:00.353835+0000","last_clean":"2024-03-28T16:51:00.353835+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:51:00.353835+0000","last_undegraded":"2024-03-28T16:51:00.353835+0000","last_fullsized":"2024-03-28T16:51:00.353835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.353557+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:18.400713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.443394+0000","last_change":"2024-03-28T16:50:49.443394+0000","last_active":"2024-03-28T16:50:49.443394+0000","last_peered":"2024-03-28T16:50:49.443394+0000","last_clean":"2024-03-28T16:50:49.443394+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:50:49.443394+0000","last_undegraded":"2024-03-28T16:50:49.443394+0000","last_fullsized":"2024-03-28T16:50:49.443394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.443035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:33:21.896718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.533009+0000","last_change":"2024-03-28T16:49:04.744040+0000","last_active":"2024-03-28T16:50:45.533009+0000","last_peered":"2024-03-28T16:50:45.533009+0000","last_clean":"2024-03-28T16:50:45.533009+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:50:45.533009+0000","last_undegraded":"2024-03-28T16:50:45.533009+0000","last_fullsized":"2024-03-28T16:50:45.533009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.486418+0000","last_change":"2024-03-28T16:51:19.486556+0000","last_active":"2024-03-28T16:51:19.486418+0000","last_peered":"2024-03-28T16:51:19.486418+0000","last_clean":"2024-03-28T16:51:19.486418+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:51:19.486418+0000","last_undegraded":"2024-03-28T16:51:19.486418+0000","last_fullsized":"2024-03-28T16:51:19.486418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.486207+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:42.867614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.236966+0000","last_change":"2024-03-28T16:51:33.237031+0000","last_active":"2024-03-28T16:51:33.236966+0000","last_peered":"2024-03-28T16:51:33.236966+0000","last_clean":"2024-03-28T16:51:33.236966+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:51:33.236966+0000","last_undegraded":"2024-03-28T16:51:33.236966+0000","last_fullsized":"2024-03-28T16:51:33.236966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:33.236770+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:33.236770+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.236770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:05:19.456972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":209,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.379662+0000","last_change":"2024-03-28T16:51:06.379662+0000","last_active":"2024-03-28T16:51:06.379662+0000","last_peered":"2024-03-28T16:51:06.379662+0000","last_clean":"2024-03-28T16:51:06.379662+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:51:06.379662+0000","last_undegraded":"2024-03-28T16:51:06.379662+0000","last_fullsized":"2024-03-28T16:51:06.379662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.379283+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:41.522575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":256,"num_read_kb":255,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'42","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.233580+0000","last_change":"2024-03-28T16:50:55.233580+0000","last_active":"2024-03-28T16:50:55.233580+0000","last_peered":"2024-03-28T16:50:55.233580+0000","last_clean":"2024-03-28T16:50:55.233580+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:55.233580+0000","last_undegraded":"2024-03-28T16:50:55.233580+0000","last_fullsized":"2024-03-28T16:50:55.233580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'42","last_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_deep_scrub":"31'42","last_deep_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.233264+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:43.214573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":56,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.129943+0000","last_change":"2024-03-28T16:50:47.129943+0000","last_active":"2024-03-28T16:50:47.129943+0000","last_peered":"2024-03-28T16:50:47.129943+0000","last_clean":"2024-03-28T16:50:47.129943+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:50:47.129943+0000","last_undegraded":"2024-03-28T16:50:47.129943+0000","last_fullsized":"2024-03-28T16:50:47.129943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.129698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:49.507343+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874520+0000","last_change":"2024-03-28T16:49:04.752570+0000","last_active":"2024-03-28T16:50:46.874520+0000","last_peered":"2024-03-28T16:50:46.874520+0000","last_clean":"2024-03-28T16:50:46.874520+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:50:46.874520+0000","last_undegraded":"2024-03-28T16:50:46.874520+0000","last_fullsized":"2024-03-28T16:50:46.874520+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.093412+0000","last_change":"2024-03-28T16:51:26.093486+0000","last_active":"2024-03-28T16:51:26.093412+0000","last_peered":"2024-03-28T16:51:26.093412+0000","last_clean":"2024-03-28T16:51:26.093412+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:51:26.093412+0000","last_undegraded":"2024-03-28T16:51:26.093412+0000","last_fullsized":"2024-03-28T16:51:26.093412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:26.093216+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:26.093216+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.093216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:08.375290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.237688+0000","last_change":"2024-03-28T16:51:26.237758+0000","last_active":"2024-03-28T16:51:26.237688+0000","last_peered":"2024-03-28T16:51:26.237688+0000","last_clean":"2024-03-28T16:51:26.237688+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:51:26.237688+0000","last_undegraded":"2024-03-28T16:51:26.237688+0000","last_fullsized":"2024-03-28T16:51:26.237688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:26.237512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:26.237512+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.237512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:57.241670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.166465+0000","last_change":"2024-03-28T16:50:58.166465+0000","last_active":"2024-03-28T16:50:58.166465+0000","last_peered":"2024-03-28T16:50:58.166465+0000","last_clean":"2024-03-28T16:50:58.166465+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:50:58.166465+0000","last_undegraded":"2024-03-28T16:50:58.166465+0000","last_fullsized":"2024-03-28T16:50:58.166465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.166221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:21.547629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.370703+0000","last_change":"2024-03-28T16:51:02.370703+0000","last_active":"2024-03-28T16:51:02.370703+0000","last_peered":"2024-03-28T16:51:02.370703+0000","last_clean":"2024-03-28T16:51:02.370703+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:51:02.370703+0000","last_undegraded":"2024-03-28T16:51:02.370703+0000","last_fullsized":"2024-03-28T16:51:02.370703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.370450+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:15.038783+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.448158+0000","last_change":"2024-03-28T16:50:50.448158+0000","last_active":"2024-03-28T16:50:50.448158+0000","last_peered":"2024-03-28T16:50:50.448158+0000","last_clean":"2024-03-28T16:50:50.448158+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:50:50.448158+0000","last_undegraded":"2024-03-28T16:50:50.448158+0000","last_fullsized":"2024-03-28T16:50:50.448158+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.447911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:45.521899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.151593+0000","last_change":"2024-03-28T16:51:31.151647+0000","last_active":"2024-03-28T16:51:31.151593+0000","last_peered":"2024-03-28T16:51:31.151593+0000","last_clean":"2024-03-28T16:51:31.151593+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:51:31.151593+0000","last_undegraded":"2024-03-28T16:51:31.151593+0000","last_fullsized":"2024-03-28T16:51:31.151593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:31.151371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:31.151371+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.151371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:46.085551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:49:41.566962+0000","last_change":"2024-03-28T16:49:37.600567+0000","last_active":"2024-03-28T16:49:41.566962+0000","last_peered":"2024-03-28T16:49:41.566962+0000","last_clean":"2024-03-28T16:49:41.566962+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:49:41.566962+0000","last_undegraded":"2024-03-28T16:49:41.566962+0000","last_fullsized":"2024-03-28T16:49:41.566962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.369809+0000","last_change":"2024-03-28T16:51:07.369837+0000","last_active":"2024-03-28T16:51:07.369809+0000","last_peered":"2024-03-28T16:51:07.369809+0000","last_clean":"2024-03-28T16:51:07.369809+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:51:07.369809+0000","last_undegraded":"2024-03-28T16:51:07.369809+0000","last_fullsized":"2024-03-28T16:51:07.369809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.369707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:56.803658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'60","reported_seq":133,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.252559+0000","last_change":"2024-03-28T16:50:56.252559+0000","last_active":"2024-03-28T16:50:56.252559+0000","last_peered":"2024-03-28T16:50:56.252559+0000","last_clean":"2024-03-28T16:50:56.252559+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:56.252559+0000","last_undegraded":"2024-03-28T16:50:56.252559+0000","last_fullsized":"2024-03-28T16:50:56.252559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'60","last_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_deep_scrub":"31'60","last_deep_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.252305+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:41.009336+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.444559+0000","last_change":"2024-03-28T16:50:48.444559+0000","last_active":"2024-03-28T16:50:48.444559+0000","last_peered":"2024-03-28T16:50:48.444559+0000","last_clean":"2024-03-28T16:50:48.444559+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:50:48.444559+0000","last_undegraded":"2024-03-28T16:50:48.444559+0000","last_fullsized":"2024-03-28T16:50:48.444559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.444297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:24:42.861356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'11","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:45.533003+0000","last_change":"2024-03-28T16:49:04.752525+0000","last_active":"2024-03-28T16:50:45.533003+0000","last_peered":"2024-03-28T16:50:45.533003+0000","last_clean":"2024-03-28T16:50:45.533003+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:50:45.533003+0000","last_undegraded":"2024-03-28T16:50:45.533003+0000","last_fullsized":"2024-03-28T16:50:45.533003+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.399302+0000","last_change":"2024-03-28T16:51:15.399354+0000","last_active":"2024-03-28T16:51:15.399302+0000","last_peered":"2024-03-28T16:51:15.399302+0000","last_clean":"2024-03-28T16:51:15.399302+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:51:15.399302+0000","last_undegraded":"2024-03-28T16:51:15.399302+0000","last_fullsized":"2024-03-28T16:51:15.399302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.399113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:12.560237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.016129+0000","last_change":"2024-03-28T16:51:37.016182+0000","last_active":"2024-03-28T16:51:37.016129+0000","last_peered":"2024-03-28T16:51:37.016129+0000","last_clean":"2024-03-28T16:51:37.016129+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:51:37.016129+0000","last_undegraded":"2024-03-28T16:51:37.016129+0000","last_fullsized":"2024-03-28T16:51:37.016129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.015919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.015919+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.015919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:14.095891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.409622+0000","last_change":"2024-03-28T16:51:09.409674+0000","last_active":"2024-03-28T16:51:09.409622+0000","last_peered":"2024-03-28T16:51:09.409622+0000","last_clean":"2024-03-28T16:51:09.409622+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:51:09.409622+0000","last_undegraded":"2024-03-28T16:51:09.409622+0000","last_fullsized":"2024-03-28T16:51:09.409622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.409427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:27.456583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'43","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.182861+0000","last_change":"2024-03-28T16:50:57.182861+0000","last_active":"2024-03-28T16:50:57.182861+0000","last_peered":"2024-03-28T16:50:57.182861+0000","last_clean":"2024-03-28T16:50:57.182861+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:57.182861+0000","last_undegraded":"2024-03-28T16:50:57.182861+0000","last_fullsized":"2024-03-28T16:50:57.182861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.182585+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.262964+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.238776+0000","last_change":"2024-03-28T16:50:53.238776+0000","last_active":"2024-03-28T16:50:53.238776+0000","last_peered":"2024-03-28T16:50:53.238776+0000","last_clean":"2024-03-28T16:50:53.238776+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:50:53.238776+0000","last_undegraded":"2024-03-28T16:50:53.238776+0000","last_fullsized":"2024-03-28T16:50:53.238776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.238528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:50.127619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874303+0000","last_change":"2024-03-28T16:49:04.754981+0000","last_active":"2024-03-28T16:50:46.874303+0000","last_peered":"2024-03-28T16:50:46.874303+0000","last_clean":"2024-03-28T16:50:46.874303+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:50:46.874303+0000","last_undegraded":"2024-03-28T16:50:46.874303+0000","last_fullsized":"2024-03-28T16:50:46.874303+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.175511+0000","last_change":"2024-03-28T16:51:02.175511+0000","last_active":"2024-03-28T16:51:02.175511+0000","last_peered":"2024-03-28T16:51:02.175511+0000","last_clean":"2024-03-28T16:51:02.175511+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:51:02.175511+0000","last_undegraded":"2024-03-28T16:51:02.175511+0000","last_fullsized":"2024-03-28T16:51:02.175511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.175255+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:28.260113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.155413+0000","last_change":"2024-03-28T16:51:32.155474+0000","last_active":"2024-03-28T16:51:32.155413+0000","last_peered":"2024-03-28T16:51:32.155413+0000","last_clean":"2024-03-28T16:51:32.155413+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:51:32.155413+0000","last_undegraded":"2024-03-28T16:51:32.155413+0000","last_fullsized":"2024-03-28T16:51:32.155413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.155231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.155231+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.155231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:35.685453+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.228518+0000","last_change":"2024-03-28T16:51:38.228577+0000","last_active":"2024-03-28T16:51:38.228518+0000","last_peered":"2024-03-28T16:51:38.228518+0000","last_clean":"2024-03-28T16:51:38.228518+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:51:38.228518+0000","last_undegraded":"2024-03-28T16:51:38.228518+0000","last_fullsized":"2024-03-28T16:51:38.228518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.228325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.228325+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.228325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:56.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.342055+0000","last_change":"2024-03-28T16:50:58.342055+0000","last_active":"2024-03-28T16:50:58.342055+0000","last_peered":"2024-03-28T16:50:58.342055+0000","last_clean":"2024-03-28T16:50:58.342055+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:58.342055+0000","last_undegraded":"2024-03-28T16:50:58.342055+0000","last_fullsized":"2024-03-28T16:50:58.342055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.341815+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:35.862752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.375363+0000","last_change":"2024-03-28T16:51:05.375363+0000","last_active":"2024-03-28T16:51:05.375363+0000","last_peered":"2024-03-28T16:51:05.375363+0000","last_clean":"2024-03-28T16:51:05.375363+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:51:05.375363+0000","last_undegraded":"2024-03-28T16:51:05.375363+0000","last_fullsized":"2024-03-28T16:51:05.375363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.375113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:40.169672+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.118142+0000","last_change":"2024-03-28T16:51:29.118208+0000","last_active":"2024-03-28T16:51:29.118142+0000","last_peered":"2024-03-28T16:51:29.118142+0000","last_clean":"2024-03-28T16:51:29.118142+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:51:29.118142+0000","last_undegraded":"2024-03-28T16:51:29.118142+0000","last_fullsized":"2024-03-28T16:51:29.118142+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:29.117944+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:29.117944+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.117944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:12.949724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.562342+0000","last_change":"2024-03-28T16:51:29.562487+0000","last_active":"2024-03-28T16:51:29.562342+0000","last_peered":"2024-03-28T16:51:29.562342+0000","last_clean":"2024-03-28T16:51:29.562342+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:51:29.562342+0000","last_undegraded":"2024-03-28T16:51:29.562342+0000","last_fullsized":"2024-03-28T16:51:29.562342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:29.562080+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:29.562080+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.562080+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:48.297089+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871444+0000","last_change":"2024-03-28T16:48:54.674069+0000","last_active":"2024-03-28T16:50:46.871444+0000","last_peered":"2024-03-28T16:50:46.871444+0000","last_clean":"2024-03-28T16:50:46.871444+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:50:46.871444+0000","last_undegraded":"2024-03-28T16:50:46.871444+0000","last_fullsized":"2024-03-28T16:50:46.871444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.303980+0000","last_change":"2024-03-28T16:51:04.303980+0000","last_active":"2024-03-28T16:51:04.303980+0000","last_peered":"2024-03-28T16:51:04.303980+0000","last_clean":"2024-03-28T16:51:04.303980+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:51:04.303980+0000","last_undegraded":"2024-03-28T16:51:04.303980+0000","last_fullsized":"2024-03-28T16:51:04.303980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.303737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:22.080932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.229100+0000","last_change":"2024-03-28T16:51:01.229100+0000","last_active":"2024-03-28T16:51:01.229100+0000","last_peered":"2024-03-28T16:51:01.229100+0000","last_clean":"2024-03-28T16:51:01.229100+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:51:01.229100+0000","last_undegraded":"2024-03-28T16:51:01.229100+0000","last_fullsized":"2024-03-28T16:51:01.229100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.228821+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:13.034030+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'249","reported_seq":310,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.440393+0000","last_change":"2024-03-28T16:50:52.440393+0000","last_active":"2024-03-28T16:50:52.440393+0000","last_peered":"2024-03-28T16:50:52.440393+0000","last_clean":"2024-03-28T16:50:52.440393+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:52.440393+0000","last_undegraded":"2024-03-28T16:50:52.440393+0000","last_fullsized":"2024-03-28T16:50:52.440393+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.440130+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:54:49.215852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.089330+0000","last_change":"2024-03-28T16:51:33.089387+0000","last_active":"2024-03-28T16:51:33.089330+0000","last_peered":"2024-03-28T16:51:33.089330+0000","last_clean":"2024-03-28T16:51:33.089330+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:51:33.089330+0000","last_undegraded":"2024-03-28T16:51:33.089330+0000","last_fullsized":"2024-03-28T16:51:33.089330+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.089137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.089137+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.089137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:01:20.214432+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'6","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.352050+0000","last_change":"2024-03-28T16:51:20.352050+0000","last_active":"2024-03-28T16:51:20.352050+0000","last_peered":"2024-03-28T16:51:20.352050+0000","last_clean":"2024-03-28T16:51:20.352050+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:51:20.352050+0000","last_undegraded":"2024-03-28T16:51:20.352050+0000","last_fullsized":"2024-03-28T16:51:20.352050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.351803+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:25.271104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.050474+0000","last_change":"2024-03-28T16:51:36.050529+0000","last_active":"2024-03-28T16:51:36.050474+0000","last_peered":"2024-03-28T16:51:36.050474+0000","last_clean":"2024-03-28T16:51:36.050474+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:51:36.050474+0000","last_undegraded":"2024-03-28T16:51:36.050474+0000","last_fullsized":"2024-03-28T16:51:36.050474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.050281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.050281+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.050281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:01.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.331654+0000","last_change":"2024-03-28T16:51:02.331654+0000","last_active":"2024-03-28T16:51:02.331654+0000","last_peered":"2024-03-28T16:51:02.331654+0000","last_clean":"2024-03-28T16:51:02.331654+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:51:02.331654+0000","last_undegraded":"2024-03-28T16:51:02.331654+0000","last_fullsized":"2024-03-28T16:51:02.331654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.331416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:20.378572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'43","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.202390+0000","last_change":"2024-03-28T16:50:50.202390+0000","last_active":"2024-03-28T16:50:50.202390+0000","last_peered":"2024-03-28T16:50:50.202390+0000","last_clean":"2024-03-28T16:50:50.202390+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:50.202390+0000","last_undegraded":"2024-03-28T16:50:50.202390+0000","last_fullsized":"2024-03-28T16:50:50.202390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.202141+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.086676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.276938+0000","last_change":"2024-03-28T16:50:48.276938+0000","last_active":"2024-03-28T16:50:48.276938+0000","last_peered":"2024-03-28T16:50:48.276938+0000","last_clean":"2024-03-28T16:50:48.276938+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:50:48.276938+0000","last_undegraded":"2024-03-28T16:50:48.276938+0000","last_fullsized":"2024-03-28T16:50:48.276938+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.276698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:29.219774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:43.038235+0000","last_change":"2024-03-28T16:51:43.038288+0000","last_active":"2024-03-28T16:51:43.038235+0000","last_peered":"2024-03-28T16:51:43.038235+0000","last_clean":"2024-03-28T16:51:43.038235+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:51:43.038235+0000","last_undegraded":"2024-03-28T16:51:43.038235+0000","last_fullsized":"2024-03-28T16:51:43.038235+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:43.038045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:43.038045+0000","last_clean_scrub_stamp":"2024-03-28T16:51:43.038045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:50.550015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'4","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.310265+0000","last_change":"2024-03-28T16:51:24.310265+0000","last_active":"2024-03-28T16:51:24.310265+0000","last_peered":"2024-03-28T16:51:24.310265+0000","last_clean":"2024-03-28T16:51:24.310265+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:51:24.310265+0000","last_undegraded":"2024-03-28T16:51:24.310265+0000","last_fullsized":"2024-03-28T16:51:24.310265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.310025+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:21.471148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871437+0000","last_change":"2024-03-28T16:48:54.669871+0000","last_active":"2024-03-28T16:50:46.871437+0000","last_peered":"2024-03-28T16:50:46.871437+0000","last_clean":"2024-03-28T16:50:46.871437+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:50:46.871437+0000","last_undegraded":"2024-03-28T16:50:46.871437+0000","last_fullsized":"2024-03-28T16:50:46.871437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.396495+0000","last_change":"2024-03-28T16:51:03.396495+0000","last_active":"2024-03-28T16:51:03.396495+0000","last_peered":"2024-03-28T16:51:03.396495+0000","last_clean":"2024-03-28T16:51:03.396495+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:51:03.396495+0000","last_undegraded":"2024-03-28T16:51:03.396495+0000","last_fullsized":"2024-03-28T16:51:03.396495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.396253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:26.295446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.307759+0000","last_change":"2024-03-28T16:50:59.307759+0000","last_active":"2024-03-28T16:50:59.307759+0000","last_peered":"2024-03-28T16:50:59.307759+0000","last_clean":"2024-03-28T16:50:59.307759+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:59.307759+0000","last_undegraded":"2024-03-28T16:50:59.307759+0000","last_fullsized":"2024-03-28T16:50:59.307759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.307494+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:28.146893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.090891+0000","last_change":"2024-03-28T16:51:13.090891+0000","last_active":"2024-03-28T16:51:13.090891+0000","last_peered":"2024-03-28T16:51:13.090891+0000","last_clean":"2024-03-28T16:51:13.090891+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:51:13.090891+0000","last_undegraded":"2024-03-28T16:51:13.090891+0000","last_fullsized":"2024-03-28T16:51:13.090891+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.090537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:44:11.869557+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874510+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:50:46.874510+0000","last_peered":"2024-03-28T16:50:46.874510+0000","last_clean":"2024-03-28T16:50:46.874510+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:50:46.874510+0000","last_undegraded":"2024-03-28T16:50:46.874510+0000","last_fullsized":"2024-03-28T16:50:46.874510+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.064823+0000","last_change":"2024-03-28T16:51:35.064895+0000","last_active":"2024-03-28T16:51:35.064823+0000","last_peered":"2024-03-28T16:51:35.064823+0000","last_clean":"2024-03-28T16:51:35.064823+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:51:35.064823+0000","last_undegraded":"2024-03-28T16:51:35.064823+0000","last_fullsized":"2024-03-28T16:51:35.064823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:35.064632+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:35.064632+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.064632+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:41.776672+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.598680+0000","last_change":"2024-03-28T16:51:24.598733+0000","last_active":"2024-03-28T16:51:24.598680+0000","last_peered":"2024-03-28T16:51:24.598680+0000","last_clean":"2024-03-28T16:51:24.598680+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:51:24.598680+0000","last_undegraded":"2024-03-28T16:51:24.598680+0000","last_fullsized":"2024-03-28T16:51:24.598680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:24.598488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:24.598488+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.598488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:05:10.818690+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.256599+0000","last_change":"2024-03-28T16:50:59.256599+0000","last_active":"2024-03-28T16:50:59.256599+0000","last_peered":"2024-03-28T16:50:59.256599+0000","last_clean":"2024-03-28T16:50:59.256599+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:50:59.256599+0000","last_undegraded":"2024-03-28T16:50:59.256599+0000","last_fullsized":"2024-03-28T16:50:59.256599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.256359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:34.399651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'28","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.213439+0000","last_change":"2024-03-28T16:50:59.213439+0000","last_active":"2024-03-28T16:50:59.213439+0000","last_peered":"2024-03-28T16:50:59.213439+0000","last_clean":"2024-03-28T16:50:59.213439+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:59.213439+0000","last_undegraded":"2024-03-28T16:50:59.213439+0000","last_fullsized":"2024-03-28T16:50:59.213439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'28","last_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_deep_scrub":"31'28","last_deep_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.213187+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:38.889215+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.080305+0000","last_change":"2024-03-28T16:51:17.080305+0000","last_active":"2024-03-28T16:51:17.080305+0000","last_peered":"2024-03-28T16:51:17.080305+0000","last_clean":"2024-03-28T16:51:17.080305+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:51:17.080305+0000","last_undegraded":"2024-03-28T16:51:17.080305+0000","last_fullsized":"2024-03-28T16:51:17.080305+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.080049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:12.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874360+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:50:46.874360+0000","last_peered":"2024-03-28T16:50:46.874360+0000","last_clean":"2024-03-28T16:50:46.874360+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:50:46.874360+0000","last_undegraded":"2024-03-28T16:50:46.874360+0000","last_fullsized":"2024-03-28T16:50:46.874360+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'12","reported_seq":66,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.347902+0000","last_change":"2024-03-28T16:51:13.347902+0000","last_active":"2024-03-28T16:51:13.347902+0000","last_peered":"2024-03-28T16:51:13.347902+0000","last_clean":"2024-03-28T16:51:13.347902+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:51:13.347902+0000","last_undegraded":"2024-03-28T16:51:13.347902+0000","last_fullsized":"2024-03-28T16:51:13.347902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.347663+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:39.911911+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.128855+0000","last_change":"2024-03-28T16:51:34.128912+0000","last_active":"2024-03-28T16:51:34.128855+0000","last_peered":"2024-03-28T16:51:34.128855+0000","last_clean":"2024-03-28T16:51:34.128855+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:51:34.128855+0000","last_undegraded":"2024-03-28T16:51:34.128855+0000","last_fullsized":"2024-03-28T16:51:34.128855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:34.128651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:34.128651+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.128651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:18.350401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.215895+0000","last_change":"2024-03-28T16:50:58.215895+0000","last_active":"2024-03-28T16:50:58.215895+0000","last_peered":"2024-03-28T16:50:58.215895+0000","last_clean":"2024-03-28T16:50:58.215895+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:50:58.215895+0000","last_undegraded":"2024-03-28T16:50:58.215895+0000","last_fullsized":"2024-03-28T16:50:58.215895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.215651+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:42.972682+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'35","reported_seq":90,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.272061+0000","last_change":"2024-03-28T16:51:05.272061+0000","last_active":"2024-03-28T16:51:05.272061+0000","last_peered":"2024-03-28T16:51:05.272061+0000","last_clean":"2024-03-28T16:51:05.272061+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:51:05.272061+0000","last_undegraded":"2024-03-28T16:51:05.272061+0000","last_fullsized":"2024-03-28T16:51:05.272061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'35","last_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_deep_scrub":"31'35","last_deep_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.271785+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:01.679261+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1150,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.209342+0000","last_change":"2024-03-28T16:50:50.209342+0000","last_active":"2024-03-28T16:50:50.209342+0000","last_peered":"2024-03-28T16:50:50.209342+0000","last_clean":"2024-03-28T16:50:50.209342+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:50:50.209342+0000","last_undegraded":"2024-03-28T16:50:50.209342+0000","last_fullsized":"2024-03-28T16:50:50.209342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.209010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:45.672112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874325+0000","last_change":"2024-03-28T16:49:04.753745+0000","last_active":"2024-03-28T16:50:46.874325+0000","last_peered":"2024-03-28T16:50:46.874325+0000","last_clean":"2024-03-28T16:50:46.874325+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:50:46.874325+0000","last_undegraded":"2024-03-28T16:50:46.874325+0000","last_fullsized":"2024-03-28T16:50:46.874325+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.212601+0000","last_change":"2024-03-28T16:51:41.212741+0000","last_active":"2024-03-28T16:51:41.212601+0000","last_peered":"2024-03-28T16:51:41.212601+0000","last_clean":"2024-03-28T16:51:41.212601+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:51:41.212601+0000","last_undegraded":"2024-03-28T16:51:41.212601+0000","last_fullsized":"2024-03-28T16:51:41.212601+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:41.212399+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:41.212399+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.212399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:25.434150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.209561+0000","last_change":"2024-03-28T16:51:32.209624+0000","last_active":"2024-03-28T16:51:32.209561+0000","last_peered":"2024-03-28T16:51:32.209561+0000","last_clean":"2024-03-28T16:51:32.209561+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:51:32.209561+0000","last_undegraded":"2024-03-28T16:51:32.209561+0000","last_fullsized":"2024-03-28T16:51:32.209561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.209381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.209381+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.209381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:58.294239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.553584+0000","last_change":"2024-03-28T16:51:27.553649+0000","last_active":"2024-03-28T16:51:27.553584+0000","last_peered":"2024-03-28T16:51:27.553584+0000","last_clean":"2024-03-28T16:51:27.553584+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:51:27.553584+0000","last_undegraded":"2024-03-28T16:51:27.553584+0000","last_fullsized":"2024-03-28T16:51:27.553584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:27.553385+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:27.553385+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.553385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:32.733520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.448938+0000","last_change":"2024-03-28T16:51:37.449001+0000","last_active":"2024-03-28T16:51:37.448938+0000","last_peered":"2024-03-28T16:51:37.448938+0000","last_clean":"2024-03-28T16:51:37.448938+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:51:37.448938+0000","last_undegraded":"2024-03-28T16:51:37.448938+0000","last_fullsized":"2024-03-28T16:51:37.448938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:37.448740+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:37.448740+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.448740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:45.427028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.141067+0000","last_change":"2024-03-28T16:51:32.141133+0000","last_active":"2024-03-28T16:51:32.141067+0000","last_peered":"2024-03-28T16:51:32.141067+0000","last_clean":"2024-03-28T16:51:32.141067+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:51:32.141067+0000","last_undegraded":"2024-03-28T16:51:32.141067+0000","last_fullsized":"2024-03-28T16:51:32.141067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:32.140882+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:32.140882+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.140882+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:30.700161+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871513+0000","last_change":"2024-03-28T16:48:54.657491+0000","last_active":"2024-03-28T16:50:46.871513+0000","last_peered":"2024-03-28T16:50:46.871513+0000","last_clean":"2024-03-28T16:50:46.871513+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:50:46.871513+0000","last_undegraded":"2024-03-28T16:50:46.871513+0000","last_fullsized":"2024-03-28T16:50:46.871513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.497847+0000","last_change":"2024-03-28T16:51:31.497957+0000","last_active":"2024-03-28T16:51:31.497847+0000","last_peered":"2024-03-28T16:51:31.497847+0000","last_clean":"2024-03-28T16:51:31.497847+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:51:31.497847+0000","last_undegraded":"2024-03-28T16:51:31.497847+0000","last_fullsized":"2024-03-28T16:51:31.497847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:31.497632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:31.497632+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.497632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:23.506227+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.564274+0000","last_change":"2024-03-28T16:51:26.564336+0000","last_active":"2024-03-28T16:51:26.564274+0000","last_peered":"2024-03-28T16:51:26.564274+0000","last_clean":"2024-03-28T16:51:26.564274+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:51:26.564274+0000","last_undegraded":"2024-03-28T16:51:26.564274+0000","last_fullsized":"2024-03-28T16:51:26.564274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:26.564092+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:26.564092+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.564092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:44.943347+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.523436+0000","last_change":"2024-03-28T16:51:21.523575+0000","last_active":"2024-03-28T16:51:21.523436+0000","last_peered":"2024-03-28T16:51:21.523436+0000","last_clean":"2024-03-28T16:51:21.523436+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:51:21.523436+0000","last_undegraded":"2024-03-28T16:51:21.523436+0000","last_fullsized":"2024-03-28T16:51:21.523436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.523213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:56.404435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.467825+0000","last_change":"2024-03-28T16:51:38.467878+0000","last_active":"2024-03-28T16:51:38.467825+0000","last_peered":"2024-03-28T16:51:38.467825+0000","last_clean":"2024-03-28T16:51:38.467825+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:51:38.467825+0000","last_undegraded":"2024-03-28T16:51:38.467825+0000","last_fullsized":"2024-03-28T16:51:38.467825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.467639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.467639+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.467639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:55:12.097451+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":71,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.278285+0000","last_change":"2024-03-28T16:51:19.278345+0000","last_active":"2024-03-28T16:51:19.278285+0000","last_peered":"2024-03-28T16:51:19.278285+0000","last_clean":"2024-03-28T16:51:19.278285+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:51:19.278285+0000","last_undegraded":"2024-03-28T16:51:19.278285+0000","last_fullsized":"2024-03-28T16:51:19.278285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.278071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:29.679472+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.988224+0000","last_change":"2024-03-28T16:51:37.988292+0000","last_active":"2024-03-28T16:51:37.988224+0000","last_peered":"2024-03-28T16:51:37.988224+0000","last_clean":"2024-03-28T16:51:37.988224+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:51:37.988224+0000","last_undegraded":"2024-03-28T16:51:37.988224+0000","last_fullsized":"2024-03-28T16:51:37.988224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.988041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.988041+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.988041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:35.010568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.446697+0000","last_change":"2024-03-28T16:51:18.446757+0000","last_active":"2024-03-28T16:51:18.446697+0000","last_peered":"2024-03-28T16:51:18.446697+0000","last_clean":"2024-03-28T16:51:18.446697+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:51:18.446697+0000","last_undegraded":"2024-03-28T16:51:18.446697+0000","last_fullsized":"2024-03-28T16:51:18.446697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.446501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:51.526879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.283525+0000","last_change":"2024-03-28T16:51:25.283581+0000","last_active":"2024-03-28T16:51:25.283525+0000","last_peered":"2024-03-28T16:51:25.283525+0000","last_clean":"2024-03-28T16:51:25.283525+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:51:25.283525+0000","last_undegraded":"2024-03-28T16:51:25.283525+0000","last_fullsized":"2024-03-28T16:51:25.283525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.283348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:28.047908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.316649+0000","last_change":"2024-03-28T16:51:21.316736+0000","last_active":"2024-03-28T16:51:21.316649+0000","last_peered":"2024-03-28T16:51:21.316649+0000","last_clean":"2024-03-28T16:51:21.316649+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:51:21.316649+0000","last_undegraded":"2024-03-28T16:51:21.316649+0000","last_fullsized":"2024-03-28T16:51:21.316649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.316476+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:10:20.931612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.420032+0000","last_change":"2024-03-28T16:51:40.420091+0000","last_active":"2024-03-28T16:51:40.420032+0000","last_peered":"2024-03-28T16:51:40.420032+0000","last_clean":"2024-03-28T16:51:40.420032+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:51:40.420032+0000","last_undegraded":"2024-03-28T16:51:40.420032+0000","last_fullsized":"2024-03-28T16:51:40.420032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:40.419842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:40.419842+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.419842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:10.034139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.289210+0000","last_change":"2024-03-28T16:51:10.289210+0000","last_active":"2024-03-28T16:51:10.289210+0000","last_peered":"2024-03-28T16:51:10.289210+0000","last_clean":"2024-03-28T16:51:10.289210+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:51:10.289210+0000","last_undegraded":"2024-03-28T16:51:10.289210+0000","last_fullsized":"2024-03-28T16:51:10.289210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.288939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:52.264862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871590+0000","last_change":"2024-03-28T16:48:54.657662+0000","last_active":"2024-03-28T16:50:46.871590+0000","last_peered":"2024-03-28T16:50:46.871590+0000","last_clean":"2024-03-28T16:50:46.871590+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:50:46.871590+0000","last_undegraded":"2024-03-28T16:50:46.871590+0000","last_fullsized":"2024-03-28T16:50:46.871590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.144122+0000","last_change":"2024-03-28T16:51:27.144188+0000","last_active":"2024-03-28T16:51:27.144122+0000","last_peered":"2024-03-28T16:51:27.144122+0000","last_clean":"2024-03-28T16:51:27.144122+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:51:27.144122+0000","last_undegraded":"2024-03-28T16:51:27.144122+0000","last_fullsized":"2024-03-28T16:51:27.144122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:27.143940+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:27.143940+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.143940+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:09:52.594727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.482752+0000","last_change":"2024-03-28T16:51:32.482808+0000","last_active":"2024-03-28T16:51:32.482752+0000","last_peered":"2024-03-28T16:51:32.482752+0000","last_clean":"2024-03-28T16:51:32.482752+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:51:32.482752+0000","last_undegraded":"2024-03-28T16:51:32.482752+0000","last_fullsized":"2024-03-28T16:51:32.482752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.482535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.482535+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.482535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:16.704285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.557168+0000","last_change":"2024-03-28T16:51:23.557231+0000","last_active":"2024-03-28T16:51:23.557168+0000","last_peered":"2024-03-28T16:51:23.557168+0000","last_clean":"2024-03-28T16:51:23.557168+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:51:23.557168+0000","last_undegraded":"2024-03-28T16:51:23.557168+0000","last_fullsized":"2024-03-28T16:51:23.557168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:23.556975+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:23.556975+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.556975+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:49.641832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.301018+0000","last_change":"2024-03-28T16:51:22.301084+0000","last_active":"2024-03-28T16:51:22.301018+0000","last_peered":"2024-03-28T16:51:22.301018+0000","last_clean":"2024-03-28T16:51:22.301018+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:51:22.301018+0000","last_undegraded":"2024-03-28T16:51:22.301018+0000","last_fullsized":"2024-03-28T16:51:22.301018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.300834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:39.676855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.261019+0000","last_change":"2024-03-28T16:51:18.261152+0000","last_active":"2024-03-28T16:51:18.261019+0000","last_peered":"2024-03-28T16:51:18.261019+0000","last_clean":"2024-03-28T16:51:18.261019+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:51:18.261019+0000","last_undegraded":"2024-03-28T16:51:18.261019+0000","last_fullsized":"2024-03-28T16:51:18.261019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.260778+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:15.421901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.256951+0000","last_change":"2024-03-28T16:51:24.257006+0000","last_active":"2024-03-28T16:51:24.256951+0000","last_peered":"2024-03-28T16:51:24.256951+0000","last_clean":"2024-03-28T16:51:24.256951+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:51:24.256951+0000","last_undegraded":"2024-03-28T16:51:24.256951+0000","last_fullsized":"2024-03-28T16:51:24.256951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.256765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:47.235830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.319730+0000","last_change":"2024-03-28T16:51:14.319730+0000","last_active":"2024-03-28T16:51:14.319730+0000","last_peered":"2024-03-28T16:51:14.319730+0000","last_clean":"2024-03-28T16:51:14.319730+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:51:14.319730+0000","last_undegraded":"2024-03-28T16:51:14.319730+0000","last_fullsized":"2024-03-28T16:51:14.319730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.319444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:39.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.080203+0000","last_change":"2024-03-28T16:51:36.080262+0000","last_active":"2024-03-28T16:51:36.080203+0000","last_peered":"2024-03-28T16:51:36.080203+0000","last_clean":"2024-03-28T16:51:36.080203+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:51:36.080203+0000","last_undegraded":"2024-03-28T16:51:36.080203+0000","last_fullsized":"2024-03-28T16:51:36.080203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.080017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.080017+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.080017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:46.963031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.281665+0000","last_change":"2024-03-28T16:51:06.281665+0000","last_active":"2024-03-28T16:51:06.281665+0000","last_peered":"2024-03-28T16:51:06.281665+0000","last_clean":"2024-03-28T16:51:06.281665+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:51:06.281665+0000","last_undegraded":"2024-03-28T16:51:06.281665+0000","last_fullsized":"2024-03-28T16:51:06.281665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.281413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:32.845661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871650+0000","last_change":"2024-03-28T16:48:54.663003+0000","last_active":"2024-03-28T16:50:46.871650+0000","last_peered":"2024-03-28T16:50:46.871650+0000","last_clean":"2024-03-28T16:50:46.871650+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:50:46.871650+0000","last_undegraded":"2024-03-28T16:50:46.871650+0000","last_fullsized":"2024-03-28T16:50:46.871650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.107093+0000","last_change":"2024-03-28T16:51:23.107159+0000","last_active":"2024-03-28T16:51:23.107093+0000","last_peered":"2024-03-28T16:51:23.107093+0000","last_clean":"2024-03-28T16:51:23.107093+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:51:23.107093+0000","last_undegraded":"2024-03-28T16:51:23.107093+0000","last_fullsized":"2024-03-28T16:51:23.107093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:23.106864+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:23.106864+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.106864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:31.824473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.338591+0000","last_change":"2024-03-28T16:51:23.338657+0000","last_active":"2024-03-28T16:51:23.338591+0000","last_peered":"2024-03-28T16:51:23.338591+0000","last_clean":"2024-03-28T16:51:23.338591+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:51:23.338591+0000","last_undegraded":"2024-03-28T16:51:23.338591+0000","last_fullsized":"2024-03-28T16:51:23.338591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:37.364884+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'10","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.275847+0000","last_change":"2024-03-28T16:51:23.275918+0000","last_active":"2024-03-28T16:51:23.275847+0000","last_peered":"2024-03-28T16:51:23.275847+0000","last_clean":"2024-03-28T16:51:23.275847+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:51:23.275847+0000","last_undegraded":"2024-03-28T16:51:23.275847+0000","last_fullsized":"2024-03-28T16:51:23.275847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.275672+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:58.156894+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.083211+0000","last_change":"2024-03-28T16:51:40.083323+0000","last_active":"2024-03-28T16:51:40.083211+0000","last_peered":"2024-03-28T16:51:40.083211+0000","last_clean":"2024-03-28T16:51:40.083211+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:51:40.083211+0000","last_undegraded":"2024-03-28T16:51:40.083211+0000","last_fullsized":"2024-03-28T16:51:40.083211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:40.082968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:40.082968+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.082968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:49:31.036900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.871739+0000","last_change":"2024-03-28T16:48:54.665260+0000","last_active":"2024-03-28T16:50:46.871739+0000","last_peered":"2024-03-28T16:50:46.871739+0000","last_clean":"2024-03-28T16:50:46.871739+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:50:46.871739+0000","last_undegraded":"2024-03-28T16:50:46.871739+0000","last_fullsized":"2024-03-28T16:50:46.871739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:48:53.621721+0000","last_clean_scrub_stamp":"2024-03-28T16:48:53.621721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.440088+0000","last_change":"2024-03-28T16:51:16.440088+0000","last_active":"2024-03-28T16:51:16.440088+0000","last_peered":"2024-03-28T16:51:16.440088+0000","last_clean":"2024-03-28T16:51:16.440088+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:51:16.440088+0000","last_undegraded":"2024-03-28T16:51:16.440088+0000","last_fullsized":"2024-03-28T16:51:16.440088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.439837+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:19.204397+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.539510+0000","last_change":"2024-03-28T16:51:30.539564+0000","last_active":"2024-03-28T16:51:30.539510+0000","last_peered":"2024-03-28T16:51:30.539510+0000","last_clean":"2024-03-28T16:51:30.539510+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:51:30.539510+0000","last_undegraded":"2024-03-28T16:51:30.539510+0000","last_fullsized":"2024-03-28T16:51:30.539510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.539307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.539307+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.539307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:55:52.038876+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.253966+0000","last_change":"2024-03-28T16:51:29.254037+0000","last_active":"2024-03-28T16:51:29.253966+0000","last_peered":"2024-03-28T16:51:29.253966+0000","last_clean":"2024-03-28T16:51:29.253966+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:51:29.253966+0000","last_undegraded":"2024-03-28T16:51:29.253966+0000","last_fullsized":"2024-03-28T16:51:29.253966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:29.253788+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:29.253788+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.253788+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:02:14.899395+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1992,"ondisk_log_size":1992,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1539,"num_read_kb":1325,"num_write":581,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1750,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1422,"ondisk_log_size":1422,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":866,"num_read_kb":864,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705703,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44076,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327764,"statfs":{"total":96636764160,"available":96591630336,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66600000000000004,"15min":0.66700000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79900000000000004,"5min":1.246,"15min":1.246},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81899999999999995,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.57599999999999996}]},{"osd":1,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.26700000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.79000000000000004,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.76600000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.63900000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77400000000000002,"5min":1.671,"15min":1.671},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76400000000000001,"5min":1.331,"15min":1.331},"last":0.53500000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705704,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.65500000000000003,"15min":0.63700000000000001},"min":{"1min":0.42399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.98,"5min":2.98,"15min":2.98},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.53900000000000003,"15min":0.51900000000000002},"min":{"1min":0.26900000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":2.6280000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.56100000000000005,"15min":0.54000000000000004},"min":{"1min":0.28599999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":3.0800000000000001,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Mar 28 16:50:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.57899999999999996,"15min":0.55800000000000005},"min":{"1min":0.30399999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.61399999999999999,"15min":0.59899999999999998},"min":{"1min":0.32200000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":2.7589999999999999,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.622}]}]},{"osd":1,"up_from":9,"seq":38654705704,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.45300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79600000000000004,"5min":0.878,"15min":0.878},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65000000000000002,"15min":0.64800000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78800000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.24099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69399999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.50900000000000001,"15min":0.499},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67500000000000004}]},{"osd":3,"last update":"Thu Mar 28 16:50:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.83799999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.621},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.27100000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.86299999999999999,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59299999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88700000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.40600000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90000000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.46600000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90800000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92900000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.89600000000000002}]},{"osd":3,"last update":"Thu Mar 28 16:50:46 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.75900000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94999999999999996,"5min":1.117,"15min":1.117},"last":0.629}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:51:48.496 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.497 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.497 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.497 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.497 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.498 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.498 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.498 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.499 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.499 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.499 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.499 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-03-28T16:48:57.649745+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=57, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.499 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.500 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.500 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.500 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-03-28T16:48:55.634750+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=55, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.500 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.501 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.501 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.501 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.501 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.501 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.502 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.502 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-28T16:48:47.598556+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=47, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.502 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.502 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.503 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.503 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.503 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.503 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.504 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.504 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.504 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.504 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.505 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.505 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.505 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.505 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.506 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-28T16:48:53.621721+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=48, tm_sec=53, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:51:48.506 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-28T16:52:08.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:52:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.635+0000 7fe473a15640 1 Processor -- start 2024-03-28T16:52:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.635+0000 7fe473a15640 1 -- start start 2024-03-28T16:52:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.635+0000 7fe473a15640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe46c128af0 0x7fe46c128ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:08.676 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.635+0000 7fe473a15640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c128580 con 0x7fe46c12ba80 2024-03-28T16:52:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.635+0000 7fe473a15640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46c092580 con 0x7fe46c128af0 2024-03-28T16:52:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.635+0000 7fe4717b1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe46c128af0 0x7fe46c128ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.635+0000 7fe470fb0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe46c12ba80 0x7fe46c12bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41578/0 (socket says 172.21.15.119:41578) 2024-03-28T16:52:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe470fb0640 1 -- 172.21.15.119:0/2996511447 learned_addr learned my addr 172.21.15.119:0/2996511447 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:52:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe4717b1640 1 -- 172.21.15.119:0/2996511447 >> v1:172.21.15.119:6789/0 conn(0x7fe46c12ba80 legacy=0x7fe46c12bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:08.677 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe4717b1640 1 -- 172.21.15.119:0/2996511447 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c0930d0 con 0x7fe46c128af0 2024-03-28T16:52:08.680 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe4717b1640 1 --2- 172.21.15.119:0/2996511447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c128ed0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe468009740 tx=0x7fe46801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5e0e163466a94a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe463fff640 1 -- 172.21.15.119:0/2996511447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4680ab530 con 0x7fe46c128af0 2024-03-28T16:52:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe463fff640 1 -- 172.21.15.119:0/2996511447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4680a7050 con 0x7fe46c128af0 2024-03-28T16:52:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe463fff640 1 -- 172.21.15.119:0/2996511447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4680ab7e0 con 0x7fe46c128af0 2024-03-28T16:52:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe473a15640 1 -- 172.21.15.119:0/2996511447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 msgr2=0x7fe46c128ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe473a15640 1 --2- 172.21.15.119:0/2996511447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c128ed0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe468009740 tx=0x7fe46801c370 comp rx=0 tx=0).stop 2024-03-28T16:52:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe473a15640 1 -- 172.21.15.119:0/2996511447 shutdown_connections 2024-03-28T16:52:08.681 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe473a15640 1 --2- 172.21.15.119:0/2996511447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c128ed0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe473a15640 1 -- 172.21.15.119:0/2996511447 >> 172.21.15.119:0/2996511447 conn(0x7fe46c09c970 msgr2=0x7fe46c098e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe473a15640 1 -- 172.21.15.119:0/2996511447 shutdown_connections 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.639+0000 7fe473a15640 1 -- 172.21.15.119:0/2996511447 wait complete. 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe473a15640 1 Processor -- start 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe473a15640 1 -- start start 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe473a15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c1c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe473a15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c0707a0 con 0x7fe46c128af0 2024-03-28T16:52:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe4717b1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c1c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe4717b1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c1c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36198/0 (socket says 172.21.15.119:36198) 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe4717b1640 1 -- 172.21.15.119:0/1751245024 learned_addr learned my addr 172.21.15.119:0/1751245024 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe4717b1640 1 -- 172.21.15.119:0/1751245024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468009420 con 0x7fe46c128af0 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe4717b1640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c1c3a00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe468018f80 tx=0x7fe46801abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe461ffb640 1 -- 172.21.15.119:0/1751245024 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4680b7660 con 0x7fe46c128af0 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe461ffb640 1 -- 172.21.15.119:0/1751245024 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4680b77c0 con 0x7fe46c128af0 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c071eb0 con 0x7fe46c128af0 2024-03-28T16:52:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe461ffb640 1 -- 172.21.15.119:0/1751245024 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4680b7aa0 con 0x7fe46c128af0 2024-03-28T16:52:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c0703d0 con 0x7fe46c128af0 2024-03-28T16:52:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46c070ed0 con 0x7fe46c128af0 2024-03-28T16:52:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe461ffb640 1 -- 172.21.15.119:0/1751245024 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fe4680b7c00 con 0x7fe46c128af0 2024-03-28T16:52:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe461ffb640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe450035a10 0x7fe450037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe461ffb640 1 -- 172.21.15.119:0/1751245024 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe4680e3430 con 0x7fe46c128af0 2024-03-28T16:52:08.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.643+0000 7fe470fb0640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe450035a10 0x7fe450037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:08.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.647+0000 7fe470fb0640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe450035a10 0x7fe450037ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe45c001610 tx=0x7fe45c007ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:08.685 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.647+0000 7fe461ffb640 1 -- 172.21.15.119:0/1751245024 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4680c7b90 con 0x7fe46c128af0 2024-03-28T16:52:08.803 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.799+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe46c092980 con 0x7fe450035a10 2024-03-28T16:52:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.847+0000 7fe461ffb640 1 -- 172.21.15.119:0/1751245024 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+673452 (secure 0 0 0) 0x7fe45c001300 con 0x7fe450035a10 2024-03-28T16:52:08.856 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:52:08.862 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:52:08.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe450035a10 msgr2=0x7fe450037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:08.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe450035a10 0x7fe450037ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe45c001610 tx=0x7fe45c007ad0 comp rx=0 tx=0).stop 2024-03-28T16:52:08.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 msgr2=0x7fe46c1c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:08.863 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c1c3a00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe468018f80 tx=0x7fe46801abb0 comp rx=0 tx=0).stop 2024-03-28T16:52:08.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 shutdown_connections 2024-03-28T16:52:08.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fe450035a10 0x7fe450037ed0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:08.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 --2- 172.21.15.119:0/1751245024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe46c128af0 0x7fe46c1c3a00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:08.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 >> 172.21.15.119:0/1751245024 conn(0x7fe46c09c970 msgr2=0x7fe46c12cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:52:08.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 shutdown_connections 2024-03-28T16:52:08.864 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:08.855+0000 7fe473a15640 1 -- 172.21.15.119:0/1751245024 wait complete. 2024-03-28T16:52:08.881 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-03-28T16:52:08.578063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":4029,"num_read_kb":3679,"num_write":1543,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2298,"num_omap_keys":13,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4057,"ondisk_log_size":4057,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":164928,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":162105,"kb_avail":377322432,"statfs":{"total":386547056640,"available":386378170368,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":165996310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006321"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.238075+0000","last_change":"2024-03-28T16:51:28.238133+0000","last_active":"2024-03-28T16:51:28.238075+0000","last_peered":"2024-03-28T16:51:28.238075+0000","last_clean":"2024-03-28T16:51:28.238075+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:51:28.238075+0000","last_undegraded":"2024-03-28T16:51:28.238075+0000","last_fullsized":"2024-03-28T16:51:28.238075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:28.237880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:28.237880+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.237880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:51.619287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.303646+0000","last_change":"2024-03-28T16:51:11.303646+0000","last_active":"2024-03-28T16:51:11.303646+0000","last_peered":"2024-03-28T16:51:11.303646+0000","last_clean":"2024-03-28T16:51:11.303646+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:51:11.303646+0000","last_undegraded":"2024-03-28T16:51:11.303646+0000","last_fullsized":"2024-03-28T16:51:11.303646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.303263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:59.401107+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.569902+0000","last_change":"2024-03-28T16:51:22.569959+0000","last_active":"2024-03-28T16:51:22.569902+0000","last_peered":"2024-03-28T16:51:22.569902+0000","last_clean":"2024-03-28T16:51:22.569902+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:51:22.569902+0000","last_undegraded":"2024-03-28T16:51:22.569902+0000","last_fullsized":"2024-03-28T16:51:22.569902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:22.569729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:22.569729+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.569729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:34.374938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.098164+0000","last_change":"2024-03-28T16:51:22.098164+0000","last_active":"2024-03-28T16:51:22.098164+0000","last_peered":"2024-03-28T16:51:22.098164+0000","last_clean":"2024-03-28T16:51:22.098164+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:51:22.098164+0000","last_undegraded":"2024-03-28T16:51:22.098164+0000","last_fullsized":"2024-03-28T16:51:22.098164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:22.097907+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:22.097907+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.097907+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:26.268880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.105158+0000","last_change":"2024-03-28T16:51:39.105220+0000","last_active":"2024-03-28T16:51:39.105158+0000","last_peered":"2024-03-28T16:51:39.105158+0000","last_clean":"2024-03-28T16:51:39.105158+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:51:39.105158+0000","last_undegraded":"2024-03-28T16:51:39.105158+0000","last_fullsized":"2024-03-28T16:51:39.105158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.104968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.104968+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.104968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:12:14.485249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'4","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.105540+0000","last_change":"2024-03-28T16:51:19.105540+0000","last_active":"2024-03-28T16:51:19.105540+0000","last_peered":"2024-03-28T16:51:19.105540+0000","last_clean":"2024-03-28T16:51:19.105540+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:51:19.105540+0000","last_undegraded":"2024-03-28T16:51:19.105540+0000","last_fullsized":"2024-03-28T16:51:19.105540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.105244+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:35:56.759918+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'3","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.232984+0000","last_change":"2024-03-28T16:51:29.233046+0000","last_active":"2024-03-28T16:51:29.232984+0000","last_peered":"2024-03-28T16:51:29.232984+0000","last_clean":"2024-03-28T16:51:29.232984+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:51:29.232984+0000","last_undegraded":"2024-03-28T16:51:29.232984+0000","last_fullsized":"2024-03-28T16:51:29.232984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:51:29.232791+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:51:29.232791+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.232791+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:46.608812+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.197100+0000","last_change":"2024-03-28T16:51:37.197158+0000","last_active":"2024-03-28T16:51:37.197100+0000","last_peered":"2024-03-28T16:51:37.197100+0000","last_clean":"2024-03-28T16:51:37.197100+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:51:37.197100+0000","last_undegraded":"2024-03-28T16:51:37.197100+0000","last_fullsized":"2024-03-28T16:51:37.197100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:37.196925+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:37.196925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.196925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:33.164623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:52.054519+0000","last_change":"2024-03-28T16:51:52.054648+0000","last_active":"2024-03-28T16:51:52.054519+0000","last_peered":"2024-03-28T16:51:52.054519+0000","last_clean":"2024-03-28T16:51:52.054519+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:51:52.054519+0000","last_undegraded":"2024-03-28T16:51:52.054519+0000","last_fullsized":"2024-03-28T16:51:52.054519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:52.054305+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:52.054305+0000","last_clean_scrub_stamp":"2024-03-28T16:51:52.054305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:28.349668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.305502+0000","last_change":"2024-03-28T16:51:16.305559+0000","last_active":"2024-03-28T16:51:16.305502+0000","last_peered":"2024-03-28T16:51:16.305502+0000","last_clean":"2024-03-28T16:51:16.305502+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:51:16.305502+0000","last_undegraded":"2024-03-28T16:51:16.305502+0000","last_fullsized":"2024-03-28T16:51:16.305502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.305318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:43.571649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.321896+0000","last_change":"2024-03-28T16:51:21.321896+0000","last_active":"2024-03-28T16:51:21.321896+0000","last_peered":"2024-03-28T16:51:21.321896+0000","last_clean":"2024-03-28T16:51:21.321896+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:51:21.321896+0000","last_undegraded":"2024-03-28T16:51:21.321896+0000","last_fullsized":"2024-03-28T16:51:21.321896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.321658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:30.540834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.057665+0000","last_change":"2024-03-28T16:51:20.057727+0000","last_active":"2024-03-28T16:51:20.057665+0000","last_peered":"2024-03-28T16:51:20.057665+0000","last_clean":"2024-03-28T16:51:20.057665+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:51:20.057665+0000","last_undegraded":"2024-03-28T16:51:20.057665+0000","last_fullsized":"2024-03-28T16:51:20.057665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.057477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:08.878405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:47.069078+0000","last_change":"2024-03-28T16:51:47.069144+0000","last_active":"2024-03-28T16:51:47.069078+0000","last_peered":"2024-03-28T16:51:47.069078+0000","last_clean":"2024-03-28T16:51:47.069078+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:51:47.069078+0000","last_undegraded":"2024-03-28T16:51:47.069078+0000","last_fullsized":"2024-03-28T16:51:47.069078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:47.068883+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:47.068883+0000","last_clean_scrub_stamp":"2024-03-28T16:51:47.068883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:46:12.543407+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.206567+0000","last_change":"2024-03-28T16:51:34.206624+0000","last_active":"2024-03-28T16:51:34.206567+0000","last_peered":"2024-03-28T16:51:34.206567+0000","last_clean":"2024-03-28T16:51:34.206567+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:51:34.206567+0000","last_undegraded":"2024-03-28T16:51:34.206567+0000","last_fullsized":"2024-03-28T16:51:34.206567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:51:34.206377+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:51:34.206377+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.206377+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:21.188653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.467047+0000","last_change":"2024-03-28T16:51:39.467111+0000","last_active":"2024-03-28T16:51:39.467047+0000","last_peered":"2024-03-28T16:51:39.467047+0000","last_clean":"2024-03-28T16:51:39.467047+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:51:39.467047+0000","last_undegraded":"2024-03-28T16:51:39.467047+0000","last_fullsized":"2024-03-28T16:51:39.467047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.466858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.466858+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.466858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:31.475945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.353532+0000","last_change":"2024-03-28T16:51:12.353604+0000","last_active":"2024-03-28T16:51:12.353532+0000","last_peered":"2024-03-28T16:51:12.353532+0000","last_clean":"2024-03-28T16:51:12.353532+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:51:12.353532+0000","last_undegraded":"2024-03-28T16:51:12.353532+0000","last_fullsized":"2024-03-28T16:51:12.353532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.353309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:42.028446+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.145909+0000","last_change":"2024-03-28T16:51:28.145970+0000","last_active":"2024-03-28T16:51:28.145909+0000","last_peered":"2024-03-28T16:51:28.145909+0000","last_clean":"2024-03-28T16:51:28.145909+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:51:28.145909+0000","last_undegraded":"2024-03-28T16:51:28.145909+0000","last_fullsized":"2024-03-28T16:51:28.145909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:28.145685+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:28.145685+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.145685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:12.221861+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.234222+0000","last_change":"2024-03-28T16:51:28.234290+0000","last_active":"2024-03-28T16:51:28.234222+0000","last_peered":"2024-03-28T16:51:28.234222+0000","last_clean":"2024-03-28T16:51:28.234222+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:51:28.234222+0000","last_undegraded":"2024-03-28T16:51:28.234222+0000","last_fullsized":"2024-03-28T16:51:28.234222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:28.234045+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:28.234045+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.234045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:10:44.817774+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'10","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.246752+0000","last_change":"2024-03-28T16:51:07.246752+0000","last_active":"2024-03-28T16:51:07.246752+0000","last_peered":"2024-03-28T16:51:07.246752+0000","last_clean":"2024-03-28T16:51:07.246752+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:51:07.246752+0000","last_undegraded":"2024-03-28T16:51:07.246752+0000","last_fullsized":"2024-03-28T16:51:07.246752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.246384+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:42.040943+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.241054+0000","last_change":"2024-03-28T16:51:39.241109+0000","last_active":"2024-03-28T16:51:39.241054+0000","last_peered":"2024-03-28T16:51:39.241054+0000","last_clean":"2024-03-28T16:51:39.241054+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:51:39.241054+0000","last_undegraded":"2024-03-28T16:51:39.241054+0000","last_fullsized":"2024-03-28T16:51:39.241054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.240890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.240890+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.240890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:00.740460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.251563+0000","last_change":"2024-03-28T16:51:08.251563+0000","last_active":"2024-03-28T16:51:08.251563+0000","last_peered":"2024-03-28T16:51:08.251563+0000","last_clean":"2024-03-28T16:51:08.251563+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:51:08.251563+0000","last_undegraded":"2024-03-28T16:51:08.251563+0000","last_fullsized":"2024-03-28T16:51:08.251563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.251179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:34.785916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.213499+0000","last_change":"2024-03-28T16:51:30.213568+0000","last_active":"2024-03-28T16:51:30.213499+0000","last_peered":"2024-03-28T16:51:30.213499+0000","last_clean":"2024-03-28T16:51:30.213499+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:51:30.213499+0000","last_undegraded":"2024-03-28T16:51:30.213499+0000","last_fullsized":"2024-03-28T16:51:30.213499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.213281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.213281+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.213281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:48.948290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.549877+0000","last_change":"2024-03-28T16:51:25.549934+0000","last_active":"2024-03-28T16:51:25.549877+0000","last_peered":"2024-03-28T16:51:25.549877+0000","last_clean":"2024-03-28T16:51:25.549877+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:51:25.549877+0000","last_undegraded":"2024-03-28T16:51:25.549877+0000","last_fullsized":"2024-03-28T16:51:25.549877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:25.549686+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:25.549686+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.549686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:12.531962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.161305+0000","last_change":"2024-03-28T16:51:33.161357+0000","last_active":"2024-03-28T16:51:33.161305+0000","last_peered":"2024-03-28T16:51:33.161305+0000","last_clean":"2024-03-28T16:51:33.161305+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:51:33.161305+0000","last_undegraded":"2024-03-28T16:51:33.161305+0000","last_fullsized":"2024-03-28T16:51:33.161305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.161112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.161112+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.161112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:23.989489+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.320871+0000","last_change":"2024-03-28T16:51:11.320871+0000","last_active":"2024-03-28T16:51:11.320871+0000","last_peered":"2024-03-28T16:51:11.320871+0000","last_clean":"2024-03-28T16:51:11.320871+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:51:11.320871+0000","last_undegraded":"2024-03-28T16:51:11.320871+0000","last_fullsized":"2024-03-28T16:51:11.320871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.320440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:29.097635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.480322+0000","last_change":"2024-03-28T16:51:20.480322+0000","last_active":"2024-03-28T16:51:20.480322+0000","last_peered":"2024-03-28T16:51:20.480322+0000","last_clean":"2024-03-28T16:51:20.480322+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:51:20.480322+0000","last_undegraded":"2024-03-28T16:51:20.480322+0000","last_fullsized":"2024-03-28T16:51:20.480322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.479971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:37.855992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.332059+0000","last_change":"2024-03-28T16:51:15.332059+0000","last_active":"2024-03-28T16:51:15.332059+0000","last_peered":"2024-03-28T16:51:15.332059+0000","last_clean":"2024-03-28T16:51:15.332059+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:51:15.332059+0000","last_undegraded":"2024-03-28T16:51:15.332059+0000","last_fullsized":"2024-03-28T16:51:15.332059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.331796+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:37:44.933269+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:51.016244+0000","last_change":"2024-03-28T16:51:51.016313+0000","last_active":"2024-03-28T16:51:51.016244+0000","last_peered":"2024-03-28T16:51:51.016244+0000","last_clean":"2024-03-28T16:51:51.016244+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:51:51.016244+0000","last_undegraded":"2024-03-28T16:51:51.016244+0000","last_fullsized":"2024-03-28T16:51:51.016244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:51.016042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:51.016042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:51.016042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:47:25.750167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'20","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.046926+0000","last_change":"2024-03-28T16:51:40.046995+0000","last_active":"2024-03-28T16:51:40.046926+0000","last_peered":"2024-03-28T16:51:40.046926+0000","last_clean":"2024-03-28T16:51:40.046926+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:51:40.046926+0000","last_undegraded":"2024-03-28T16:51:40.046926+0000","last_fullsized":"2024-03-28T16:51:40.046926+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'20","last_scrub_stamp":"2024-03-28T16:51:40.046736+0000","last_deep_scrub":"31'20","last_deep_scrub_stamp":"2024-03-28T16:51:40.046736+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.046736+0000","objects_scrubbed":1,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:55:13.676549+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.132778+0000","last_change":"2024-03-28T16:51:24.132846+0000","last_active":"2024-03-28T16:51:24.132778+0000","last_peered":"2024-03-28T16:51:24.132778+0000","last_clean":"2024-03-28T16:51:24.132778+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:51:24.132778+0000","last_undegraded":"2024-03-28T16:51:24.132778+0000","last_fullsized":"2024-03-28T16:51:24.132778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:24.132547+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:24.132547+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.132547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:23.773840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:45.028278+0000","last_change":"2024-03-28T16:51:45.028430+0000","last_active":"2024-03-28T16:51:45.028278+0000","last_peered":"2024-03-28T16:51:45.028278+0000","last_clean":"2024-03-28T16:51:45.028278+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:51:45.028278+0000","last_undegraded":"2024-03-28T16:51:45.028278+0000","last_fullsized":"2024-03-28T16:51:45.028278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:45.028042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:45.028042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:45.028042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:31:06.107975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.138003+0000","last_change":"2024-03-28T16:50:50.138003+0000","last_active":"2024-03-28T16:50:50.138003+0000","last_peered":"2024-03-28T16:50:50.138003+0000","last_clean":"2024-03-28T16:50:50.138003+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:50:50.138003+0000","last_undegraded":"2024-03-28T16:50:50.138003+0000","last_fullsized":"2024-03-28T16:50:50.138003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.137765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:11.908122+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.226657+0000","last_change":"2024-03-28T16:51:10.226657+0000","last_active":"2024-03-28T16:51:10.226657+0000","last_peered":"2024-03-28T16:51:10.226657+0000","last_clean":"2024-03-28T16:51:10.226657+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:51:10.226657+0000","last_undegraded":"2024-03-28T16:51:10.226657+0000","last_fullsized":"2024-03-28T16:51:10.226657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.226407+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:01.054784+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.251369+0000","last_change":"2024-03-28T16:50:57.251369+0000","last_active":"2024-03-28T16:50:57.251369+0000","last_peered":"2024-03-28T16:50:57.251369+0000","last_clean":"2024-03-28T16:50:57.251369+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:50:57.251369+0000","last_undegraded":"2024-03-28T16:50:57.251369+0000","last_fullsized":"2024-03-28T16:50:57.251369+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.251130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:45.232439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'7","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.077837+0000","last_change":"2024-03-28T16:51:41.077908+0000","last_active":"2024-03-28T16:51:41.077837+0000","last_peered":"2024-03-28T16:51:41.077837+0000","last_clean":"2024-03-28T16:51:41.077837+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:51:41.077837+0000","last_undegraded":"2024-03-28T16:51:41.077837+0000","last_fullsized":"2024-03-28T16:51:41.077837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'7","last_scrub_stamp":"2024-03-28T16:51:41.077657+0000","last_deep_scrub":"31'7","last_deep_scrub_stamp":"2024-03-28T16:51:41.077657+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.077657+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:33.086744+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.082932+0000","last_change":"2024-03-28T16:51:34.082995+0000","last_active":"2024-03-28T16:51:34.082932+0000","last_peered":"2024-03-28T16:51:34.082932+0000","last_clean":"2024-03-28T16:51:34.082932+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:51:34.082932+0000","last_undegraded":"2024-03-28T16:51:34.082932+0000","last_fullsized":"2024-03-28T16:51:34.082932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:34.082730+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:34.082730+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.082730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:12:07.452135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.286192+0000","last_change":"2024-03-28T16:51:27.286244+0000","last_active":"2024-03-28T16:51:27.286192+0000","last_peered":"2024-03-28T16:51:27.286192+0000","last_clean":"2024-03-28T16:51:27.286192+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:51:27.286192+0000","last_undegraded":"2024-03-28T16:51:27.286192+0000","last_fullsized":"2024-03-28T16:51:27.286192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:27.286005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:27.286005+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.286005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:00.366383+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.328962+0000","last_change":"2024-03-28T16:51:04.328962+0000","last_active":"2024-03-28T16:51:04.328962+0000","last_peered":"2024-03-28T16:51:04.328962+0000","last_clean":"2024-03-28T16:51:04.328962+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:51:04.328962+0000","last_undegraded":"2024-03-28T16:51:04.328962+0000","last_fullsized":"2024-03-28T16:51:04.328962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.328732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:30.892979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"31'10","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.284276+0000","last_change":"2024-03-28T16:50:55.284276+0000","last_active":"2024-03-28T16:50:55.284276+0000","last_peered":"2024-03-28T16:50:55.284276+0000","last_clean":"2024-03-28T16:50:55.284276+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:50:55.284276+0000","last_undegraded":"2024-03-28T16:50:55.284276+0000","last_fullsized":"2024-03-28T16:50:55.284276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.283889+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:07.952223+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.156607+0000","last_change":"2024-03-28T16:50:49.156607+0000","last_active":"2024-03-28T16:50:49.156607+0000","last_peered":"2024-03-28T16:50:49.156607+0000","last_clean":"2024-03-28T16:50:49.156607+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:50:49.156607+0000","last_undegraded":"2024-03-28T16:50:49.156607+0000","last_fullsized":"2024-03-28T16:50:49.156607+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.156374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:19.754691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:46.532606+0000","last_change":"2024-03-28T16:51:46.532668+0000","last_active":"2024-03-28T16:51:46.532606+0000","last_peered":"2024-03-28T16:51:46.532606+0000","last_clean":"2024-03-28T16:51:46.532606+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:51:46.532606+0000","last_undegraded":"2024-03-28T16:51:46.532606+0000","last_fullsized":"2024-03-28T16:51:46.532606+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:46.532395+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:46.532395+0000","last_clean_scrub_stamp":"2024-03-28T16:51:46.532395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:10.071486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":68,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.147922+0000","last_change":"2024-03-28T16:51:30.147995+0000","last_active":"2024-03-28T16:51:30.147922+0000","last_peered":"2024-03-28T16:51:30.147922+0000","last_clean":"2024-03-28T16:51:30.147922+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:51:30.147922+0000","last_undegraded":"2024-03-28T16:51:30.147922+0000","last_fullsized":"2024-03-28T16:51:30.147922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:30.147724+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:30.147724+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.147724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:42:47.609297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:54.069947+0000","last_change":"2024-03-28T16:51:54.069999+0000","last_active":"2024-03-28T16:51:54.069947+0000","last_peered":"2024-03-28T16:51:54.069947+0000","last_clean":"2024-03-28T16:51:54.069947+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:51:54.069947+0000","last_undegraded":"2024-03-28T16:51:54.069947+0000","last_fullsized":"2024-03-28T16:51:54.069947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:54.069722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:54.069722+0000","last_clean_scrub_stamp":"2024-03-28T16:51:54.069722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:51:26.808328+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'6","reported_seq":216,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.337837+0000","last_change":"2024-03-28T16:51:08.337837+0000","last_active":"2024-03-28T16:51:08.337837+0000","last_peered":"2024-03-28T16:51:08.337837+0000","last_clean":"2024-03-28T16:51:08.337837+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:51:08.337837+0000","last_undegraded":"2024-03-28T16:51:08.337837+0000","last_fullsized":"2024-03-28T16:51:08.337837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.337418+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:50.313341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":270,"num_read_kb":269,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'16","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.357686+0000","last_change":"2024-03-28T16:50:59.357686+0000","last_active":"2024-03-28T16:50:59.357686+0000","last_peered":"2024-03-28T16:50:59.357686+0000","last_clean":"2024-03-28T16:50:59.357686+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:50:59.357686+0000","last_undegraded":"2024-03-28T16:50:59.357686+0000","last_fullsized":"2024-03-28T16:50:59.357686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.357393+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:54.820495+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'249","reported_seq":342,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.231850+0000","last_change":"2024-03-28T16:51:08.231850+0000","last_active":"2024-03-28T16:51:08.231850+0000","last_peered":"2024-03-28T16:51:08.231850+0000","last_clean":"2024-03-28T16:51:08.231850+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:51:08.231850+0000","last_undegraded":"2024-03-28T16:51:08.231850+0000","last_fullsized":"2024-03-28T16:51:08.231850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.231583+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:26.966593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.350955+0000","last_change":"2024-03-28T16:51:04.350955+0000","last_active":"2024-03-28T16:51:04.350955+0000","last_peered":"2024-03-28T16:51:04.350955+0000","last_clean":"2024-03-28T16:51:04.350955+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:51:04.350955+0000","last_undegraded":"2024-03-28T16:51:04.350955+0000","last_fullsized":"2024-03-28T16:51:04.350955+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.350588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:52.331897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.382333+0000","last_change":"2024-03-28T16:51:14.382333+0000","last_active":"2024-03-28T16:51:14.382333+0000","last_peered":"2024-03-28T16:51:14.382333+0000","last_clean":"2024-03-28T16:51:14.382333+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:51:14.382333+0000","last_undegraded":"2024-03-28T16:51:14.382333+0000","last_fullsized":"2024-03-28T16:51:14.382333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.382083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:28.408578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.082705+0000","last_change":"2024-03-28T16:51:35.082806+0000","last_active":"2024-03-28T16:51:35.082705+0000","last_peered":"2024-03-28T16:51:35.082705+0000","last_clean":"2024-03-28T16:51:35.082705+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:51:35.082705+0000","last_undegraded":"2024-03-28T16:51:35.082705+0000","last_fullsized":"2024-03-28T16:51:35.082705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:35.082491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:35.082491+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.082491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:11.308208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.124265+0000","last_change":"2024-03-28T16:50:49.124265+0000","last_active":"2024-03-28T16:50:49.124265+0000","last_peered":"2024-03-28T16:50:49.124265+0000","last_clean":"2024-03-28T16:50:49.124265+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:50:49.124265+0000","last_undegraded":"2024-03-28T16:50:49.124265+0000","last_fullsized":"2024-03-28T16:50:49.124265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.124014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.365150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.425199+0000","last_change":"2024-03-28T16:50:57.425199+0000","last_active":"2024-03-28T16:50:57.425199+0000","last_peered":"2024-03-28T16:50:57.425199+0000","last_clean":"2024-03-28T16:50:57.425199+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:57.425199+0000","last_undegraded":"2024-03-28T16:50:57.425199+0000","last_fullsized":"2024-03-28T16:50:57.425199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.424946+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:19.309481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'249","reported_seq":357,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.296501+0000","last_change":"2024-03-28T16:51:09.296501+0000","last_active":"2024-03-28T16:51:09.296501+0000","last_peered":"2024-03-28T16:51:09.296501+0000","last_clean":"2024-03-28T16:51:09.296501+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:51:09.296501+0000","last_undegraded":"2024-03-28T16:51:09.296501+0000","last_fullsized":"2024-03-28T16:51:09.296501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.296245+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:57.394089+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.285771+0000","last_change":"2024-03-28T16:51:06.285771+0000","last_active":"2024-03-28T16:51:06.285771+0000","last_peered":"2024-03-28T16:51:06.285771+0000","last_clean":"2024-03-28T16:51:06.285771+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:51:06.285771+0000","last_undegraded":"2024-03-28T16:51:06.285771+0000","last_fullsized":"2024-03-28T16:51:06.285771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.285504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:11.465640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.389203+0000","last_change":"2024-03-28T16:51:17.389264+0000","last_active":"2024-03-28T16:51:17.389203+0000","last_peered":"2024-03-28T16:51:17.389203+0000","last_clean":"2024-03-28T16:51:17.389203+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:51:17.389203+0000","last_undegraded":"2024-03-28T16:51:17.389203+0000","last_fullsized":"2024-03-28T16:51:17.389203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.389016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:59.364938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.302414+0000","last_change":"2024-03-28T16:51:00.302414+0000","last_active":"2024-03-28T16:51:00.302414+0000","last_peered":"2024-03-28T16:51:00.302414+0000","last_clean":"2024-03-28T16:51:00.302414+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:51:00.302414+0000","last_undegraded":"2024-03-28T16:51:00.302414+0000","last_fullsized":"2024-03-28T16:51:00.302414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.302183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:46.487988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'49","reported_seq":109,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.444852+0000","last_change":"2024-03-28T16:50:55.444852+0000","last_active":"2024-03-28T16:50:55.444852+0000","last_peered":"2024-03-28T16:50:55.444852+0000","last_clean":"2024-03-28T16:50:55.444852+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:55.444852+0000","last_undegraded":"2024-03-28T16:50:55.444852+0000","last_fullsized":"2024-03-28T16:50:55.444852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'49","last_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_deep_scrub":"31'49","last_deep_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.444545+0000","objects_scrubbed":9,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:56.707599+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.195172+0000","last_change":"2024-03-28T16:50:53.195172+0000","last_active":"2024-03-28T16:50:53.195172+0000","last_peered":"2024-03-28T16:50:53.195172+0000","last_clean":"2024-03-28T16:50:53.195172+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:50:53.195172+0000","last_undegraded":"2024-03-28T16:50:53.195172+0000","last_fullsized":"2024-03-28T16:50:53.195172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.194894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:05.863228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.397864+0000","last_change":"2024-03-28T16:50:56.397864+0000","last_active":"2024-03-28T16:50:56.397864+0000","last_peered":"2024-03-28T16:50:56.397864+0000","last_clean":"2024-03-28T16:50:56.397864+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:56.397864+0000","last_undegraded":"2024-03-28T16:50:56.397864+0000","last_fullsized":"2024-03-28T16:50:56.397864+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.397465+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:26.995782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'24","reported_seq":112,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.267353+0000","last_change":"2024-03-28T16:51:17.267421+0000","last_active":"2024-03-28T16:51:17.267353+0000","last_peered":"2024-03-28T16:51:17.267353+0000","last_clean":"2024-03-28T16:51:17.267353+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:51:17.267353+0000","last_undegraded":"2024-03-28T16:51:17.267353+0000","last_fullsized":"2024-03-28T16:51:17.267353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'24","last_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_deep_scrub":"31'24","last_deep_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.267112+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:31.293607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.358051+0000","last_change":"2024-03-28T16:51:15.358174+0000","last_active":"2024-03-28T16:51:15.358051+0000","last_peered":"2024-03-28T16:51:15.358051+0000","last_clean":"2024-03-28T16:51:15.358051+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:51:15.358051+0000","last_undegraded":"2024-03-28T16:51:15.358051+0000","last_fullsized":"2024-03-28T16:51:15.358051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.357818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:41.892555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.371282+0000","last_change":"2024-03-28T16:51:02.371282+0000","last_active":"2024-03-28T16:51:02.371282+0000","last_peered":"2024-03-28T16:51:02.371282+0000","last_clean":"2024-03-28T16:51:02.371282+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:51:02.371282+0000","last_undegraded":"2024-03-28T16:51:02.371282+0000","last_fullsized":"2024-03-28T16:51:02.371282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.371035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:20.148230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.191604+0000","last_change":"2024-03-28T16:51:11.191604+0000","last_active":"2024-03-28T16:51:11.191604+0000","last_peered":"2024-03-28T16:51:11.191604+0000","last_clean":"2024-03-28T16:51:11.191604+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:51:11.191604+0000","last_undegraded":"2024-03-28T16:51:11.191604+0000","last_fullsized":"2024-03-28T16:51:11.191604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.191343+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:47:25.401107+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.183048+0000","last_change":"2024-03-28T16:50:51.183048+0000","last_active":"2024-03-28T16:50:51.183048+0000","last_peered":"2024-03-28T16:50:51.183048+0000","last_clean":"2024-03-28T16:50:51.183048+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:50:51.183048+0000","last_undegraded":"2024-03-28T16:50:51.183048+0000","last_fullsized":"2024-03-28T16:50:51.183048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.182678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:48.071769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'4","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.320355+0000","last_change":"2024-03-28T16:51:05.320355+0000","last_active":"2024-03-28T16:51:05.320355+0000","last_peered":"2024-03-28T16:51:05.320355+0000","last_clean":"2024-03-28T16:51:05.320355+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:51:05.320355+0000","last_undegraded":"2024-03-28T16:51:05.320355+0000","last_fullsized":"2024-03-28T16:51:05.320355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.320076+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:34.995213+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.314961+0000","last_change":"2024-03-28T16:51:14.315025+0000","last_active":"2024-03-28T16:51:14.314961+0000","last_peered":"2024-03-28T16:51:14.314961+0000","last_clean":"2024-03-28T16:51:14.314961+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:51:14.314961+0000","last_undegraded":"2024-03-28T16:51:14.314961+0000","last_fullsized":"2024-03-28T16:51:14.314961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.314772+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:49.109332+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.126840+0000","last_change":"2024-03-28T16:51:38.126897+0000","last_active":"2024-03-28T16:51:38.126840+0000","last_peered":"2024-03-28T16:51:38.126840+0000","last_clean":"2024-03-28T16:51:38.126840+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:51:38.126840+0000","last_undegraded":"2024-03-28T16:51:38.126840+0000","last_fullsized":"2024-03-28T16:51:38.126840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.126656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.126656+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.126656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:49.967866+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.352892+0000","last_change":"2024-03-28T16:51:11.352892+0000","last_active":"2024-03-28T16:51:11.352892+0000","last_peered":"2024-03-28T16:51:11.352892+0000","last_clean":"2024-03-28T16:51:11.352892+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:51:11.352892+0000","last_undegraded":"2024-03-28T16:51:11.352892+0000","last_fullsized":"2024-03-28T16:51:11.352892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.352650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:16.271950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.096269+0000","last_change":"2024-03-28T16:51:18.096365+0000","last_active":"2024-03-28T16:51:18.096269+0000","last_peered":"2024-03-28T16:51:18.096269+0000","last_clean":"2024-03-28T16:51:18.096269+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:51:18.096269+0000","last_undegraded":"2024-03-28T16:51:18.096269+0000","last_fullsized":"2024-03-28T16:51:18.096269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.096048+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:10.105136+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'249","reported_seq":352,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.171801+0000","last_change":"2024-03-28T16:50:52.171801+0000","last_active":"2024-03-28T16:50:52.171801+0000","last_peered":"2024-03-28T16:50:52.171801+0000","last_clean":"2024-03-28T16:50:52.171801+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:52.171801+0000","last_undegraded":"2024-03-28T16:50:52.171801+0000","last_fullsized":"2024-03-28T16:50:52.171801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.171465+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:19.659817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:45.557316+0000","last_change":"2024-03-28T16:51:45.557374+0000","last_active":"2024-03-28T16:51:45.557316+0000","last_peered":"2024-03-28T16:51:45.557316+0000","last_clean":"2024-03-28T16:51:45.557316+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:51:45.557316+0000","last_undegraded":"2024-03-28T16:51:45.557316+0000","last_fullsized":"2024-03-28T16:51:45.557316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:45.557138+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:45.557138+0000","last_clean_scrub_stamp":"2024-03-28T16:51:45.557138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:45.198431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.394496+0000","last_change":"2024-03-28T16:51:18.394559+0000","last_active":"2024-03-28T16:51:18.394496+0000","last_peered":"2024-03-28T16:51:18.394496+0000","last_clean":"2024-03-28T16:51:18.394496+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:51:18.394496+0000","last_undegraded":"2024-03-28T16:51:18.394496+0000","last_fullsized":"2024-03-28T16:51:18.394496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.394309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:06.492152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:01.981153+0000","last_change":"2024-03-28T16:52:01.981221+0000","last_active":"2024-03-28T16:52:01.981153+0000","last_peered":"2024-03-28T16:52:01.981153+0000","last_clean":"2024-03-28T16:52:01.981153+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:52:01.981153+0000","last_undegraded":"2024-03-28T16:52:01.981153+0000","last_fullsized":"2024-03-28T16:52:01.981153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:52:01.980962+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:52:01.980962+0000","last_clean_scrub_stamp":"2024-03-28T16:52:01.980962+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:45:01.171370+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.386277+0000","last_change":"2024-03-28T16:51:12.386390+0000","last_active":"2024-03-28T16:51:12.386277+0000","last_peered":"2024-03-28T16:51:12.386277+0000","last_clean":"2024-03-28T16:51:12.386277+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:51:12.386277+0000","last_undegraded":"2024-03-28T16:51:12.386277+0000","last_fullsized":"2024-03-28T16:51:12.386277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.386042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:21.605218+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'61","reported_seq":134,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.103088+0000","last_change":"2024-03-28T16:51:14.103172+0000","last_active":"2024-03-28T16:51:14.103088+0000","last_peered":"2024-03-28T16:51:14.103088+0000","last_clean":"2024-03-28T16:51:14.103088+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:51:14.103088+0000","last_undegraded":"2024-03-28T16:51:14.103088+0000","last_fullsized":"2024-03-28T16:51:14.103088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'61","last_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_deep_scrub":"31'61","last_deep_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.102888+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:13:35.860974+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.181042+0000","last_change":"2024-03-28T16:50:54.181042+0000","last_active":"2024-03-28T16:50:54.181042+0000","last_peered":"2024-03-28T16:50:54.181042+0000","last_clean":"2024-03-28T16:50:54.181042+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:50:54.181042+0000","last_undegraded":"2024-03-28T16:50:54.181042+0000","last_fullsized":"2024-03-28T16:50:54.181042+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.180810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:17.080003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.393450+0000","last_change":"2024-03-28T16:51:13.393450+0000","last_active":"2024-03-28T16:51:13.393450+0000","last_peered":"2024-03-28T16:51:13.393450+0000","last_clean":"2024-03-28T16:51:13.393450+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:51:13.393450+0000","last_undegraded":"2024-03-28T16:51:13.393450+0000","last_fullsized":"2024-03-28T16:51:13.393450+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.393212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:22.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.203924+0000","last_change":"2024-03-28T16:51:27.203985+0000","last_active":"2024-03-28T16:51:27.203924+0000","last_peered":"2024-03-28T16:51:27.203924+0000","last_clean":"2024-03-28T16:51:27.203924+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:51:27.203924+0000","last_undegraded":"2024-03-28T16:51:27.203924+0000","last_fullsized":"2024-03-28T16:51:27.203924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:27.203737+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:27.203737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.203737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:14.186013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.551323+0000","last_change":"2024-03-28T16:51:28.551377+0000","last_active":"2024-03-28T16:51:28.551323+0000","last_peered":"2024-03-28T16:51:28.551323+0000","last_clean":"2024-03-28T16:51:28.551323+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:51:28.551323+0000","last_undegraded":"2024-03-28T16:51:28.551323+0000","last_fullsized":"2024-03-28T16:51:28.551323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:28.551113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:28.551113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.551113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:24.518811+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.365035+0000","last_change":"2024-03-28T16:51:05.365035+0000","last_active":"2024-03-28T16:51:05.365035+0000","last_peered":"2024-03-28T16:51:05.365035+0000","last_clean":"2024-03-28T16:51:05.365035+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:51:05.365035+0000","last_undegraded":"2024-03-28T16:51:05.365035+0000","last_fullsized":"2024-03-28T16:51:05.365035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.364765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:50.121796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'37","reported_seq":93,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.221214+0000","last_change":"2024-03-28T16:50:51.221214+0000","last_active":"2024-03-28T16:50:51.221214+0000","last_peered":"2024-03-28T16:50:51.221214+0000","last_clean":"2024-03-28T16:50:51.221214+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:51.221214+0000","last_undegraded":"2024-03-28T16:50:51.221214+0000","last_fullsized":"2024-03-28T16:50:51.221214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'37","last_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_deep_scrub":"31'37","last_deep_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.220960+0000","objects_scrubbed":5,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:47.323997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.275109+0000","last_change":"2024-03-28T16:51:07.275109+0000","last_active":"2024-03-28T16:51:07.275109+0000","last_peered":"2024-03-28T16:51:07.275109+0000","last_clean":"2024-03-28T16:51:07.275109+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:51:07.275109+0000","last_undegraded":"2024-03-28T16:51:07.275109+0000","last_fullsized":"2024-03-28T16:51:07.275109+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.274868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:03.242566+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.076599+0000","last_change":"2024-03-28T16:51:37.076651+0000","last_active":"2024-03-28T16:51:37.076599+0000","last_peered":"2024-03-28T16:51:37.076599+0000","last_clean":"2024-03-28T16:51:37.076599+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:51:37.076599+0000","last_undegraded":"2024-03-28T16:51:37.076599+0000","last_fullsized":"2024-03-28T16:51:37.076599+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.076411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.076411+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.076411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:47.198210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.193692+0000","last_change":"2024-03-28T16:51:00.193692+0000","last_active":"2024-03-28T16:51:00.193692+0000","last_peered":"2024-03-28T16:51:00.193692+0000","last_clean":"2024-03-28T16:51:00.193692+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:51:00.193692+0000","last_undegraded":"2024-03-28T16:51:00.193692+0000","last_fullsized":"2024-03-28T16:51:00.193692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.193439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:35.074661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:49.006497+0000","last_change":"2024-03-28T16:51:49.006621+0000","last_active":"2024-03-28T16:51:49.006497+0000","last_peered":"2024-03-28T16:51:49.006497+0000","last_clean":"2024-03-28T16:51:49.006497+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:51:49.006497+0000","last_undegraded":"2024-03-28T16:51:49.006497+0000","last_fullsized":"2024-03-28T16:51:49.006497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:49.006205+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:49.006205+0000","last_clean_scrub_stamp":"2024-03-28T16:51:49.006205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:42.130598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.345768+0000","last_change":"2024-03-28T16:51:10.345768+0000","last_active":"2024-03-28T16:51:10.345768+0000","last_peered":"2024-03-28T16:51:10.345768+0000","last_clean":"2024-03-28T16:51:10.345768+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:51:10.345768+0000","last_undegraded":"2024-03-28T16:51:10.345768+0000","last_fullsized":"2024-03-28T16:51:10.345768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.345522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:45.930075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.389452+0000","last_change":"2024-03-28T16:51:01.389452+0000","last_active":"2024-03-28T16:51:01.389452+0000","last_peered":"2024-03-28T16:51:01.389452+0000","last_clean":"2024-03-28T16:51:01.389452+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:51:01.389452+0000","last_undegraded":"2024-03-28T16:51:01.389452+0000","last_fullsized":"2024-03-28T16:51:01.389452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.389173+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:28.877525+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'249","reported_seq":335,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.428834+0000","last_change":"2024-03-28T16:50:47.428834+0000","last_active":"2024-03-28T16:50:47.428834+0000","last_peered":"2024-03-28T16:50:47.428834+0000","last_clean":"2024-03-28T16:50:47.428834+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:47.428834+0000","last_undegraded":"2024-03-28T16:50:47.428834+0000","last_fullsized":"2024-03-28T16:50:47.428834+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.428456+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:01.404074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'14","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:47.490733+0000","last_change":"2024-03-28T16:51:47.490803+0000","last_active":"2024-03-28T16:51:47.490733+0000","last_peered":"2024-03-28T16:51:47.490733+0000","last_clean":"2024-03-28T16:51:47.490733+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:51:47.490733+0000","last_undegraded":"2024-03-28T16:51:47.490733+0000","last_fullsized":"2024-03-28T16:51:47.490733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'14","last_scrub_stamp":"2024-03-28T16:51:47.490545+0000","last_deep_scrub":"31'14","last_deep_scrub_stamp":"2024-03-28T16:51:47.490545+0000","last_clean_scrub_stamp":"2024-03-28T16:51:47.490545+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:29.772619+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":170,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.337241+0000","last_change":"2024-03-28T16:51:22.337362+0000","last_active":"2024-03-28T16:51:22.337241+0000","last_peered":"2024-03-28T16:51:22.337241+0000","last_clean":"2024-03-28T16:51:22.337241+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:51:22.337241+0000","last_undegraded":"2024-03-28T16:51:22.337241+0000","last_fullsized":"2024-03-28T16:51:22.337241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.336956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:31.109615+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:44.014911+0000","last_change":"2024-03-28T16:51:44.014970+0000","last_active":"2024-03-28T16:51:44.014911+0000","last_peered":"2024-03-28T16:51:44.014911+0000","last_clean":"2024-03-28T16:51:44.014911+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:51:44.014911+0000","last_undegraded":"2024-03-28T16:51:44.014911+0000","last_fullsized":"2024-03-28T16:51:44.014911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:44.014723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:44.014723+0000","last_clean_scrub_stamp":"2024-03-28T16:51:44.014723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:58:04.858719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'5","reported_seq":394,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:46.100936+0000","last_change":"2024-03-28T16:51:46.101089+0000","last_active":"2024-03-28T16:51:46.100936+0000","last_peered":"2024-03-28T16:51:46.100936+0000","last_clean":"2024-03-28T16:51:46.100936+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:51:46.100936+0000","last_undegraded":"2024-03-28T16:51:46.100936+0000","last_fullsized":"2024-03-28T16:51:46.100936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'5","last_scrub_stamp":"2024-03-28T16:51:46.100668+0000","last_deep_scrub":"31'5","last_deep_scrub_stamp":"2024-03-28T16:51:46.100668+0000","last_clean_scrub_stamp":"2024-03-28T16:51:46.100668+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:54.818277+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":340,"num_read_kb":340,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.144955+0000","last_change":"2024-03-28T16:51:12.144955+0000","last_active":"2024-03-28T16:51:12.144955+0000","last_peered":"2024-03-28T16:51:12.144955+0000","last_clean":"2024-03-28T16:51:12.144955+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:51:12.144955+0000","last_undegraded":"2024-03-28T16:51:12.144955+0000","last_fullsized":"2024-03-28T16:51:12.144955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.144489+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:36.042267+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'249","reported_seq":358,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.109690+0000","last_change":"2024-03-28T16:50:48.109690+0000","last_active":"2024-03-28T16:50:48.109690+0000","last_peered":"2024-03-28T16:50:48.109690+0000","last_clean":"2024-03-28T16:50:48.109690+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:48.109690+0000","last_undegraded":"2024-03-28T16:50:48.109690+0000","last_fullsized":"2024-03-28T16:50:48.109690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.109433+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:49.372487+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.412356+0000","last_change":"2024-03-28T16:51:08.412356+0000","last_active":"2024-03-28T16:51:08.412356+0000","last_peered":"2024-03-28T16:51:08.412356+0000","last_clean":"2024-03-28T16:51:08.412356+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:51:08.412356+0000","last_undegraded":"2024-03-28T16:51:08.412356+0000","last_fullsized":"2024-03-28T16:51:08.412356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.412104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:37.251504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.269854+0000","last_change":"2024-03-28T16:51:25.269969+0000","last_active":"2024-03-28T16:51:25.269854+0000","last_peered":"2024-03-28T16:51:25.269854+0000","last_clean":"2024-03-28T16:51:25.269854+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:51:25.269854+0000","last_undegraded":"2024-03-28T16:51:25.269854+0000","last_fullsized":"2024-03-28T16:51:25.269854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.269642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:53:12.659384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:03.003289+0000","last_change":"2024-03-28T16:52:03.003438+0000","last_active":"2024-03-28T16:52:03.003289+0000","last_peered":"2024-03-28T16:52:03.003289+0000","last_clean":"2024-03-28T16:52:03.003289+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:52:03.003289+0000","last_undegraded":"2024-03-28T16:52:03.003289+0000","last_fullsized":"2024-03-28T16:52:03.003289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:03.003033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:03.003033+0000","last_clean_scrub_stamp":"2024-03-28T16:52:03.003033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:49:50.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.165481+0000","last_change":"2024-03-28T16:50:48.165481+0000","last_active":"2024-03-28T16:50:48.165481+0000","last_peered":"2024-03-28T16:50:48.165481+0000","last_clean":"2024-03-28T16:50:48.165481+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:50:48.165481+0000","last_undegraded":"2024-03-28T16:50:48.165481+0000","last_fullsized":"2024-03-28T16:50:48.165481+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.165239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:40:14.898173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'46","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.272646+0000","last_change":"2024-03-28T16:50:56.272646+0000","last_active":"2024-03-28T16:50:56.272646+0000","last_peered":"2024-03-28T16:50:56.272646+0000","last_clean":"2024-03-28T16:50:56.272646+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:56.272646+0000","last_undegraded":"2024-03-28T16:50:56.272646+0000","last_fullsized":"2024-03-28T16:50:56.272646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'46","last_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_deep_scrub":"31'46","last_deep_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.272353+0000","objects_scrubbed":8,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:19.171546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.431537+0000","last_change":"2024-03-28T16:50:53.431537+0000","last_active":"2024-03-28T16:50:53.431537+0000","last_peered":"2024-03-28T16:50:53.431537+0000","last_clean":"2024-03-28T16:50:53.431537+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:50:53.431537+0000","last_undegraded":"2024-03-28T16:50:53.431537+0000","last_fullsized":"2024-03-28T16:50:53.431537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.431293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:35.681957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.348998+0000","last_change":"2024-03-28T16:51:01.348998+0000","last_active":"2024-03-28T16:51:01.348998+0000","last_peered":"2024-03-28T16:51:01.348998+0000","last_clean":"2024-03-28T16:51:01.348998+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:51:01.348998+0000","last_undegraded":"2024-03-28T16:51:01.348998+0000","last_fullsized":"2024-03-28T16:51:01.348998+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.348719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:30.188118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.197537+0000","last_change":"2024-03-28T16:51:03.197596+0000","last_active":"2024-03-28T16:51:03.197537+0000","last_peered":"2024-03-28T16:51:03.197537+0000","last_clean":"2024-03-28T16:51:03.197537+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:51:03.197537+0000","last_undegraded":"2024-03-28T16:51:03.197537+0000","last_fullsized":"2024-03-28T16:51:03.197537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.197346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:04:49.417548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"31'119","reported_seq":210,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.205423+0000","last_change":"2024-03-28T16:51:31.205497+0000","last_active":"2024-03-28T16:51:31.205423+0000","last_peered":"2024-03-28T16:51:31.205423+0000","last_clean":"2024-03-28T16:51:31.205423+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:51:31.205423+0000","last_undegraded":"2024-03-28T16:51:31.205423+0000","last_fullsized":"2024-03-28T16:51:31.205423+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'119","last_scrub_stamp":"2024-03-28T16:51:31.205230+0000","last_deep_scrub":"31'119","last_deep_scrub_stamp":"2024-03-28T16:51:31.205230+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.205230+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:47:05.795250+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:57.981295+0000","last_change":"2024-03-28T16:51:57.981352+0000","last_active":"2024-03-28T16:51:57.981295+0000","last_peered":"2024-03-28T16:51:57.981295+0000","last_clean":"2024-03-28T16:51:57.981295+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:51:57.981295+0000","last_undegraded":"2024-03-28T16:51:57.981295+0000","last_fullsized":"2024-03-28T16:51:57.981295+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:57.981069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:57.981069+0000","last_clean_scrub_stamp":"2024-03-28T16:51:57.981069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:37.245129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.064738+0000","last_change":"2024-03-28T16:51:41.064875+0000","last_active":"2024-03-28T16:51:41.064738+0000","last_peered":"2024-03-28T16:51:41.064738+0000","last_clean":"2024-03-28T16:51:41.064738+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:51:41.064738+0000","last_undegraded":"2024-03-28T16:51:41.064738+0000","last_fullsized":"2024-03-28T16:51:41.064738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:41.064504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:41.064504+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.064504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:51.380462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.295412+0000","last_change":"2024-03-28T16:51:03.295412+0000","last_active":"2024-03-28T16:51:03.295412+0000","last_peered":"2024-03-28T16:51:03.295412+0000","last_clean":"2024-03-28T16:51:03.295412+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:51:03.295412+0000","last_undegraded":"2024-03-28T16:51:03.295412+0000","last_fullsized":"2024-03-28T16:51:03.295412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.295139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:08.548996+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'81","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.160163+0000","last_change":"2024-03-28T16:50:53.160163+0000","last_active":"2024-03-28T16:50:53.160163+0000","last_peered":"2024-03-28T16:50:53.160163+0000","last_clean":"2024-03-28T16:50:53.160163+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:53.160163+0000","last_undegraded":"2024-03-28T16:50:53.160163+0000","last_fullsized":"2024-03-28T16:50:53.160163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'81","last_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_deep_scrub":"31'81","last_deep_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.159787+0000","objects_scrubbed":5,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:32.543735+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.290671+0000","last_change":"2024-03-28T16:50:57.290671+0000","last_active":"2024-03-28T16:50:57.290671+0000","last_peered":"2024-03-28T16:50:57.290671+0000","last_clean":"2024-03-28T16:50:57.290671+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:50:57.290671+0000","last_undegraded":"2024-03-28T16:50:57.290671+0000","last_fullsized":"2024-03-28T16:50:57.290671+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.290416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:32.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'53","reported_seq":125,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.247075+0000","last_change":"2024-03-28T16:51:35.247207+0000","last_active":"2024-03-28T16:51:35.247075+0000","last_peered":"2024-03-28T16:51:35.247075+0000","last_clean":"2024-03-28T16:51:35.247075+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:51:35.247075+0000","last_undegraded":"2024-03-28T16:51:35.247075+0000","last_fullsized":"2024-03-28T16:51:35.247075+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'53","last_scrub_stamp":"2024-03-28T16:51:35.246864+0000","last_deep_scrub":"31'53","last_deep_scrub_stamp":"2024-03-28T16:51:35.246864+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.246864+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:53.626119+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'12","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.204617+0000","last_change":"2024-03-28T16:51:40.204617+0000","last_active":"2024-03-28T16:51:40.204617+0000","last_peered":"2024-03-28T16:51:40.204617+0000","last_clean":"2024-03-28T16:51:40.204617+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:51:40.204617+0000","last_undegraded":"2024-03-28T16:51:40.204617+0000","last_fullsized":"2024-03-28T16:51:40.204617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:40.204359+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:40.204359+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.204359+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:32.212954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:55.983074+0000","last_change":"2024-03-28T16:51:55.983136+0000","last_active":"2024-03-28T16:51:55.983074+0000","last_peered":"2024-03-28T16:51:55.983074+0000","last_clean":"2024-03-28T16:51:55.983074+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:51:55.983074+0000","last_undegraded":"2024-03-28T16:51:55.983074+0000","last_fullsized":"2024-03-28T16:51:55.983074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:55.982888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:55.982888+0000","last_clean_scrub_stamp":"2024-03-28T16:51:55.982888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:23:08.339948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.432079+0000","last_change":"2024-03-28T16:51:41.432131+0000","last_active":"2024-03-28T16:51:41.432079+0000","last_peered":"2024-03-28T16:51:41.432079+0000","last_clean":"2024-03-28T16:51:41.432079+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:51:41.432079+0000","last_undegraded":"2024-03-28T16:51:41.432079+0000","last_fullsized":"2024-03-28T16:51:41.432079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:41.431882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:41.431882+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.431882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:48.943852+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.191091+0000","last_change":"2024-03-28T16:50:47.191091+0000","last_active":"2024-03-28T16:50:47.191091+0000","last_peered":"2024-03-28T16:50:47.191091+0000","last_clean":"2024-03-28T16:50:47.191091+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:50:47.191091+0000","last_undegraded":"2024-03-28T16:50:47.191091+0000","last_fullsized":"2024-03-28T16:50:47.191091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.190844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:09.946965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'260","reported_seq":455,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.259946+0000","last_change":"2024-03-28T16:50:54.259946+0000","last_active":"2024-03-28T16:50:54.259946+0000","last_peered":"2024-03-28T16:50:54.259946+0000","last_clean":"2024-03-28T16:50:54.259946+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:54.259946+0000","last_undegraded":"2024-03-28T16:50:54.259946+0000","last_fullsized":"2024-03-28T16:50:54.259946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'260","last_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_deep_scrub":"31'260","last_deep_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.259667+0000","objects_scrubbed":10,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:21.748018+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":342,"num_read_kb":327,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.065783+0000","last_change":"2024-03-28T16:51:21.065839+0000","last_active":"2024-03-28T16:51:21.065783+0000","last_peered":"2024-03-28T16:51:21.065783+0000","last_clean":"2024-03-28T16:51:21.065783+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:51:21.065783+0000","last_undegraded":"2024-03-28T16:51:21.065783+0000","last_fullsized":"2024-03-28T16:51:21.065783+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.065552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:21.706323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'5","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:42.068714+0000","last_change":"2024-03-28T16:51:42.068792+0000","last_active":"2024-03-28T16:51:42.068714+0000","last_peered":"2024-03-28T16:51:42.068714+0000","last_clean":"2024-03-28T16:51:42.068714+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:51:42.068714+0000","last_undegraded":"2024-03-28T16:51:42.068714+0000","last_fullsized":"2024-03-28T16:51:42.068714+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'5","last_scrub_stamp":"2024-03-28T16:51:42.068482+0000","last_deep_scrub":"31'5","last_deep_scrub_stamp":"2024-03-28T16:51:42.068482+0000","last_clean_scrub_stamp":"2024-03-28T16:51:42.068482+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:11.682779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:43.514905+0000","last_change":"2024-03-28T16:51:43.514968+0000","last_active":"2024-03-28T16:51:43.514905+0000","last_peered":"2024-03-28T16:51:43.514905+0000","last_clean":"2024-03-28T16:51:43.514905+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:51:43.514905+0000","last_undegraded":"2024-03-28T16:51:43.514905+0000","last_fullsized":"2024-03-28T16:51:43.514905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:43.514692+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:43.514692+0000","last_clean_scrub_stamp":"2024-03-28T16:51:43.514692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:47.685665+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.236189+0000","last_change":"2024-03-28T16:51:26.236274+0000","last_active":"2024-03-28T16:51:26.236189+0000","last_peered":"2024-03-28T16:51:26.236189+0000","last_clean":"2024-03-28T16:51:26.236189+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:51:26.236189+0000","last_undegraded":"2024-03-28T16:51:26.236189+0000","last_fullsized":"2024-03-28T16:51:26.236189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:26.235994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:26.235994+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.235994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:43.654590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:42.463543+0000","last_change":"2024-03-28T16:51:42.463599+0000","last_active":"2024-03-28T16:51:42.463543+0000","last_peered":"2024-03-28T16:51:42.463543+0000","last_clean":"2024-03-28T16:51:42.463543+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:51:42.463543+0000","last_undegraded":"2024-03-28T16:51:42.463543+0000","last_fullsized":"2024-03-28T16:51:42.463543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:42.463354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:42.463354+0000","last_clean_scrub_stamp":"2024-03-28T16:51:42.463354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:57:05.874302+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.414766+0000","last_change":"2024-03-28T16:51:07.414766+0000","last_active":"2024-03-28T16:51:07.414766+0000","last_peered":"2024-03-28T16:51:07.414766+0000","last_clean":"2024-03-28T16:51:07.414766+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:51:07.414766+0000","last_undegraded":"2024-03-28T16:51:07.414766+0000","last_fullsized":"2024-03-28T16:51:07.414766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.414513+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:53.600318+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'23","reported_seq":82,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.409006+0000","last_change":"2024-03-28T16:50:54.409006+0000","last_active":"2024-03-28T16:50:54.409006+0000","last_peered":"2024-03-28T16:50:54.409006+0000","last_clean":"2024-03-28T16:50:54.409006+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:54.409006+0000","last_undegraded":"2024-03-28T16:50:54.409006+0000","last_fullsized":"2024-03-28T16:50:54.409006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'23","last_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_deep_scrub":"31'23","last_deep_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.408718+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:56.786363+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.341170+0000","last_change":"2024-03-28T16:51:01.341170+0000","last_active":"2024-03-28T16:51:01.341170+0000","last_peered":"2024-03-28T16:51:01.341170+0000","last_clean":"2024-03-28T16:51:01.341170+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:51:01.341170+0000","last_undegraded":"2024-03-28T16:51:01.341170+0000","last_fullsized":"2024-03-28T16:51:01.341170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.340925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:06.594781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874154+0000","last_change":"2024-03-28T16:49:04.757468+0000","last_active":"2024-03-28T16:50:46.874154+0000","last_peered":"2024-03-28T16:50:46.874154+0000","last_clean":"2024-03-28T16:50:46.874154+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:50:46.874154+0000","last_undegraded":"2024-03-28T16:50:46.874154+0000","last_fullsized":"2024-03-28T16:50:46.874154+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.501412+0000","last_change":"2024-03-28T16:51:33.501547+0000","last_active":"2024-03-28T16:51:33.501412+0000","last_peered":"2024-03-28T16:51:33.501412+0000","last_clean":"2024-03-28T16:51:33.501412+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:51:33.501412+0000","last_undegraded":"2024-03-28T16:51:33.501412+0000","last_fullsized":"2024-03-28T16:51:33.501412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.501165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.501165+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.501165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:09.726882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:55.020918+0000","last_change":"2024-03-28T16:51:55.020972+0000","last_active":"2024-03-28T16:51:55.020918+0000","last_peered":"2024-03-28T16:51:55.020918+0000","last_clean":"2024-03-28T16:51:55.020918+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:51:55.020918+0000","last_undegraded":"2024-03-28T16:51:55.020918+0000","last_fullsized":"2024-03-28T16:51:55.020918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:55.020730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:55.020730+0000","last_clean_scrub_stamp":"2024-03-28T16:51:55.020730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:49:39.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.385902+0000","last_change":"2024-03-28T16:51:19.386018+0000","last_active":"2024-03-28T16:51:19.385902+0000","last_peered":"2024-03-28T16:51:19.385902+0000","last_clean":"2024-03-28T16:51:19.385902+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:51:19.385902+0000","last_undegraded":"2024-03-28T16:51:19.385902+0000","last_fullsized":"2024-03-28T16:51:19.385902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.385614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:54.970167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.375959+0000","last_change":"2024-03-28T16:51:06.375959+0000","last_active":"2024-03-28T16:51:06.375959+0000","last_peered":"2024-03-28T16:51:06.375959+0000","last_clean":"2024-03-28T16:51:06.375959+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:51:06.375959+0000","last_undegraded":"2024-03-28T16:51:06.375959+0000","last_fullsized":"2024-03-28T16:51:06.375959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.375714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:32.910452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.195780+0000","last_change":"2024-03-28T16:50:51.195780+0000","last_active":"2024-03-28T16:50:51.195780+0000","last_peered":"2024-03-28T16:50:51.195780+0000","last_clean":"2024-03-28T16:50:51.195780+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:50:51.195780+0000","last_undegraded":"2024-03-28T16:50:51.195780+0000","last_fullsized":"2024-03-28T16:50:51.195780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'19","last_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_deep_scrub":"31'19","last_deep_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.195522+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:16.598840+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.126759+0000","last_change":"2024-03-28T16:51:25.126813+0000","last_active":"2024-03-28T16:51:25.126759+0000","last_peered":"2024-03-28T16:51:25.126759+0000","last_clean":"2024-03-28T16:51:25.126759+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:51:25.126759+0000","last_undegraded":"2024-03-28T16:51:25.126759+0000","last_fullsized":"2024-03-28T16:51:25.126759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:25.126557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:25.126557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.126557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:42.133827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.302529+0000","last_change":"2024-03-28T16:51:13.302529+0000","last_active":"2024-03-28T16:51:13.302529+0000","last_peered":"2024-03-28T16:51:13.302529+0000","last_clean":"2024-03-28T16:51:13.302529+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:51:13.302529+0000","last_undegraded":"2024-03-28T16:51:13.302529+0000","last_fullsized":"2024-03-28T16:51:13.302529+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.302200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:35.072558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":68,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.328005+0000","last_change":"2024-03-28T16:51:12.328075+0000","last_active":"2024-03-28T16:51:12.328005+0000","last_peered":"2024-03-28T16:51:12.328005+0000","last_clean":"2024-03-28T16:51:12.328005+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:51:12.328005+0000","last_undegraded":"2024-03-28T16:51:12.328005+0000","last_fullsized":"2024-03-28T16:51:12.328005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.327822+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:47.912375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.230294+0000","last_change":"2024-03-28T16:51:30.230347+0000","last_active":"2024-03-28T16:51:30.230294+0000","last_peered":"2024-03-28T16:51:30.230294+0000","last_clean":"2024-03-28T16:51:30.230294+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:51:30.230294+0000","last_undegraded":"2024-03-28T16:51:30.230294+0000","last_fullsized":"2024-03-28T16:51:30.230294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.230107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.230107+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.230107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:06:05.111329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.444458+0000","last_change":"2024-03-28T16:51:36.444515+0000","last_active":"2024-03-28T16:51:36.444458+0000","last_peered":"2024-03-28T16:51:36.444458+0000","last_clean":"2024-03-28T16:51:36.444458+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:51:36.444458+0000","last_undegraded":"2024-03-28T16:51:36.444458+0000","last_fullsized":"2024-03-28T16:51:36.444458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.444268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.444268+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.444268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:33.466795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.355429+0000","last_change":"2024-03-28T16:51:03.355429+0000","last_active":"2024-03-28T16:51:03.355429+0000","last_peered":"2024-03-28T16:51:03.355429+0000","last_clean":"2024-03-28T16:51:03.355429+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:51:03.355429+0000","last_undegraded":"2024-03-28T16:51:03.355429+0000","last_fullsized":"2024-03-28T16:51:03.355429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.355165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:33.030302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'69","reported_seq":145,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.306941+0000","last_change":"2024-03-28T16:50:46.306941+0000","last_active":"2024-03-28T16:50:46.306941+0000","last_peered":"2024-03-28T16:50:46.306941+0000","last_clean":"2024-03-28T16:50:46.306941+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:46.306941+0000","last_undegraded":"2024-03-28T16:50:46.306941+0000","last_fullsized":"2024-03-28T16:50:46.306941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'69","last_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_deep_scrub":"31'69","last_deep_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.306710+0000","objects_scrubbed":8,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:28.604401+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.441291+0000","last_change":"2024-03-28T16:50:51.441291+0000","last_active":"2024-03-28T16:50:51.441291+0000","last_peered":"2024-03-28T16:50:51.441291+0000","last_clean":"2024-03-28T16:50:51.441291+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:50:51.441291+0000","last_undegraded":"2024-03-28T16:50:51.441291+0000","last_fullsized":"2024-03-28T16:50:51.441291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.441051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:13:01.864029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'11","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:44.568909+0000","last_change":"2024-03-28T16:51:44.568983+0000","last_active":"2024-03-28T16:51:44.568909+0000","last_peered":"2024-03-28T16:51:44.568909+0000","last_clean":"2024-03-28T16:51:44.568909+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:51:44.568909+0000","last_undegraded":"2024-03-28T16:51:44.568909+0000","last_fullsized":"2024-03-28T16:51:44.568909+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'11","last_scrub_stamp":"2024-03-28T16:51:44.568723+0000","last_deep_scrub":"31'11","last_deep_scrub_stamp":"2024-03-28T16:51:44.568723+0000","last_clean_scrub_stamp":"2024-03-28T16:51:44.568723+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:53.286332+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.006930+0000","last_change":"2024-03-28T16:51:39.006995+0000","last_active":"2024-03-28T16:51:39.006930+0000","last_peered":"2024-03-28T16:51:39.006930+0000","last_clean":"2024-03-28T16:51:39.006930+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:51:39.006930+0000","last_undegraded":"2024-03-28T16:51:39.006930+0000","last_fullsized":"2024-03-28T16:51:39.006930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:39.006741+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:39.006741+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.006741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:46.985028+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.491771+0000","last_change":"2024-03-28T16:51:34.491822+0000","last_active":"2024-03-28T16:51:34.491771+0000","last_peered":"2024-03-28T16:51:34.491771+0000","last_clean":"2024-03-28T16:51:34.491771+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:51:34.491771+0000","last_undegraded":"2024-03-28T16:51:34.491771+0000","last_fullsized":"2024-03-28T16:51:34.491771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:34.491575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:34.491575+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.491575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:59.766359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:56.935354+0000","last_change":"2024-03-28T16:51:56.935407+0000","last_active":"2024-03-28T16:51:56.935354+0000","last_peered":"2024-03-28T16:51:56.935354+0000","last_clean":"2024-03-28T16:51:56.935354+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:51:56.935354+0000","last_undegraded":"2024-03-28T16:51:56.935354+0000","last_fullsized":"2024-03-28T16:51:56.935354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:56.935166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:56.935166+0000","last_clean_scrub_stamp":"2024-03-28T16:51:56.935166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:00:15.278636+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.159039+0000","last_change":"2024-03-28T16:50:55.159039+0000","last_active":"2024-03-28T16:50:55.159039+0000","last_peered":"2024-03-28T16:50:55.159039+0000","last_clean":"2024-03-28T16:50:55.159039+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:50:55.159039+0000","last_undegraded":"2024-03-28T16:50:55.159039+0000","last_fullsized":"2024-03-28T16:50:55.159039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.158776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:53:10.542165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"31'21","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.252537+0000","last_change":"2024-03-28T16:50:52.252537+0000","last_active":"2024-03-28T16:50:52.252537+0000","last_peered":"2024-03-28T16:50:52.252537+0000","last_clean":"2024-03-28T16:50:52.252537+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:50:52.252537+0000","last_undegraded":"2024-03-28T16:50:52.252537+0000","last_fullsized":"2024-03-28T16:50:52.252537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'21","last_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_deep_scrub":"31'21","last_deep_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.252240+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:47.715342+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'249","reported_seq":304,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.104945+0000","last_change":"2024-03-28T16:50:46.104945+0000","last_active":"2024-03-28T16:50:46.104945+0000","last_peered":"2024-03-28T16:50:46.104945+0000","last_clean":"2024-03-28T16:50:46.104945+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:46.104945+0000","last_undegraded":"2024-03-28T16:50:46.104945+0000","last_fullsized":"2024-03-28T16:50:46.104945+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.104655+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:39.533970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874227+0000","last_change":"2024-03-28T16:49:04.763537+0000","last_active":"2024-03-28T16:50:46.874227+0000","last_peered":"2024-03-28T16:50:46.874227+0000","last_clean":"2024-03-28T16:50:46.874227+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:50:46.874227+0000","last_undegraded":"2024-03-28T16:50:46.874227+0000","last_fullsized":"2024-03-28T16:50:46.874227+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.222398+0000","last_change":"2024-03-28T16:51:04.222527+0000","last_active":"2024-03-28T16:51:04.222398+0000","last_peered":"2024-03-28T16:51:04.222398+0000","last_clean":"2024-03-28T16:51:04.222398+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:51:04.222398+0000","last_undegraded":"2024-03-28T16:51:04.222398+0000","last_fullsized":"2024-03-28T16:51:04.222398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.222151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:13.382250+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:50.030518+0000","last_change":"2024-03-28T16:51:50.030630+0000","last_active":"2024-03-28T16:51:50.030518+0000","last_peered":"2024-03-28T16:51:50.030518+0000","last_clean":"2024-03-28T16:51:50.030518+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:51:50.030518+0000","last_undegraded":"2024-03-28T16:51:50.030518+0000","last_fullsized":"2024-03-28T16:51:50.030518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:50.030315+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:50.030315+0000","last_clean_scrub_stamp":"2024-03-28T16:51:50.030315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:56:37.849059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.449142+0000","last_change":"2024-03-28T16:51:35.449195+0000","last_active":"2024-03-28T16:51:35.449142+0000","last_peered":"2024-03-28T16:51:35.449142+0000","last_clean":"2024-03-28T16:51:35.449142+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:51:35.449142+0000","last_undegraded":"2024-03-28T16:51:35.449142+0000","last_fullsized":"2024-03-28T16:51:35.449142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:35.448941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:35.448941+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.448941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:12.528913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.163965+0000","last_change":"2024-03-28T16:50:54.163965+0000","last_active":"2024-03-28T16:50:54.163965+0000","last_peered":"2024-03-28T16:50:54.163965+0000","last_clean":"2024-03-28T16:50:54.163965+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:50:54.163965+0000","last_undegraded":"2024-03-28T16:50:54.163965+0000","last_fullsized":"2024-03-28T16:50:54.163965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.163592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:27.253015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.038093+0000","last_change":"2024-03-28T16:51:16.038179+0000","last_active":"2024-03-28T16:51:16.038093+0000","last_peered":"2024-03-28T16:51:16.038093+0000","last_clean":"2024-03-28T16:51:16.038093+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:51:16.038093+0000","last_undegraded":"2024-03-28T16:51:16.038093+0000","last_fullsized":"2024-03-28T16:51:16.038093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.037881+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:44.064566+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'249","reported_seq":337,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.233041+0000","last_change":"2024-03-28T16:50:49.233041+0000","last_active":"2024-03-28T16:50:49.233041+0000","last_peered":"2024-03-28T16:50:49.233041+0000","last_clean":"2024-03-28T16:50:49.233041+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:49.233041+0000","last_undegraded":"2024-03-28T16:50:49.233041+0000","last_fullsized":"2024-03-28T16:50:49.233041+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.232798+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:57.227518+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.267309+0000","last_change":"2024-03-28T16:51:09.267309+0000","last_active":"2024-03-28T16:51:09.267309+0000","last_peered":"2024-03-28T16:51:09.267309+0000","last_clean":"2024-03-28T16:51:09.267309+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:51:09.267309+0000","last_undegraded":"2024-03-28T16:51:09.267309+0000","last_fullsized":"2024-03-28T16:51:09.267309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.267056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:58.701007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.363436+0000","last_change":"2024-03-28T16:51:16.363501+0000","last_active":"2024-03-28T16:51:16.363436+0000","last_peered":"2024-03-28T16:51:16.363436+0000","last_clean":"2024-03-28T16:51:16.363436+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:51:16.363436+0000","last_undegraded":"2024-03-28T16:51:16.363436+0000","last_fullsized":"2024-03-28T16:51:16.363436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.363254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:07:05.797204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.287919+0000","last_change":"2024-03-28T16:51:20.287986+0000","last_active":"2024-03-28T16:51:20.287919+0000","last_peered":"2024-03-28T16:51:20.287919+0000","last_clean":"2024-03-28T16:51:20.287919+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:51:20.287919+0000","last_undegraded":"2024-03-28T16:51:20.287919+0000","last_fullsized":"2024-03-28T16:51:20.287919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.287729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:53.368108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:58.945626+0000","last_change":"2024-03-28T16:51:58.945678+0000","last_active":"2024-03-28T16:51:58.945626+0000","last_peered":"2024-03-28T16:51:58.945626+0000","last_clean":"2024-03-28T16:51:58.945626+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:51:58.945626+0000","last_undegraded":"2024-03-28T16:51:58.945626+0000","last_fullsized":"2024-03-28T16:51:58.945626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:58.945435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:58.945435+0000","last_clean_scrub_stamp":"2024-03-28T16:51:58.945435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:30:27.534851+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.166685+0000","last_change":"2024-03-28T16:50:56.166685+0000","last_active":"2024-03-28T16:50:56.166685+0000","last_peered":"2024-03-28T16:50:56.166685+0000","last_clean":"2024-03-28T16:50:56.166685+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:50:56.166685+0000","last_undegraded":"2024-03-28T16:50:56.166685+0000","last_fullsized":"2024-03-28T16:50:56.166685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.166428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:06.567829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'90","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.384285+0000","last_change":"2024-03-28T16:50:58.384285+0000","last_active":"2024-03-28T16:50:58.384285+0000","last_peered":"2024-03-28T16:50:58.384285+0000","last_clean":"2024-03-28T16:50:58.384285+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:58.384285+0000","last_undegraded":"2024-03-28T16:50:58.384285+0000","last_fullsized":"2024-03-28T16:50:58.384285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'90","last_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_deep_scrub":"31'90","last_deep_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.383995+0000","objects_scrubbed":8,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:54.487031+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.285325+0000","last_change":"2024-03-28T16:50:47.285325+0000","last_active":"2024-03-28T16:50:47.285325+0000","last_peered":"2024-03-28T16:50:47.285325+0000","last_clean":"2024-03-28T16:50:47.285325+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:50:47.285325+0000","last_undegraded":"2024-03-28T16:50:47.285325+0000","last_fullsized":"2024-03-28T16:50:47.285325+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.285202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:46:10.170506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.228376+0000","last_change":"2024-03-28T16:51:36.228433+0000","last_active":"2024-03-28T16:51:36.228376+0000","last_peered":"2024-03-28T16:51:36.228376+0000","last_clean":"2024-03-28T16:51:36.228376+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:51:36.228376+0000","last_undegraded":"2024-03-28T16:51:36.228376+0000","last_fullsized":"2024-03-28T16:51:36.228376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.228185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.228185+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.228185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:41.408321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.344614+0000","last_change":"2024-03-28T16:51:00.344614+0000","last_active":"2024-03-28T16:51:00.344614+0000","last_peered":"2024-03-28T16:51:00.344614+0000","last_clean":"2024-03-28T16:51:00.344614+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:51:00.344614+0000","last_undegraded":"2024-03-28T16:51:00.344614+0000","last_fullsized":"2024-03-28T16:51:00.344614+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.344363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:57.233454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.364821+0000","last_change":"2024-03-28T16:51:10.364821+0000","last_active":"2024-03-28T16:51:10.364821+0000","last_peered":"2024-03-28T16:51:10.364821+0000","last_clean":"2024-03-28T16:51:10.364821+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:51:10.364821+0000","last_undegraded":"2024-03-28T16:51:10.364821+0000","last_fullsized":"2024-03-28T16:51:10.364821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.364556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:15.618413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:46.043748+0000","last_change":"2024-03-28T16:51:46.043800+0000","last_active":"2024-03-28T16:51:46.043748+0000","last_peered":"2024-03-28T16:51:46.043748+0000","last_clean":"2024-03-28T16:51:46.043748+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:51:46.043748+0000","last_undegraded":"2024-03-28T16:51:46.043748+0000","last_fullsized":"2024-03-28T16:51:46.043748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:46.043568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:46.043568+0000","last_clean_scrub_stamp":"2024-03-28T16:51:46.043568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:33:51.512636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:45.081449+0000","last_change":"2024-03-28T16:51:45.081503+0000","last_active":"2024-03-28T16:51:45.081449+0000","last_peered":"2024-03-28T16:51:45.081449+0000","last_clean":"2024-03-28T16:51:45.081449+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:51:45.081449+0000","last_undegraded":"2024-03-28T16:51:45.081449+0000","last_fullsized":"2024-03-28T16:51:45.081449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:45.081273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:45.081273+0000","last_clean_scrub_stamp":"2024-03-28T16:51:45.081273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:49.252246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.084532+0000","last_change":"2024-03-28T16:51:15.084532+0000","last_active":"2024-03-28T16:51:15.084532+0000","last_peered":"2024-03-28T16:51:15.084532+0000","last_clean":"2024-03-28T16:51:15.084532+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:51:15.084532+0000","last_undegraded":"2024-03-28T16:51:15.084532+0000","last_fullsized":"2024-03-28T16:51:15.084532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.084235+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:01:39.553948+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.246355+0000","last_change":"2024-03-28T16:51:09.246355+0000","last_active":"2024-03-28T16:51:09.246355+0000","last_peered":"2024-03-28T16:51:09.246355+0000","last_clean":"2024-03-28T16:51:09.246355+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:51:09.246355+0000","last_undegraded":"2024-03-28T16:51:09.246355+0000","last_fullsized":"2024-03-28T16:51:09.246355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.246098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:55:30.745667+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.442625+0000","last_change":"2024-03-28T16:51:17.442696+0000","last_active":"2024-03-28T16:51:17.442625+0000","last_peered":"2024-03-28T16:51:17.442625+0000","last_clean":"2024-03-28T16:51:17.442625+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:51:17.442625+0000","last_undegraded":"2024-03-28T16:51:17.442625+0000","last_fullsized":"2024-03-28T16:51:17.442625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.442424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:34.861019+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:44.063414+0000","last_change":"2024-03-28T16:51:44.063470+0000","last_active":"2024-03-28T16:51:44.063414+0000","last_peered":"2024-03-28T16:51:44.063414+0000","last_clean":"2024-03-28T16:51:44.063414+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:51:44.063414+0000","last_undegraded":"2024-03-28T16:51:44.063414+0000","last_fullsized":"2024-03-28T16:51:44.063414+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:44.063240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:44.063240+0000","last_clean_scrub_stamp":"2024-03-28T16:51:44.063240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:57:07.474188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.199918+0000","last_change":"2024-03-28T16:51:31.200010+0000","last_active":"2024-03-28T16:51:31.199918+0000","last_peered":"2024-03-28T16:51:31.199918+0000","last_clean":"2024-03-28T16:51:31.199918+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:51:31.199918+0000","last_undegraded":"2024-03-28T16:51:31.199918+0000","last_fullsized":"2024-03-28T16:51:31.199918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:31.199694+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:31.199694+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.199694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:43.004903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.178814+0000","last_change":"2024-03-28T16:50:52.178814+0000","last_active":"2024-03-28T16:50:52.178814+0000","last_peered":"2024-03-28T16:50:52.178814+0000","last_clean":"2024-03-28T16:50:52.178814+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:50:52.178814+0000","last_undegraded":"2024-03-28T16:50:52.178814+0000","last_fullsized":"2024-03-28T16:50:52.178814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.178561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:00.951220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.353835+0000","last_change":"2024-03-28T16:51:00.353835+0000","last_active":"2024-03-28T16:51:00.353835+0000","last_peered":"2024-03-28T16:51:00.353835+0000","last_clean":"2024-03-28T16:51:00.353835+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:51:00.353835+0000","last_undegraded":"2024-03-28T16:51:00.353835+0000","last_fullsized":"2024-03-28T16:51:00.353835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.353557+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:18.400713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.443394+0000","last_change":"2024-03-28T16:50:49.443394+0000","last_active":"2024-03-28T16:50:49.443394+0000","last_peered":"2024-03-28T16:50:49.443394+0000","last_clean":"2024-03-28T16:50:49.443394+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:50:49.443394+0000","last_undegraded":"2024-03-28T16:50:49.443394+0000","last_fullsized":"2024-03-28T16:50:49.443394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.443035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:33:21.896718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:48.480062+0000","last_change":"2024-03-28T16:51:48.480176+0000","last_active":"2024-03-28T16:51:48.480062+0000","last_peered":"2024-03-28T16:51:48.480062+0000","last_clean":"2024-03-28T16:51:48.480062+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:51:48.480062+0000","last_undegraded":"2024-03-28T16:51:48.480062+0000","last_fullsized":"2024-03-28T16:51:48.480062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:48.479806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:48.479806+0000","last_clean_scrub_stamp":"2024-03-28T16:51:48.479806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:10:13.930593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.486418+0000","last_change":"2024-03-28T16:51:19.486556+0000","last_active":"2024-03-28T16:51:19.486418+0000","last_peered":"2024-03-28T16:51:19.486418+0000","last_clean":"2024-03-28T16:51:19.486418+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:51:19.486418+0000","last_undegraded":"2024-03-28T16:51:19.486418+0000","last_fullsized":"2024-03-28T16:51:19.486418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.486207+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:42.867614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.236966+0000","last_change":"2024-03-28T16:51:33.237031+0000","last_active":"2024-03-28T16:51:33.236966+0000","last_peered":"2024-03-28T16:51:33.236966+0000","last_clean":"2024-03-28T16:51:33.236966+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:51:33.236966+0000","last_undegraded":"2024-03-28T16:51:33.236966+0000","last_fullsized":"2024-03-28T16:51:33.236966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:33.236770+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:33.236770+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.236770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:05:19.456972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":209,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.379662+0000","last_change":"2024-03-28T16:51:06.379662+0000","last_active":"2024-03-28T16:51:06.379662+0000","last_peered":"2024-03-28T16:51:06.379662+0000","last_clean":"2024-03-28T16:51:06.379662+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:51:06.379662+0000","last_undegraded":"2024-03-28T16:51:06.379662+0000","last_fullsized":"2024-03-28T16:51:06.379662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.379283+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:41.522575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":256,"num_read_kb":255,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'42","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.233580+0000","last_change":"2024-03-28T16:50:55.233580+0000","last_active":"2024-03-28T16:50:55.233580+0000","last_peered":"2024-03-28T16:50:55.233580+0000","last_clean":"2024-03-28T16:50:55.233580+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:55.233580+0000","last_undegraded":"2024-03-28T16:50:55.233580+0000","last_fullsized":"2024-03-28T16:50:55.233580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'42","last_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_deep_scrub":"31'42","last_deep_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.233264+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:43.214573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":56,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.129943+0000","last_change":"2024-03-28T16:50:47.129943+0000","last_active":"2024-03-28T16:50:47.129943+0000","last_peered":"2024-03-28T16:50:47.129943+0000","last_clean":"2024-03-28T16:50:47.129943+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:50:47.129943+0000","last_undegraded":"2024-03-28T16:50:47.129943+0000","last_fullsized":"2024-03-28T16:50:47.129943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.129698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:49.507343+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:04.024275+0000","last_change":"2024-03-28T16:52:04.024329+0000","last_active":"2024-03-28T16:52:04.024275+0000","last_peered":"2024-03-28T16:52:04.024275+0000","last_clean":"2024-03-28T16:52:04.024275+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:52:04.024275+0000","last_undegraded":"2024-03-28T16:52:04.024275+0000","last_fullsized":"2024-03-28T16:52:04.024275+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:04.024095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:04.024095+0000","last_clean_scrub_stamp":"2024-03-28T16:52:04.024095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:41:48.078302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.093412+0000","last_change":"2024-03-28T16:51:26.093486+0000","last_active":"2024-03-28T16:51:26.093412+0000","last_peered":"2024-03-28T16:51:26.093412+0000","last_clean":"2024-03-28T16:51:26.093412+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:51:26.093412+0000","last_undegraded":"2024-03-28T16:51:26.093412+0000","last_fullsized":"2024-03-28T16:51:26.093412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:26.093216+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:26.093216+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.093216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:08.375290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.237688+0000","last_change":"2024-03-28T16:51:26.237758+0000","last_active":"2024-03-28T16:51:26.237688+0000","last_peered":"2024-03-28T16:51:26.237688+0000","last_clean":"2024-03-28T16:51:26.237688+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:51:26.237688+0000","last_undegraded":"2024-03-28T16:51:26.237688+0000","last_fullsized":"2024-03-28T16:51:26.237688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:26.237512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:26.237512+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.237512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:57.241670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.166465+0000","last_change":"2024-03-28T16:50:58.166465+0000","last_active":"2024-03-28T16:50:58.166465+0000","last_peered":"2024-03-28T16:50:58.166465+0000","last_clean":"2024-03-28T16:50:58.166465+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:50:58.166465+0000","last_undegraded":"2024-03-28T16:50:58.166465+0000","last_fullsized":"2024-03-28T16:50:58.166465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.166221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:21.547629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.370703+0000","last_change":"2024-03-28T16:51:02.370703+0000","last_active":"2024-03-28T16:51:02.370703+0000","last_peered":"2024-03-28T16:51:02.370703+0000","last_clean":"2024-03-28T16:51:02.370703+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:51:02.370703+0000","last_undegraded":"2024-03-28T16:51:02.370703+0000","last_fullsized":"2024-03-28T16:51:02.370703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.370450+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:15.038783+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.448158+0000","last_change":"2024-03-28T16:50:50.448158+0000","last_active":"2024-03-28T16:50:50.448158+0000","last_peered":"2024-03-28T16:50:50.448158+0000","last_clean":"2024-03-28T16:50:50.448158+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:50:50.448158+0000","last_undegraded":"2024-03-28T16:50:50.448158+0000","last_fullsized":"2024-03-28T16:50:50.448158+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.447911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:45.521899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.151593+0000","last_change":"2024-03-28T16:51:31.151647+0000","last_active":"2024-03-28T16:51:31.151593+0000","last_peered":"2024-03-28T16:51:31.151593+0000","last_clean":"2024-03-28T16:51:31.151593+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:51:31.151593+0000","last_undegraded":"2024-03-28T16:51:31.151593+0000","last_fullsized":"2024-03-28T16:51:31.151593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:31.151371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:31.151371+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.151371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:46.085551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:50.460481+0000","last_change":"2024-03-28T16:51:50.460481+0000","last_active":"2024-03-28T16:51:50.460481+0000","last_peered":"2024-03-28T16:51:50.460481+0000","last_clean":"2024-03-28T16:51:50.460481+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:51:50.460481+0000","last_undegraded":"2024-03-28T16:51:50.460481+0000","last_fullsized":"2024-03-28T16:51:50.460481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:50.460232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:50.460232+0000","last_clean_scrub_stamp":"2024-03-28T16:51:50.460232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:34.292013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.369809+0000","last_change":"2024-03-28T16:51:07.369837+0000","last_active":"2024-03-28T16:51:07.369809+0000","last_peered":"2024-03-28T16:51:07.369809+0000","last_clean":"2024-03-28T16:51:07.369809+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:51:07.369809+0000","last_undegraded":"2024-03-28T16:51:07.369809+0000","last_fullsized":"2024-03-28T16:51:07.369809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.369707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:56.803658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'60","reported_seq":133,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.252559+0000","last_change":"2024-03-28T16:50:56.252559+0000","last_active":"2024-03-28T16:50:56.252559+0000","last_peered":"2024-03-28T16:50:56.252559+0000","last_clean":"2024-03-28T16:50:56.252559+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:56.252559+0000","last_undegraded":"2024-03-28T16:50:56.252559+0000","last_fullsized":"2024-03-28T16:50:56.252559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'60","last_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_deep_scrub":"31'60","last_deep_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.252305+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:41.009336+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.444559+0000","last_change":"2024-03-28T16:50:48.444559+0000","last_active":"2024-03-28T16:50:48.444559+0000","last_peered":"2024-03-28T16:50:48.444559+0000","last_clean":"2024-03-28T16:50:48.444559+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:50:48.444559+0000","last_undegraded":"2024-03-28T16:50:48.444559+0000","last_fullsized":"2024-03-28T16:50:48.444559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.444297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:24:42.861356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'12","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:49.443162+0000","last_change":"2024-03-28T16:51:49.443231+0000","last_active":"2024-03-28T16:51:49.443162+0000","last_peered":"2024-03-28T16:51:49.443162+0000","last_clean":"2024-03-28T16:51:49.443162+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:51:49.443162+0000","last_undegraded":"2024-03-28T16:51:49.443162+0000","last_fullsized":"2024-03-28T16:51:49.443162+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:49.442980+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:49.442980+0000","last_clean_scrub_stamp":"2024-03-28T16:51:49.442980+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:33.519156+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.399302+0000","last_change":"2024-03-28T16:51:15.399354+0000","last_active":"2024-03-28T16:51:15.399302+0000","last_peered":"2024-03-28T16:51:15.399302+0000","last_clean":"2024-03-28T16:51:15.399302+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:51:15.399302+0000","last_undegraded":"2024-03-28T16:51:15.399302+0000","last_fullsized":"2024-03-28T16:51:15.399302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.399113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:12.560237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.016129+0000","last_change":"2024-03-28T16:51:37.016182+0000","last_active":"2024-03-28T16:51:37.016129+0000","last_peered":"2024-03-28T16:51:37.016129+0000","last_clean":"2024-03-28T16:51:37.016129+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:51:37.016129+0000","last_undegraded":"2024-03-28T16:51:37.016129+0000","last_fullsized":"2024-03-28T16:51:37.016129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.015919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.015919+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.015919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:14.095891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.409622+0000","last_change":"2024-03-28T16:51:09.409674+0000","last_active":"2024-03-28T16:51:09.409622+0000","last_peered":"2024-03-28T16:51:09.409622+0000","last_clean":"2024-03-28T16:51:09.409622+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:51:09.409622+0000","last_undegraded":"2024-03-28T16:51:09.409622+0000","last_fullsized":"2024-03-28T16:51:09.409622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.409427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:27.456583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'43","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.182861+0000","last_change":"2024-03-28T16:50:57.182861+0000","last_active":"2024-03-28T16:50:57.182861+0000","last_peered":"2024-03-28T16:50:57.182861+0000","last_clean":"2024-03-28T16:50:57.182861+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:57.182861+0000","last_undegraded":"2024-03-28T16:50:57.182861+0000","last_fullsized":"2024-03-28T16:50:57.182861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.182585+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.262964+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.238776+0000","last_change":"2024-03-28T16:50:53.238776+0000","last_active":"2024-03-28T16:50:53.238776+0000","last_peered":"2024-03-28T16:50:53.238776+0000","last_clean":"2024-03-28T16:50:53.238776+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:50:53.238776+0000","last_undegraded":"2024-03-28T16:50:53.238776+0000","last_fullsized":"2024-03-28T16:50:53.238776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.238528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:50.127619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:05.941415+0000","last_change":"2024-03-28T16:52:05.941484+0000","last_active":"2024-03-28T16:52:05.941415+0000","last_peered":"2024-03-28T16:52:05.941415+0000","last_clean":"2024-03-28T16:52:05.941415+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:52:05.941415+0000","last_undegraded":"2024-03-28T16:52:05.941415+0000","last_fullsized":"2024-03-28T16:52:05.941415+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:52:05.941209+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:52:05.941209+0000","last_clean_scrub_stamp":"2024-03-28T16:52:05.941209+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:38:27.650509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.175511+0000","last_change":"2024-03-28T16:51:02.175511+0000","last_active":"2024-03-28T16:51:02.175511+0000","last_peered":"2024-03-28T16:51:02.175511+0000","last_clean":"2024-03-28T16:51:02.175511+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:51:02.175511+0000","last_undegraded":"2024-03-28T16:51:02.175511+0000","last_fullsized":"2024-03-28T16:51:02.175511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.175255+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:28.260113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.155413+0000","last_change":"2024-03-28T16:51:32.155474+0000","last_active":"2024-03-28T16:51:32.155413+0000","last_peered":"2024-03-28T16:51:32.155413+0000","last_clean":"2024-03-28T16:51:32.155413+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:51:32.155413+0000","last_undegraded":"2024-03-28T16:51:32.155413+0000","last_fullsized":"2024-03-28T16:51:32.155413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.155231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.155231+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.155231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:35.685453+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.228518+0000","last_change":"2024-03-28T16:51:38.228577+0000","last_active":"2024-03-28T16:51:38.228518+0000","last_peered":"2024-03-28T16:51:38.228518+0000","last_clean":"2024-03-28T16:51:38.228518+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:51:38.228518+0000","last_undegraded":"2024-03-28T16:51:38.228518+0000","last_fullsized":"2024-03-28T16:51:38.228518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.228325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.228325+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.228325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:56.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.342055+0000","last_change":"2024-03-28T16:50:58.342055+0000","last_active":"2024-03-28T16:50:58.342055+0000","last_peered":"2024-03-28T16:50:58.342055+0000","last_clean":"2024-03-28T16:50:58.342055+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:58.342055+0000","last_undegraded":"2024-03-28T16:50:58.342055+0000","last_fullsized":"2024-03-28T16:50:58.342055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.341815+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:35.862752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.375363+0000","last_change":"2024-03-28T16:51:05.375363+0000","last_active":"2024-03-28T16:51:05.375363+0000","last_peered":"2024-03-28T16:51:05.375363+0000","last_clean":"2024-03-28T16:51:05.375363+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:51:05.375363+0000","last_undegraded":"2024-03-28T16:51:05.375363+0000","last_fullsized":"2024-03-28T16:51:05.375363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.375113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:40.169672+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.118142+0000","last_change":"2024-03-28T16:51:29.118208+0000","last_active":"2024-03-28T16:51:29.118142+0000","last_peered":"2024-03-28T16:51:29.118142+0000","last_clean":"2024-03-28T16:51:29.118142+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:51:29.118142+0000","last_undegraded":"2024-03-28T16:51:29.118142+0000","last_fullsized":"2024-03-28T16:51:29.118142+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:29.117944+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:29.117944+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.117944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:12.949724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.562342+0000","last_change":"2024-03-28T16:51:29.562487+0000","last_active":"2024-03-28T16:51:29.562342+0000","last_peered":"2024-03-28T16:51:29.562342+0000","last_clean":"2024-03-28T16:51:29.562342+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:51:29.562342+0000","last_undegraded":"2024-03-28T16:51:29.562342+0000","last_fullsized":"2024-03-28T16:51:29.562342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:29.562080+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:29.562080+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.562080+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:48.297089+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:53.057583+0000","last_change":"2024-03-28T16:51:53.057636+0000","last_active":"2024-03-28T16:51:53.057583+0000","last_peered":"2024-03-28T16:51:53.057583+0000","last_clean":"2024-03-28T16:51:53.057583+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:51:53.057583+0000","last_undegraded":"2024-03-28T16:51:53.057583+0000","last_fullsized":"2024-03-28T16:51:53.057583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:53.057385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:53.057385+0000","last_clean_scrub_stamp":"2024-03-28T16:51:53.057385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:22:56.914997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.303980+0000","last_change":"2024-03-28T16:51:04.303980+0000","last_active":"2024-03-28T16:51:04.303980+0000","last_peered":"2024-03-28T16:51:04.303980+0000","last_clean":"2024-03-28T16:51:04.303980+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:51:04.303980+0000","last_undegraded":"2024-03-28T16:51:04.303980+0000","last_fullsized":"2024-03-28T16:51:04.303980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.303737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:22.080932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.229100+0000","last_change":"2024-03-28T16:51:01.229100+0000","last_active":"2024-03-28T16:51:01.229100+0000","last_peered":"2024-03-28T16:51:01.229100+0000","last_clean":"2024-03-28T16:51:01.229100+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:51:01.229100+0000","last_undegraded":"2024-03-28T16:51:01.229100+0000","last_fullsized":"2024-03-28T16:51:01.229100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.228821+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:13.034030+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'249","reported_seq":310,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.440393+0000","last_change":"2024-03-28T16:50:52.440393+0000","last_active":"2024-03-28T16:50:52.440393+0000","last_peered":"2024-03-28T16:50:52.440393+0000","last_clean":"2024-03-28T16:50:52.440393+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:52.440393+0000","last_undegraded":"2024-03-28T16:50:52.440393+0000","last_fullsized":"2024-03-28T16:50:52.440393+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.440130+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:54:49.215852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.089330+0000","last_change":"2024-03-28T16:51:33.089387+0000","last_active":"2024-03-28T16:51:33.089330+0000","last_peered":"2024-03-28T16:51:33.089330+0000","last_clean":"2024-03-28T16:51:33.089330+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:51:33.089330+0000","last_undegraded":"2024-03-28T16:51:33.089330+0000","last_fullsized":"2024-03-28T16:51:33.089330+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.089137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.089137+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.089137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:01:20.214432+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'6","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.352050+0000","last_change":"2024-03-28T16:51:20.352050+0000","last_active":"2024-03-28T16:51:20.352050+0000","last_peered":"2024-03-28T16:51:20.352050+0000","last_clean":"2024-03-28T16:51:20.352050+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:51:20.352050+0000","last_undegraded":"2024-03-28T16:51:20.352050+0000","last_fullsized":"2024-03-28T16:51:20.352050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.351803+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:25.271104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.050474+0000","last_change":"2024-03-28T16:51:36.050529+0000","last_active":"2024-03-28T16:51:36.050474+0000","last_peered":"2024-03-28T16:51:36.050474+0000","last_clean":"2024-03-28T16:51:36.050474+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:51:36.050474+0000","last_undegraded":"2024-03-28T16:51:36.050474+0000","last_fullsized":"2024-03-28T16:51:36.050474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.050281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.050281+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.050281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:01.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.331654+0000","last_change":"2024-03-28T16:51:02.331654+0000","last_active":"2024-03-28T16:51:02.331654+0000","last_peered":"2024-03-28T16:51:02.331654+0000","last_clean":"2024-03-28T16:51:02.331654+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:51:02.331654+0000","last_undegraded":"2024-03-28T16:51:02.331654+0000","last_fullsized":"2024-03-28T16:51:02.331654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.331416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:20.378572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'43","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.202390+0000","last_change":"2024-03-28T16:50:50.202390+0000","last_active":"2024-03-28T16:50:50.202390+0000","last_peered":"2024-03-28T16:50:50.202390+0000","last_clean":"2024-03-28T16:50:50.202390+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:50.202390+0000","last_undegraded":"2024-03-28T16:50:50.202390+0000","last_fullsized":"2024-03-28T16:50:50.202390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.202141+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.086676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.276938+0000","last_change":"2024-03-28T16:50:48.276938+0000","last_active":"2024-03-28T16:50:48.276938+0000","last_peered":"2024-03-28T16:50:48.276938+0000","last_clean":"2024-03-28T16:50:48.276938+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:50:48.276938+0000","last_undegraded":"2024-03-28T16:50:48.276938+0000","last_fullsized":"2024-03-28T16:50:48.276938+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.276698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:29.219774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:43.038235+0000","last_change":"2024-03-28T16:51:43.038288+0000","last_active":"2024-03-28T16:51:43.038235+0000","last_peered":"2024-03-28T16:51:43.038235+0000","last_clean":"2024-03-28T16:51:43.038235+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:51:43.038235+0000","last_undegraded":"2024-03-28T16:51:43.038235+0000","last_fullsized":"2024-03-28T16:51:43.038235+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:43.038045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:43.038045+0000","last_clean_scrub_stamp":"2024-03-28T16:51:43.038045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:50.550015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'4","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.310265+0000","last_change":"2024-03-28T16:51:24.310265+0000","last_active":"2024-03-28T16:51:24.310265+0000","last_peered":"2024-03-28T16:51:24.310265+0000","last_clean":"2024-03-28T16:51:24.310265+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:51:24.310265+0000","last_undegraded":"2024-03-28T16:51:24.310265+0000","last_fullsized":"2024-03-28T16:51:24.310265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.310025+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:21.471148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:43.004934+0000","last_change":"2024-03-28T16:51:43.004997+0000","last_active":"2024-03-28T16:51:43.004934+0000","last_peered":"2024-03-28T16:51:43.004934+0000","last_clean":"2024-03-28T16:51:43.004934+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:51:43.004934+0000","last_undegraded":"2024-03-28T16:51:43.004934+0000","last_fullsized":"2024-03-28T16:51:43.004934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:43.004709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:43.004709+0000","last_clean_scrub_stamp":"2024-03-28T16:51:43.004709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:48.991119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.396495+0000","last_change":"2024-03-28T16:51:03.396495+0000","last_active":"2024-03-28T16:51:03.396495+0000","last_peered":"2024-03-28T16:51:03.396495+0000","last_clean":"2024-03-28T16:51:03.396495+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:51:03.396495+0000","last_undegraded":"2024-03-28T16:51:03.396495+0000","last_fullsized":"2024-03-28T16:51:03.396495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.396253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:26.295446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.307759+0000","last_change":"2024-03-28T16:50:59.307759+0000","last_active":"2024-03-28T16:50:59.307759+0000","last_peered":"2024-03-28T16:50:59.307759+0000","last_clean":"2024-03-28T16:50:59.307759+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:59.307759+0000","last_undegraded":"2024-03-28T16:50:59.307759+0000","last_fullsized":"2024-03-28T16:50:59.307759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.307494+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:28.146893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.090891+0000","last_change":"2024-03-28T16:51:13.090891+0000","last_active":"2024-03-28T16:51:13.090891+0000","last_peered":"2024-03-28T16:51:13.090891+0000","last_clean":"2024-03-28T16:51:13.090891+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:51:13.090891+0000","last_undegraded":"2024-03-28T16:51:13.090891+0000","last_fullsized":"2024-03-28T16:51:13.090891+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.090537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:44:11.869557+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874510+0000","last_change":"2024-03-28T16:49:04.761607+0000","last_active":"2024-03-28T16:50:46.874510+0000","last_peered":"2024-03-28T16:50:46.874510+0000","last_clean":"2024-03-28T16:50:46.874510+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:50:46.874510+0000","last_undegraded":"2024-03-28T16:50:46.874510+0000","last_fullsized":"2024-03-28T16:50:46.874510+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.064823+0000","last_change":"2024-03-28T16:51:35.064895+0000","last_active":"2024-03-28T16:51:35.064823+0000","last_peered":"2024-03-28T16:51:35.064823+0000","last_clean":"2024-03-28T16:51:35.064823+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:51:35.064823+0000","last_undegraded":"2024-03-28T16:51:35.064823+0000","last_fullsized":"2024-03-28T16:51:35.064823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:35.064632+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:35.064632+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.064632+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:41.776672+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.598680+0000","last_change":"2024-03-28T16:51:24.598733+0000","last_active":"2024-03-28T16:51:24.598680+0000","last_peered":"2024-03-28T16:51:24.598680+0000","last_clean":"2024-03-28T16:51:24.598680+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:51:24.598680+0000","last_undegraded":"2024-03-28T16:51:24.598680+0000","last_fullsized":"2024-03-28T16:51:24.598680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:24.598488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:24.598488+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.598488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:05:10.818690+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.256599+0000","last_change":"2024-03-28T16:50:59.256599+0000","last_active":"2024-03-28T16:50:59.256599+0000","last_peered":"2024-03-28T16:50:59.256599+0000","last_clean":"2024-03-28T16:50:59.256599+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:50:59.256599+0000","last_undegraded":"2024-03-28T16:50:59.256599+0000","last_fullsized":"2024-03-28T16:50:59.256599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.256359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:34.399651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'28","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.213439+0000","last_change":"2024-03-28T16:50:59.213439+0000","last_active":"2024-03-28T16:50:59.213439+0000","last_peered":"2024-03-28T16:50:59.213439+0000","last_clean":"2024-03-28T16:50:59.213439+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:59.213439+0000","last_undegraded":"2024-03-28T16:50:59.213439+0000","last_fullsized":"2024-03-28T16:50:59.213439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'28","last_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_deep_scrub":"31'28","last_deep_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.213187+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:38.889215+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.080305+0000","last_change":"2024-03-28T16:51:17.080305+0000","last_active":"2024-03-28T16:51:17.080305+0000","last_peered":"2024-03-28T16:51:17.080305+0000","last_clean":"2024-03-28T16:51:17.080305+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:51:17.080305+0000","last_undegraded":"2024-03-28T16:51:17.080305+0000","last_fullsized":"2024-03-28T16:51:17.080305+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.080049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:12.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.874360+0000","last_change":"2024-03-28T16:49:04.760838+0000","last_active":"2024-03-28T16:50:46.874360+0000","last_peered":"2024-03-28T16:50:46.874360+0000","last_clean":"2024-03-28T16:50:46.874360+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:50:46.874360+0000","last_undegraded":"2024-03-28T16:50:46.874360+0000","last_fullsized":"2024-03-28T16:50:46.874360+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:49:03.693154+0000","last_clean_scrub_stamp":"2024-03-28T16:49:03.693154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'12","reported_seq":66,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.347902+0000","last_change":"2024-03-28T16:51:13.347902+0000","last_active":"2024-03-28T16:51:13.347902+0000","last_peered":"2024-03-28T16:51:13.347902+0000","last_clean":"2024-03-28T16:51:13.347902+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:51:13.347902+0000","last_undegraded":"2024-03-28T16:51:13.347902+0000","last_fullsized":"2024-03-28T16:51:13.347902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.347663+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:39.911911+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.128855+0000","last_change":"2024-03-28T16:51:34.128912+0000","last_active":"2024-03-28T16:51:34.128855+0000","last_peered":"2024-03-28T16:51:34.128855+0000","last_clean":"2024-03-28T16:51:34.128855+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:51:34.128855+0000","last_undegraded":"2024-03-28T16:51:34.128855+0000","last_fullsized":"2024-03-28T16:51:34.128855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:34.128651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:34.128651+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.128651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:18.350401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.215895+0000","last_change":"2024-03-28T16:50:58.215895+0000","last_active":"2024-03-28T16:50:58.215895+0000","last_peered":"2024-03-28T16:50:58.215895+0000","last_clean":"2024-03-28T16:50:58.215895+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:50:58.215895+0000","last_undegraded":"2024-03-28T16:50:58.215895+0000","last_fullsized":"2024-03-28T16:50:58.215895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.215651+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:42.972682+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'35","reported_seq":90,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.272061+0000","last_change":"2024-03-28T16:51:05.272061+0000","last_active":"2024-03-28T16:51:05.272061+0000","last_peered":"2024-03-28T16:51:05.272061+0000","last_clean":"2024-03-28T16:51:05.272061+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:51:05.272061+0000","last_undegraded":"2024-03-28T16:51:05.272061+0000","last_fullsized":"2024-03-28T16:51:05.272061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'35","last_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_deep_scrub":"31'35","last_deep_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.271785+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:01.679261+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1150,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.209342+0000","last_change":"2024-03-28T16:50:50.209342+0000","last_active":"2024-03-28T16:50:50.209342+0000","last_peered":"2024-03-28T16:50:50.209342+0000","last_clean":"2024-03-28T16:50:50.209342+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:50:50.209342+0000","last_undegraded":"2024-03-28T16:50:50.209342+0000","last_fullsized":"2024-03-28T16:50:50.209342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.209010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:45.672112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:04.977128+0000","last_change":"2024-03-28T16:52:04.977234+0000","last_active":"2024-03-28T16:52:04.977128+0000","last_peered":"2024-03-28T16:52:04.977128+0000","last_clean":"2024-03-28T16:52:04.977128+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:52:04.977128+0000","last_undegraded":"2024-03-28T16:52:04.977128+0000","last_fullsized":"2024-03-28T16:52:04.977128+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:04.976919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:04.976919+0000","last_clean_scrub_stamp":"2024-03-28T16:52:04.976919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:15:49.081133+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.212741+0000","last_change":"2024-03-28T16:51:41.212741+0000","last_active":"2024-03-28T16:51:41.212741+0000","last_peered":"2024-03-28T16:51:41.212741+0000","last_clean":"2024-03-28T16:51:41.212741+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:51:41.212741+0000","last_undegraded":"2024-03-28T16:51:41.212741+0000","last_fullsized":"2024-03-28T16:51:41.212741+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:41.212399+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:41.212399+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.212399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:25.434150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.209561+0000","last_change":"2024-03-28T16:51:32.209624+0000","last_active":"2024-03-28T16:51:32.209561+0000","last_peered":"2024-03-28T16:51:32.209561+0000","last_clean":"2024-03-28T16:51:32.209561+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:51:32.209561+0000","last_undegraded":"2024-03-28T16:51:32.209561+0000","last_fullsized":"2024-03-28T16:51:32.209561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.209381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.209381+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.209381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:58.294239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.553584+0000","last_change":"2024-03-28T16:51:27.553649+0000","last_active":"2024-03-28T16:51:27.553584+0000","last_peered":"2024-03-28T16:51:27.553584+0000","last_clean":"2024-03-28T16:51:27.553584+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:51:27.553584+0000","last_undegraded":"2024-03-28T16:51:27.553584+0000","last_fullsized":"2024-03-28T16:51:27.553584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:27.553385+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:27.553385+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.553385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:32.733520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.448938+0000","last_change":"2024-03-28T16:51:37.449001+0000","last_active":"2024-03-28T16:51:37.448938+0000","last_peered":"2024-03-28T16:51:37.448938+0000","last_clean":"2024-03-28T16:51:37.448938+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:51:37.448938+0000","last_undegraded":"2024-03-28T16:51:37.448938+0000","last_fullsized":"2024-03-28T16:51:37.448938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:37.448740+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:37.448740+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.448740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:45.427028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.141067+0000","last_change":"2024-03-28T16:51:32.141133+0000","last_active":"2024-03-28T16:51:32.141067+0000","last_peered":"2024-03-28T16:51:32.141067+0000","last_clean":"2024-03-28T16:51:32.141067+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:51:32.141067+0000","last_undegraded":"2024-03-28T16:51:32.141067+0000","last_fullsized":"2024-03-28T16:51:32.141067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:32.140882+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:32.140882+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.140882+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:30.700161+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:59.956831+0000","last_change":"2024-03-28T16:51:59.956889+0000","last_active":"2024-03-28T16:51:59.956831+0000","last_peered":"2024-03-28T16:51:59.956831+0000","last_clean":"2024-03-28T16:51:59.956831+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:51:59.956831+0000","last_undegraded":"2024-03-28T16:51:59.956831+0000","last_fullsized":"2024-03-28T16:51:59.956831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:59.956600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:59.956600+0000","last_clean_scrub_stamp":"2024-03-28T16:51:59.956600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:55:19.674683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.497847+0000","last_change":"2024-03-28T16:51:31.497957+0000","last_active":"2024-03-28T16:51:31.497847+0000","last_peered":"2024-03-28T16:51:31.497847+0000","last_clean":"2024-03-28T16:51:31.497847+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:51:31.497847+0000","last_undegraded":"2024-03-28T16:51:31.497847+0000","last_fullsized":"2024-03-28T16:51:31.497847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:31.497632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:31.497632+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.497632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:23.506227+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.564274+0000","last_change":"2024-03-28T16:51:26.564336+0000","last_active":"2024-03-28T16:51:26.564274+0000","last_peered":"2024-03-28T16:51:26.564274+0000","last_clean":"2024-03-28T16:51:26.564274+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:51:26.564274+0000","last_undegraded":"2024-03-28T16:51:26.564274+0000","last_fullsized":"2024-03-28T16:51:26.564274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:26.564092+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:26.564092+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.564092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:44.943347+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.523436+0000","last_change":"2024-03-28T16:51:21.523575+0000","last_active":"2024-03-28T16:51:21.523436+0000","last_peered":"2024-03-28T16:51:21.523436+0000","last_clean":"2024-03-28T16:51:21.523436+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:51:21.523436+0000","last_undegraded":"2024-03-28T16:51:21.523436+0000","last_fullsized":"2024-03-28T16:51:21.523436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.523213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:56.404435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.467825+0000","last_change":"2024-03-28T16:51:38.467878+0000","last_active":"2024-03-28T16:51:38.467825+0000","last_peered":"2024-03-28T16:51:38.467825+0000","last_clean":"2024-03-28T16:51:38.467825+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:51:38.467825+0000","last_undegraded":"2024-03-28T16:51:38.467825+0000","last_fullsized":"2024-03-28T16:51:38.467825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.467639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.467639+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.467639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:55:12.097451+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":71,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.278285+0000","last_change":"2024-03-28T16:51:19.278345+0000","last_active":"2024-03-28T16:51:19.278285+0000","last_peered":"2024-03-28T16:51:19.278285+0000","last_clean":"2024-03-28T16:51:19.278285+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:51:19.278285+0000","last_undegraded":"2024-03-28T16:51:19.278285+0000","last_fullsized":"2024-03-28T16:51:19.278285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.278071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:29.679472+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.988224+0000","last_change":"2024-03-28T16:51:37.988292+0000","last_active":"2024-03-28T16:51:37.988224+0000","last_peered":"2024-03-28T16:51:37.988224+0000","last_clean":"2024-03-28T16:51:37.988224+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:51:37.988224+0000","last_undegraded":"2024-03-28T16:51:37.988224+0000","last_fullsized":"2024-03-28T16:51:37.988224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.988041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.988041+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.988041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:35.010568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.446697+0000","last_change":"2024-03-28T16:51:18.446757+0000","last_active":"2024-03-28T16:51:18.446697+0000","last_peered":"2024-03-28T16:51:18.446697+0000","last_clean":"2024-03-28T16:51:18.446697+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:51:18.446697+0000","last_undegraded":"2024-03-28T16:51:18.446697+0000","last_fullsized":"2024-03-28T16:51:18.446697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.446501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:51.526879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.283525+0000","last_change":"2024-03-28T16:51:25.283581+0000","last_active":"2024-03-28T16:51:25.283525+0000","last_peered":"2024-03-28T16:51:25.283525+0000","last_clean":"2024-03-28T16:51:25.283525+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:51:25.283525+0000","last_undegraded":"2024-03-28T16:51:25.283525+0000","last_fullsized":"2024-03-28T16:51:25.283525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.283348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:28.047908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.316649+0000","last_change":"2024-03-28T16:51:21.316736+0000","last_active":"2024-03-28T16:51:21.316649+0000","last_peered":"2024-03-28T16:51:21.316649+0000","last_clean":"2024-03-28T16:51:21.316649+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:51:21.316649+0000","last_undegraded":"2024-03-28T16:51:21.316649+0000","last_fullsized":"2024-03-28T16:51:21.316649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.316476+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:10:20.931612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.420032+0000","last_change":"2024-03-28T16:51:40.420091+0000","last_active":"2024-03-28T16:51:40.420032+0000","last_peered":"2024-03-28T16:51:40.420032+0000","last_clean":"2024-03-28T16:51:40.420032+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:51:40.420032+0000","last_undegraded":"2024-03-28T16:51:40.420032+0000","last_fullsized":"2024-03-28T16:51:40.420032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:40.419842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:40.419842+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.419842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:10.034139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.289210+0000","last_change":"2024-03-28T16:51:10.289210+0000","last_active":"2024-03-28T16:51:10.289210+0000","last_peered":"2024-03-28T16:51:10.289210+0000","last_clean":"2024-03-28T16:51:10.289210+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:51:10.289210+0000","last_undegraded":"2024-03-28T16:51:10.289210+0000","last_fullsized":"2024-03-28T16:51:10.289210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.288939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:52.264862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:00.975491+0000","last_change":"2024-03-28T16:52:00.975620+0000","last_active":"2024-03-28T16:52:00.975491+0000","last_peered":"2024-03-28T16:52:00.975491+0000","last_clean":"2024-03-28T16:52:00.975491+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:52:00.975491+0000","last_undegraded":"2024-03-28T16:52:00.975491+0000","last_fullsized":"2024-03-28T16:52:00.975491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:52:00.975275+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:52:00.975275+0000","last_clean_scrub_stamp":"2024-03-28T16:52:00.975275+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:02:12.413542+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.144122+0000","last_change":"2024-03-28T16:51:27.144188+0000","last_active":"2024-03-28T16:51:27.144122+0000","last_peered":"2024-03-28T16:51:27.144122+0000","last_clean":"2024-03-28T16:51:27.144122+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:51:27.144122+0000","last_undegraded":"2024-03-28T16:51:27.144122+0000","last_fullsized":"2024-03-28T16:51:27.144122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:27.143940+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:27.143940+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.143940+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:09:52.594727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.482752+0000","last_change":"2024-03-28T16:51:32.482808+0000","last_active":"2024-03-28T16:51:32.482752+0000","last_peered":"2024-03-28T16:51:32.482752+0000","last_clean":"2024-03-28T16:51:32.482752+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:51:32.482752+0000","last_undegraded":"2024-03-28T16:51:32.482752+0000","last_fullsized":"2024-03-28T16:51:32.482752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.482535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.482535+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.482535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:16.704285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.557168+0000","last_change":"2024-03-28T16:51:23.557231+0000","last_active":"2024-03-28T16:51:23.557168+0000","last_peered":"2024-03-28T16:51:23.557168+0000","last_clean":"2024-03-28T16:51:23.557168+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:51:23.557168+0000","last_undegraded":"2024-03-28T16:51:23.557168+0000","last_fullsized":"2024-03-28T16:51:23.557168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:23.556975+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:23.556975+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.556975+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:49.641832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.301018+0000","last_change":"2024-03-28T16:51:22.301084+0000","last_active":"2024-03-28T16:51:22.301018+0000","last_peered":"2024-03-28T16:51:22.301018+0000","last_clean":"2024-03-28T16:51:22.301018+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:51:22.301018+0000","last_undegraded":"2024-03-28T16:51:22.301018+0000","last_fullsized":"2024-03-28T16:51:22.301018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.300834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:39.676855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.261019+0000","last_change":"2024-03-28T16:51:18.261152+0000","last_active":"2024-03-28T16:51:18.261019+0000","last_peered":"2024-03-28T16:51:18.261019+0000","last_clean":"2024-03-28T16:51:18.261019+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:51:18.261019+0000","last_undegraded":"2024-03-28T16:51:18.261019+0000","last_fullsized":"2024-03-28T16:51:18.261019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.260778+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:15.421901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.256951+0000","last_change":"2024-03-28T16:51:24.257006+0000","last_active":"2024-03-28T16:51:24.256951+0000","last_peered":"2024-03-28T16:51:24.256951+0000","last_clean":"2024-03-28T16:51:24.256951+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:51:24.256951+0000","last_undegraded":"2024-03-28T16:51:24.256951+0000","last_fullsized":"2024-03-28T16:51:24.256951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.256765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:47.235830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.319730+0000","last_change":"2024-03-28T16:51:14.319730+0000","last_active":"2024-03-28T16:51:14.319730+0000","last_peered":"2024-03-28T16:51:14.319730+0000","last_clean":"2024-03-28T16:51:14.319730+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:51:14.319730+0000","last_undegraded":"2024-03-28T16:51:14.319730+0000","last_fullsized":"2024-03-28T16:51:14.319730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.319444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:39.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.080203+0000","last_change":"2024-03-28T16:51:36.080262+0000","last_active":"2024-03-28T16:51:36.080203+0000","last_peered":"2024-03-28T16:51:36.080203+0000","last_clean":"2024-03-28T16:51:36.080203+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:51:36.080203+0000","last_undegraded":"2024-03-28T16:51:36.080203+0000","last_fullsized":"2024-03-28T16:51:36.080203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.080017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.080017+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.080017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:46.963031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.281665+0000","last_change":"2024-03-28T16:51:06.281665+0000","last_active":"2024-03-28T16:51:06.281665+0000","last_peered":"2024-03-28T16:51:06.281665+0000","last_clean":"2024-03-28T16:51:06.281665+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:51:06.281665+0000","last_undegraded":"2024-03-28T16:51:06.281665+0000","last_fullsized":"2024-03-28T16:51:06.281665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.281413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:32.845661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:48.035856+0000","last_change":"2024-03-28T16:51:48.035917+0000","last_active":"2024-03-28T16:51:48.035856+0000","last_peered":"2024-03-28T16:51:48.035856+0000","last_clean":"2024-03-28T16:51:48.035856+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:51:48.035856+0000","last_undegraded":"2024-03-28T16:51:48.035856+0000","last_fullsized":"2024-03-28T16:51:48.035856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:48.035665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:48.035665+0000","last_clean_scrub_stamp":"2024-03-28T16:51:48.035665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:20.222230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.107093+0000","last_change":"2024-03-28T16:51:23.107159+0000","last_active":"2024-03-28T16:51:23.107093+0000","last_peered":"2024-03-28T16:51:23.107093+0000","last_clean":"2024-03-28T16:51:23.107093+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:51:23.107093+0000","last_undegraded":"2024-03-28T16:51:23.107093+0000","last_fullsized":"2024-03-28T16:51:23.107093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:23.106864+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:23.106864+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.106864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:31.824473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.338591+0000","last_change":"2024-03-28T16:51:23.338657+0000","last_active":"2024-03-28T16:51:23.338591+0000","last_peered":"2024-03-28T16:51:23.338591+0000","last_clean":"2024-03-28T16:51:23.338591+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:51:23.338591+0000","last_undegraded":"2024-03-28T16:51:23.338591+0000","last_fullsized":"2024-03-28T16:51:23.338591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:37.364884+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'10","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.275847+0000","last_change":"2024-03-28T16:51:23.275918+0000","last_active":"2024-03-28T16:51:23.275847+0000","last_peered":"2024-03-28T16:51:23.275847+0000","last_clean":"2024-03-28T16:51:23.275847+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:51:23.275847+0000","last_undegraded":"2024-03-28T16:51:23.275847+0000","last_fullsized":"2024-03-28T16:51:23.275847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.275672+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:58.156894+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.083211+0000","last_change":"2024-03-28T16:51:40.083323+0000","last_active":"2024-03-28T16:51:40.083211+0000","last_peered":"2024-03-28T16:51:40.083211+0000","last_clean":"2024-03-28T16:51:40.083211+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:51:40.083211+0000","last_undegraded":"2024-03-28T16:51:40.083211+0000","last_fullsized":"2024-03-28T16:51:40.083211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:40.082968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:40.082968+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.082968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:49:31.036900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:42.046958+0000","last_change":"2024-03-28T16:51:42.047018+0000","last_active":"2024-03-28T16:51:42.046958+0000","last_peered":"2024-03-28T16:51:42.046958+0000","last_clean":"2024-03-28T16:51:42.046958+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:51:42.046958+0000","last_undegraded":"2024-03-28T16:51:42.046958+0000","last_fullsized":"2024-03-28T16:51:42.046958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:42.046771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:42.046771+0000","last_clean_scrub_stamp":"2024-03-28T16:51:42.046771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:14:36.194510+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.440088+0000","last_change":"2024-03-28T16:51:16.440088+0000","last_active":"2024-03-28T16:51:16.440088+0000","last_peered":"2024-03-28T16:51:16.440088+0000","last_clean":"2024-03-28T16:51:16.440088+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:51:16.440088+0000","last_undegraded":"2024-03-28T16:51:16.440088+0000","last_fullsized":"2024-03-28T16:51:16.440088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.439837+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:19.204397+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.539510+0000","last_change":"2024-03-28T16:51:30.539564+0000","last_active":"2024-03-28T16:51:30.539510+0000","last_peered":"2024-03-28T16:51:30.539510+0000","last_clean":"2024-03-28T16:51:30.539510+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:51:30.539510+0000","last_undegraded":"2024-03-28T16:51:30.539510+0000","last_fullsized":"2024-03-28T16:51:30.539510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.539307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.539307+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.539307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:55:52.038876+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.253966+0000","last_change":"2024-03-28T16:51:29.254037+0000","last_active":"2024-03-28T16:51:29.253966+0000","last_peered":"2024-03-28T16:51:29.253966+0000","last_clean":"2024-03-28T16:51:29.253966+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:51:29.253966+0000","last_undegraded":"2024-03-28T16:51:29.253966+0000","last_fullsized":"2024-03-28T16:51:29.253966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:29.253788+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:29.253788+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.253788+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:02:14.899395+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":339,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1992,"ondisk_log_size":1992,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1539,"num_read_kb":1325,"num_write":581,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1750,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1422,"ondisk_log_size":1422,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":866,"num_read_kb":864,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705708,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44076,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327764,"statfs":{"total":96636764160,"available":96591630336,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.58999999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.87,"5min":1.246,"15min":1.246},"last":0.371},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82399999999999995,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.47399999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.82299999999999995,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.82599999999999996,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.51600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":1.671,"15min":1.671},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.53100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94399999999999995,"5min":1.331,"15min":1.331},"last":0.55200000000000005}]}]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.64100000000000001},"min":{"1min":0.57699999999999996,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":2.98,"15min":2.98},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64800000000000002,"15min":0.629},"min":{"1min":0.58199999999999996,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82999999999999996,"5min":1.01,"15min":1.01},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.433,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.75900000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57299999999999995,"15min":0.54400000000000004},"min":{"1min":0.47499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.95899999999999996,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.69799999999999995}]},{"osd":3,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.48099999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.76100000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.498,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.78600000000000003,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.64400000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705708,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.54900000000000004,"5min":0.309,"15min":0.309},"max":{"1min":0.75900000000000001,"5min":0.878,"15min":0.878},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.57099999999999995,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.91400000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.46400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69899999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52800000000000002,"15min":0.505},"min":{"1min":0.437,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.79600000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.47499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.76500000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.54900000000000004,"5min":0.246,"15min":0.246},"max":{"1min":0.82399999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.76700000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.503,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88400000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.753},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.65800000000000003},"min":{"1min":0.58299999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Mar 28 16:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.53200000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58499999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.874}]},{"osd":3,"last update":"Thu Mar 28 16:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.62,"5min":0.372,"15min":0.372},"max":{"1min":0.94199999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89400000000000002,"5min":1.117,"15min":1.117},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:52:08.893 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:52:08.894 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:52:08.895 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:52:08.895 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-03-28T16:49:03.693154+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=49, tm_sec=3, tm_wday=3, tm_yday=88, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=28, tm_hour=16, tm_min=50, tm_sec=44, tm_wday=3, tm_yday=88, tm_isdst=0) 2024-03-28T16:52:08.895 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-28T16:52:28.897 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T16:52:29.025 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0fb596640 1 Processor -- start 2024-03-28T16:52:29.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0fb596640 1 -- start start 2024-03-28T16:52:29.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0fb596640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa0f41290f0 0x7fa0f412c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:29.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0fb596640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f4095ee0 con 0x7fa0f4128d10 2024-03-28T16:52:29.026 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0fb596640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f4096320 con 0x7fa0f41290f0 2024-03-28T16:52:29.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0f8b31640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa0f41290f0 0x7fa0f412c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:29.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0f8b31640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa0f41290f0 0x7fa0f412c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59768/0 (socket says 172.21.15.119:59768) 2024-03-28T16:52:29.027 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0f9332640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa0f4128d10 0x7fa0f412bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52678/0 (socket says 172.21.15.119:52678) 2024-03-28T16:52:29.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0f8b31640 1 -- 172.21.15.119:0/4017952675 learned_addr learned my addr 172.21.15.119:0/4017952675 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:52:29.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0f8b31640 1 -- 172.21.15.119:0/4017952675 >> v1:172.21.15.119:6789/0 conn(0x7fa0f4128d10 legacy=0x7fa0f412bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.028 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.023+0000 7fa0f8b31640 1 -- 172.21.15.119:0/4017952675 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4128620 con 0x7fa0f41290f0 2024-03-28T16:52:29.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0f8b31640 1 --2- 172.21.15.119:0/4017952675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f41290f0 0x7fa0f412c230 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009e50 tx=0x7fa0e801caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fc2f11a4fce105 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:29.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0f3fff640 1 -- 172.21.15.119:0/4017952675 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e8009200 con 0x7fa0f41290f0 2024-03-28T16:52:29.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0f3fff640 1 -- 172.21.15.119:0/4017952675 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e80a6640 con 0x7fa0f41290f0 2024-03-28T16:52:29.029 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0f3fff640 1 -- 172.21.15.119:0/4017952675 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0e80a6900 con 0x7fa0f41290f0 2024-03-28T16:52:29.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 -- 172.21.15.119:0/4017952675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f41290f0 msgr2=0x7fa0f412c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 --2- 172.21.15.119:0/4017952675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f41290f0 0x7fa0f412c230 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009e50 tx=0x7fa0e801caa0 comp rx=0 tx=0).stop 2024-03-28T16:52:29.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 -- 172.21.15.119:0/4017952675 shutdown_connections 2024-03-28T16:52:29.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 --2- 172.21.15.119:0/4017952675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f41290f0 0x7fa0f412c230 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:29.030 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 -- 172.21.15.119:0/4017952675 >> 172.21.15.119:0/4017952675 conn(0x7fa0f409a2f0 msgr2=0x7fa0f409a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:52:29.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 -- 172.21.15.119:0/4017952675 shutdown_connections 2024-03-28T16:52:29.031 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 -- 172.21.15.119:0/4017952675 wait complete. 2024-03-28T16:52:29.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 Processor -- start 2024-03-28T16:52:29.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 -- start start 2024-03-28T16:52:29.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4128d10 0x7fa0f41c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:29.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0fb596640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f4130820 con 0x7fa0f4128d10 2024-03-28T16:52:29.033 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0f9332640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4128d10 0x7fa0f41c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:29.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0f9332640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4128d10 0x7fa0f41c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59780/0 (socket says 172.21.15.119:59780) 2024-03-28T16:52:29.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.027+0000 7fa0f9332640 1 -- 172.21.15.119:0/528521682 learned_addr learned my addr 172.21.15.119:0/528521682 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:52:29.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f9332640 1 -- 172.21.15.119:0/528521682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e8009b30 con 0x7fa0f4128d10 2024-03-28T16:52:29.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f9332640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4128d10 0x7fa0f41c3010 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c3a0 tx=0x7fa0ec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:29.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f1ffb640 1 -- 172.21.15.119:0/528521682 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec007410 con 0x7fa0f4128d10 2024-03-28T16:52:29.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f1ffb640 1 -- 172.21.15.119:0/528521682 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec007570 con 0x7fa0f4128d10 2024-03-28T16:52:29.034 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f4123650 con 0x7fa0f4128d10 2024-03-28T16:52:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f1ffb640 1 -- 172.21.15.119:0/528521682 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec0047f0 con 0x7fa0f4128d10 2024-03-28T16:52:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f4130ae0 con 0x7fa0f4128d10 2024-03-28T16:52:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f4067e00 con 0x7fa0f4128d10 2024-03-28T16:52:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f1ffb640 1 -- 172.21.15.119:0/528521682 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7fa0ec004950 con 0x7fa0f4128d10 2024-03-28T16:52:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f1ffb640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa0e0035a10 0x7fa0e0037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f1ffb640 1 -- 172.21.15.119:0/528521682 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa0ec0480c0 con 0x7fa0f4128d10 2024-03-28T16:52:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f8b31640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa0e0035a10 0x7fa0e0037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:29.036 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.031+0000 7fa0f8b31640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa0e0035a10 0x7fa0e0037ed0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80af000 tx=0x7fa0e80a8070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:29.037 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.035+0000 7fa0f1ffb640 1 -- 172.21.15.119:0/528521682 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0ec004e30 con 0x7fa0f4128d10 2024-03-28T16:52:29.188 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.187+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 --> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0f41234d0 con 0x7fa0e0035a10 2024-03-28T16:52:29.233 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.231+0000 7fa0f1ffb640 1 -- 172.21.15.119:0/528521682 <== mgr.4100 v2:172.21.15.170:6824/317743614 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+673618 (secure 0 0 0) 0x7fa0e80a67b0 con 0x7fa0e0035a10 2024-03-28T16:52:29.240 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-03-28T16:52:29.246 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2024-03-28T16:52:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa0e0035a10 msgr2=0x7fa0e0037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa0e0035a10 0x7fa0e0037ed0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80af000 tx=0x7fa0e80a8070 comp rx=0 tx=0).stop 2024-03-28T16:52:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4128d10 msgr2=0x7fa0f41c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4128d10 0x7fa0f41c3010 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c3a0 tx=0x7fa0ec00c860 comp rx=0 tx=0).stop 2024-03-28T16:52:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 shutdown_connections 2024-03-28T16:52:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7fa0e0035a10 0x7fa0e0037ed0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:29.248 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 --2- 172.21.15.119:0/528521682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0f4128d10 0x7fa0f41c3010 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 >> 172.21.15.119:0/528521682 conn(0x7fa0f409a2f0 msgr2=0x7fa0f4097010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:52:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 shutdown_connections 2024-03-28T16:52:29.249 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.239+0000 7fa0fb596640 1 -- 172.21.15.119:0/528521682 wait complete. 2024-03-28T16:52:29.262 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-03-28T16:52:28.588085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601336,"num_objects":231,"num_object_clones":0,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":4029,"num_read_kb":3679,"num_write":1543,"num_write_kb":31938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2298,"num_omap_keys":13,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4057,"ondisk_log_size":4057,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":164928,"kb_used_data":2784,"kb_used_omap":6,"kb_used_meta":162105,"kb_avail":377322432,"statfs":{"total":386547056640,"available":386378170368,"internally_reserved":0,"allocated":2850816,"data_stored":1709332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":165996310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006012"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.238075+0000","last_change":"2024-03-28T16:51:28.238133+0000","last_active":"2024-03-28T16:51:28.238075+0000","last_peered":"2024-03-28T16:51:28.238075+0000","last_clean":"2024-03-28T16:51:28.238075+0000","last_became_active":"2024-03-28T16:48:54.655600+0000","last_became_peered":"2024-03-28T16:48:54.655600+0000","last_unstale":"2024-03-28T16:51:28.238075+0000","last_undegraded":"2024-03-28T16:51:28.238075+0000","last_fullsized":"2024-03-28T16:51:28.238075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:28.237880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:28.237880+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.237880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:51.619287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'2","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.303646+0000","last_change":"2024-03-28T16:51:11.303646+0000","last_active":"2024-03-28T16:51:11.303646+0000","last_peered":"2024-03-28T16:51:11.303646+0000","last_clean":"2024-03-28T16:51:11.303646+0000","last_became_active":"2024-03-28T16:48:56.687002+0000","last_became_peered":"2024-03-28T16:48:56.687002+0000","last_unstale":"2024-03-28T16:51:11.303646+0000","last_undegraded":"2024-03-28T16:51:11.303646+0000","last_fullsized":"2024-03-28T16:51:11.303646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:11.303263+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.303263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:59.401107+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.569902+0000","last_change":"2024-03-28T16:51:22.569959+0000","last_active":"2024-03-28T16:51:22.569902+0000","last_peered":"2024-03-28T16:51:22.569902+0000","last_clean":"2024-03-28T16:51:22.569902+0000","last_became_active":"2024-03-28T16:48:54.665565+0000","last_became_peered":"2024-03-28T16:48:54.665565+0000","last_unstale":"2024-03-28T16:51:22.569902+0000","last_undegraded":"2024-03-28T16:51:22.569902+0000","last_fullsized":"2024-03-28T16:51:22.569902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:22.569729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:22.569729+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.569729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:34.374938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.098164+0000","last_change":"2024-03-28T16:51:22.098164+0000","last_active":"2024-03-28T16:51:22.098164+0000","last_peered":"2024-03-28T16:51:22.098164+0000","last_clean":"2024-03-28T16:51:22.098164+0000","last_became_active":"2024-03-28T16:48:56.687410+0000","last_became_peered":"2024-03-28T16:48:56.687410+0000","last_unstale":"2024-03-28T16:51:22.098164+0000","last_undegraded":"2024-03-28T16:51:22.098164+0000","last_fullsized":"2024-03-28T16:51:22.098164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:22.097907+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:22.097907+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.097907+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:26.268880+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.105158+0000","last_change":"2024-03-28T16:51:39.105220+0000","last_active":"2024-03-28T16:51:39.105158+0000","last_peered":"2024-03-28T16:51:39.105158+0000","last_clean":"2024-03-28T16:51:39.105158+0000","last_became_active":"2024-03-28T16:48:54.664620+0000","last_became_peered":"2024-03-28T16:48:54.664620+0000","last_unstale":"2024-03-28T16:51:39.105158+0000","last_undegraded":"2024-03-28T16:51:39.105158+0000","last_fullsized":"2024-03-28T16:51:39.105158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.104968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.104968+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.104968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:12:14.485249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"31'4","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.105540+0000","last_change":"2024-03-28T16:51:19.105540+0000","last_active":"2024-03-28T16:51:19.105540+0000","last_peered":"2024-03-28T16:51:19.105540+0000","last_clean":"2024-03-28T16:51:19.105540+0000","last_became_active":"2024-03-28T16:48:56.690436+0000","last_became_peered":"2024-03-28T16:48:56.690436+0000","last_unstale":"2024-03-28T16:51:19.105540+0000","last_undegraded":"2024-03-28T16:51:19.105540+0000","last_fullsized":"2024-03-28T16:51:19.105540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:19.105244+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.105244+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:35:56.759918+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"31'3","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.232984+0000","last_change":"2024-03-28T16:51:29.233046+0000","last_active":"2024-03-28T16:51:29.232984+0000","last_peered":"2024-03-28T16:51:29.232984+0000","last_clean":"2024-03-28T16:51:29.232984+0000","last_became_active":"2024-03-28T16:48:54.656256+0000","last_became_peered":"2024-03-28T16:48:54.656256+0000","last_unstale":"2024-03-28T16:51:29.232984+0000","last_undegraded":"2024-03-28T16:51:29.232984+0000","last_fullsized":"2024-03-28T16:51:29.232984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:51:29.232791+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:51:29.232791+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.232791+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:46.608812+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":11,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"31'2","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.197158+0000","last_change":"2024-03-28T16:51:37.197158+0000","last_active":"2024-03-28T16:51:37.197158+0000","last_peered":"2024-03-28T16:51:37.197158+0000","last_clean":"2024-03-28T16:51:37.197158+0000","last_became_active":"2024-03-28T16:49:37.601502+0000","last_became_peered":"2024-03-28T16:49:37.601502+0000","last_unstale":"2024-03-28T16:51:37.197158+0000","last_undegraded":"2024-03-28T16:51:37.197158+0000","last_fullsized":"2024-03-28T16:51:37.197158+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:37.196925+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:37.196925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.196925+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:33.164623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:52.054519+0000","last_change":"2024-03-28T16:51:52.054648+0000","last_active":"2024-03-28T16:51:52.054519+0000","last_peered":"2024-03-28T16:51:52.054519+0000","last_clean":"2024-03-28T16:51:52.054519+0000","last_became_active":"2024-03-28T16:48:54.661188+0000","last_became_peered":"2024-03-28T16:48:54.661188+0000","last_unstale":"2024-03-28T16:51:52.054519+0000","last_undegraded":"2024-03-28T16:51:52.054519+0000","last_fullsized":"2024-03-28T16:51:52.054519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:52.054305+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:52.054305+0000","last_clean_scrub_stamp":"2024-03-28T16:51:52.054305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:28.349668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.305502+0000","last_change":"2024-03-28T16:51:16.305559+0000","last_active":"2024-03-28T16:51:16.305502+0000","last_peered":"2024-03-28T16:51:16.305502+0000","last_clean":"2024-03-28T16:51:16.305502+0000","last_became_active":"2024-03-28T16:48:56.670664+0000","last_became_peered":"2024-03-28T16:48:56.670664+0000","last_unstale":"2024-03-28T16:51:16.305502+0000","last_undegraded":"2024-03-28T16:51:16.305502+0000","last_fullsized":"2024-03-28T16:51:16.305502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:16.305318+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.305318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:43.571649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.321896+0000","last_change":"2024-03-28T16:51:21.321896+0000","last_active":"2024-03-28T16:51:21.321896+0000","last_peered":"2024-03-28T16:51:21.321896+0000","last_clean":"2024-03-28T16:51:21.321896+0000","last_became_active":"2024-03-28T16:48:54.665155+0000","last_became_peered":"2024-03-28T16:48:54.665155+0000","last_unstale":"2024-03-28T16:51:21.321896+0000","last_undegraded":"2024-03-28T16:51:21.321896+0000","last_fullsized":"2024-03-28T16:51:21.321896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.321658+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.321658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:30.540834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.057665+0000","last_change":"2024-03-28T16:51:20.057727+0000","last_active":"2024-03-28T16:51:20.057665+0000","last_peered":"2024-03-28T16:51:20.057665+0000","last_clean":"2024-03-28T16:51:20.057665+0000","last_became_active":"2024-03-28T16:48:56.688076+0000","last_became_peered":"2024-03-28T16:48:56.688076+0000","last_unstale":"2024-03-28T16:51:20.057665+0000","last_undegraded":"2024-03-28T16:51:20.057665+0000","last_fullsized":"2024-03-28T16:51:20.057665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-03-28T16:51:20.057477+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.057477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:01:08.878405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"31'4","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:47.069078+0000","last_change":"2024-03-28T16:51:47.069144+0000","last_active":"2024-03-28T16:51:47.069078+0000","last_peered":"2024-03-28T16:51:47.069078+0000","last_clean":"2024-03-28T16:51:47.069078+0000","last_became_active":"2024-03-28T16:48:54.671834+0000","last_became_peered":"2024-03-28T16:48:54.671834+0000","last_unstale":"2024-03-28T16:51:47.069078+0000","last_undegraded":"2024-03-28T16:51:47.069078+0000","last_fullsized":"2024-03-28T16:51:47.069078+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:47.068883+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:47.068883+0000","last_clean_scrub_stamp":"2024-03-28T16:51:47.068883+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:46:12.543407+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"31'16","reported_seq":61,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.206567+0000","last_change":"2024-03-28T16:51:34.206624+0000","last_active":"2024-03-28T16:51:34.206567+0000","last_peered":"2024-03-28T16:51:34.206567+0000","last_clean":"2024-03-28T16:51:34.206567+0000","last_became_active":"2024-03-28T16:48:56.695370+0000","last_became_peered":"2024-03-28T16:48:56.695370+0000","last_unstale":"2024-03-28T16:51:34.206567+0000","last_undegraded":"2024-03-28T16:51:34.206567+0000","last_fullsized":"2024-03-28T16:51:34.206567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:51:34.206377+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:51:34.206377+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.206377+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:21.188653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":35,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.467047+0000","last_change":"2024-03-28T16:51:39.467111+0000","last_active":"2024-03-28T16:51:39.467047+0000","last_peered":"2024-03-28T16:51:39.467047+0000","last_clean":"2024-03-28T16:51:39.467047+0000","last_became_active":"2024-03-28T16:48:54.657071+0000","last_became_peered":"2024-03-28T16:48:54.657071+0000","last_unstale":"2024-03-28T16:51:39.467047+0000","last_undegraded":"2024-03-28T16:51:39.467047+0000","last_fullsized":"2024-03-28T16:51:39.467047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.466858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.466858+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.466858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:31.475945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.353532+0000","last_change":"2024-03-28T16:51:12.353604+0000","last_active":"2024-03-28T16:51:12.353532+0000","last_peered":"2024-03-28T16:51:12.353532+0000","last_clean":"2024-03-28T16:51:12.353532+0000","last_became_active":"2024-03-28T16:48:56.680373+0000","last_became_peered":"2024-03-28T16:48:56.680373+0000","last_unstale":"2024-03-28T16:51:12.353532+0000","last_undegraded":"2024-03-28T16:51:12.353532+0000","last_fullsized":"2024-03-28T16:51:12.353532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:12.353309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.353309+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:42.028446+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.145909+0000","last_change":"2024-03-28T16:51:28.145970+0000","last_active":"2024-03-28T16:51:28.145909+0000","last_peered":"2024-03-28T16:51:28.145909+0000","last_clean":"2024-03-28T16:51:28.145909+0000","last_became_active":"2024-03-28T16:48:56.676827+0000","last_became_peered":"2024-03-28T16:48:56.676827+0000","last_unstale":"2024-03-28T16:51:28.145909+0000","last_undegraded":"2024-03-28T16:51:28.145909+0000","last_fullsized":"2024-03-28T16:51:28.145909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:28.145685+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:28.145685+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.145685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:12.221861+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.234222+0000","last_change":"2024-03-28T16:51:28.234290+0000","last_active":"2024-03-28T16:51:28.234222+0000","last_peered":"2024-03-28T16:51:28.234222+0000","last_clean":"2024-03-28T16:51:28.234222+0000","last_became_active":"2024-03-28T16:48:54.670444+0000","last_became_peered":"2024-03-28T16:48:54.670444+0000","last_unstale":"2024-03-28T16:51:28.234222+0000","last_undegraded":"2024-03-28T16:51:28.234222+0000","last_fullsized":"2024-03-28T16:51:28.234222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:28.234045+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:28.234045+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.234045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:10:44.817774+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"31'10","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.246752+0000","last_change":"2024-03-28T16:51:07.246752+0000","last_active":"2024-03-28T16:51:07.246752+0000","last_peered":"2024-03-28T16:51:07.246752+0000","last_clean":"2024-03-28T16:51:07.246752+0000","last_became_active":"2024-03-28T16:48:56.692585+0000","last_became_peered":"2024-03-28T16:48:56.692585+0000","last_unstale":"2024-03-28T16:51:07.246752+0000","last_undegraded":"2024-03-28T16:51:07.246752+0000","last_fullsized":"2024-03-28T16:51:07.246752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:07.246384+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.246384+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:42.040943+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.241054+0000","last_change":"2024-03-28T16:51:39.241109+0000","last_active":"2024-03-28T16:51:39.241054+0000","last_peered":"2024-03-28T16:51:39.241054+0000","last_clean":"2024-03-28T16:51:39.241054+0000","last_became_active":"2024-03-28T16:49:37.609965+0000","last_became_peered":"2024-03-28T16:49:37.609965+0000","last_unstale":"2024-03-28T16:51:39.241054+0000","last_undegraded":"2024-03-28T16:51:39.241054+0000","last_fullsized":"2024-03-28T16:51:39.241054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:39.240890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:39.240890+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.240890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:56:00.740460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.251563+0000","last_change":"2024-03-28T16:51:08.251563+0000","last_active":"2024-03-28T16:51:08.251563+0000","last_peered":"2024-03-28T16:51:08.251563+0000","last_clean":"2024-03-28T16:51:08.251563+0000","last_became_active":"2024-03-28T16:48:56.683941+0000","last_became_peered":"2024-03-28T16:48:56.683941+0000","last_unstale":"2024-03-28T16:51:08.251563+0000","last_undegraded":"2024-03-28T16:51:08.251563+0000","last_fullsized":"2024-03-28T16:51:08.251563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:08.251179+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.251179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:34.785916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.213499+0000","last_change":"2024-03-28T16:51:30.213568+0000","last_active":"2024-03-28T16:51:30.213499+0000","last_peered":"2024-03-28T16:51:30.213499+0000","last_clean":"2024-03-28T16:51:30.213499+0000","last_became_active":"2024-03-28T16:48:54.669637+0000","last_became_peered":"2024-03-28T16:48:54.669637+0000","last_unstale":"2024-03-28T16:51:30.213499+0000","last_undegraded":"2024-03-28T16:51:30.213499+0000","last_fullsized":"2024-03-28T16:51:30.213499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.213281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.213281+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.213281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:48.948290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.549877+0000","last_change":"2024-03-28T16:51:25.549934+0000","last_active":"2024-03-28T16:51:25.549877+0000","last_peered":"2024-03-28T16:51:25.549877+0000","last_clean":"2024-03-28T16:51:25.549877+0000","last_became_active":"2024-03-28T16:48:56.677833+0000","last_became_peered":"2024-03-28T16:48:56.677833+0000","last_unstale":"2024-03-28T16:51:25.549877+0000","last_undegraded":"2024-03-28T16:51:25.549877+0000","last_fullsized":"2024-03-28T16:51:25.549877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:25.549686+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:25.549686+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.549686+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:12.531962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.161305+0000","last_change":"2024-03-28T16:51:33.161357+0000","last_active":"2024-03-28T16:51:33.161305+0000","last_peered":"2024-03-28T16:51:33.161305+0000","last_clean":"2024-03-28T16:51:33.161305+0000","last_became_active":"2024-03-28T16:48:54.654085+0000","last_became_peered":"2024-03-28T16:48:54.654085+0000","last_unstale":"2024-03-28T16:51:33.161305+0000","last_undegraded":"2024-03-28T16:51:33.161305+0000","last_fullsized":"2024-03-28T16:51:33.161305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.161112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.161112+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.161112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:23.989489+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"31'8","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.320871+0000","last_change":"2024-03-28T16:51:11.320871+0000","last_active":"2024-03-28T16:51:11.320871+0000","last_peered":"2024-03-28T16:51:11.320871+0000","last_clean":"2024-03-28T16:51:11.320871+0000","last_became_active":"2024-03-28T16:48:56.680328+0000","last_became_peered":"2024-03-28T16:48:56.680328+0000","last_unstale":"2024-03-28T16:51:11.320871+0000","last_undegraded":"2024-03-28T16:51:11.320871+0000","last_fullsized":"2024-03-28T16:51:11.320871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:11.320440+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.320440+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:29.097635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.480322+0000","last_change":"2024-03-28T16:51:20.480322+0000","last_active":"2024-03-28T16:51:20.480322+0000","last_peered":"2024-03-28T16:51:20.480322+0000","last_clean":"2024-03-28T16:51:20.480322+0000","last_became_active":"2024-03-28T16:48:54.669879+0000","last_became_peered":"2024-03-28T16:48:54.669879+0000","last_unstale":"2024-03-28T16:51:20.480322+0000","last_undegraded":"2024-03-28T16:51:20.480322+0000","last_fullsized":"2024-03-28T16:51:20.480322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.479971+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.479971+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:37.855992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.332059+0000","last_change":"2024-03-28T16:51:15.332059+0000","last_active":"2024-03-28T16:51:15.332059+0000","last_peered":"2024-03-28T16:51:15.332059+0000","last_clean":"2024-03-28T16:51:15.332059+0000","last_became_active":"2024-03-28T16:48:56.671062+0000","last_became_peered":"2024-03-28T16:48:56.671062+0000","last_unstale":"2024-03-28T16:51:15.332059+0000","last_undegraded":"2024-03-28T16:51:15.332059+0000","last_fullsized":"2024-03-28T16:51:15.332059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:15.331796+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.331796+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:37:44.933269+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:51.016244+0000","last_change":"2024-03-28T16:51:51.016313+0000","last_active":"2024-03-28T16:51:51.016244+0000","last_peered":"2024-03-28T16:51:51.016244+0000","last_clean":"2024-03-28T16:51:51.016244+0000","last_became_active":"2024-03-28T16:48:54.672490+0000","last_became_peered":"2024-03-28T16:48:54.672490+0000","last_unstale":"2024-03-28T16:51:51.016244+0000","last_undegraded":"2024-03-28T16:51:51.016244+0000","last_fullsized":"2024-03-28T16:51:51.016244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:51.016042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:51.016042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:51.016042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:47:25.750167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"31'20","reported_seq":88,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.046926+0000","last_change":"2024-03-28T16:51:40.046995+0000","last_active":"2024-03-28T16:51:40.046926+0000","last_peered":"2024-03-28T16:51:40.046926+0000","last_clean":"2024-03-28T16:51:40.046926+0000","last_became_active":"2024-03-28T16:49:04.738328+0000","last_became_peered":"2024-03-28T16:49:04.738328+0000","last_unstale":"2024-03-28T16:51:40.046926+0000","last_undegraded":"2024-03-28T16:51:40.046926+0000","last_fullsized":"2024-03-28T16:51:40.046926+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'20","last_scrub_stamp":"2024-03-28T16:51:40.046736+0000","last_deep_scrub":"31'20","last_deep_scrub_stamp":"2024-03-28T16:51:40.046736+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.046736+0000","objects_scrubbed":1,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:55:13.676549+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":141,"num_read_kb":140,"num_write":53,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'4","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.132778+0000","last_change":"2024-03-28T16:51:24.132846+0000","last_active":"2024-03-28T16:51:24.132778+0000","last_peered":"2024-03-28T16:51:24.132778+0000","last_clean":"2024-03-28T16:51:24.132778+0000","last_became_active":"2024-03-28T16:48:56.683489+0000","last_became_peered":"2024-03-28T16:48:56.683489+0000","last_unstale":"2024-03-28T16:51:24.132778+0000","last_undegraded":"2024-03-28T16:51:24.132778+0000","last_fullsized":"2024-03-28T16:51:24.132778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:24.132547+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:24.132547+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.132547+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:23.773840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:45.028278+0000","last_change":"2024-03-28T16:51:45.028430+0000","last_active":"2024-03-28T16:51:45.028278+0000","last_peered":"2024-03-28T16:51:45.028278+0000","last_clean":"2024-03-28T16:51:45.028278+0000","last_became_active":"2024-03-28T16:48:54.669652+0000","last_became_peered":"2024-03-28T16:48:54.669652+0000","last_unstale":"2024-03-28T16:51:45.028278+0000","last_undegraded":"2024-03-28T16:51:45.028278+0000","last_fullsized":"2024-03-28T16:51:45.028278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:45.028042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:45.028042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:45.028042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:31:06.107975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.138003+0000","last_change":"2024-03-28T16:50:50.138003+0000","last_active":"2024-03-28T16:50:50.138003+0000","last_peered":"2024-03-28T16:50:50.138003+0000","last_clean":"2024-03-28T16:50:50.138003+0000","last_became_active":"2024-03-28T16:48:58.699304+0000","last_became_peered":"2024-03-28T16:48:58.699304+0000","last_unstale":"2024-03-28T16:50:50.138003+0000","last_undegraded":"2024-03-28T16:50:50.138003+0000","last_fullsized":"2024-03-28T16:50:50.138003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.137765+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.137765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:11.908122+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.226657+0000","last_change":"2024-03-28T16:51:10.226657+0000","last_active":"2024-03-28T16:51:10.226657+0000","last_peered":"2024-03-28T16:51:10.226657+0000","last_clean":"2024-03-28T16:51:10.226657+0000","last_became_active":"2024-03-28T16:49:00.724726+0000","last_became_peered":"2024-03-28T16:49:00.724726+0000","last_unstale":"2024-03-28T16:51:10.226657+0000","last_undegraded":"2024-03-28T16:51:10.226657+0000","last_fullsized":"2024-03-28T16:51:10.226657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:10.226407+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.226407+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:53:01.054784+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.251369+0000","last_change":"2024-03-28T16:50:57.251369+0000","last_active":"2024-03-28T16:50:57.251369+0000","last_peered":"2024-03-28T16:50:57.251369+0000","last_clean":"2024-03-28T16:50:57.251369+0000","last_became_active":"2024-03-28T16:49:02.738720+0000","last_became_peered":"2024-03-28T16:49:02.738720+0000","last_unstale":"2024-03-28T16:50:57.251369+0000","last_undegraded":"2024-03-28T16:50:57.251369+0000","last_fullsized":"2024-03-28T16:50:57.251369+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.251130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.251130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:45.232439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"31'7","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.077837+0000","last_change":"2024-03-28T16:51:41.077908+0000","last_active":"2024-03-28T16:51:41.077837+0000","last_peered":"2024-03-28T16:51:41.077837+0000","last_clean":"2024-03-28T16:51:41.077837+0000","last_became_active":"2024-03-28T16:49:04.742250+0000","last_became_peered":"2024-03-28T16:49:04.742250+0000","last_unstale":"2024-03-28T16:51:41.077837+0000","last_undegraded":"2024-03-28T16:51:41.077837+0000","last_fullsized":"2024-03-28T16:51:41.077837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'7","last_scrub_stamp":"2024-03-28T16:51:41.077657+0000","last_deep_scrub":"31'7","last_deep_scrub_stamp":"2024-03-28T16:51:41.077657+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.077657+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:33.086744+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"31'10","reported_seq":75,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.082932+0000","last_change":"2024-03-28T16:51:34.082995+0000","last_active":"2024-03-28T16:51:34.082932+0000","last_peered":"2024-03-28T16:51:34.082932+0000","last_clean":"2024-03-28T16:51:34.082932+0000","last_became_active":"2024-03-28T16:48:56.670575+0000","last_became_peered":"2024-03-28T16:48:56.670575+0000","last_unstale":"2024-03-28T16:51:34.082932+0000","last_undegraded":"2024-03-28T16:51:34.082932+0000","last_fullsized":"2024-03-28T16:51:34.082932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:34.082730+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:34.082730+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.082730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:12:07.452135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.286192+0000","last_change":"2024-03-28T16:51:27.286244+0000","last_active":"2024-03-28T16:51:27.286192+0000","last_peered":"2024-03-28T16:51:27.286192+0000","last_clean":"2024-03-28T16:51:27.286192+0000","last_became_active":"2024-03-28T16:48:54.655407+0000","last_became_peered":"2024-03-28T16:48:54.655407+0000","last_unstale":"2024-03-28T16:51:27.286192+0000","last_undegraded":"2024-03-28T16:51:27.286192+0000","last_fullsized":"2024-03-28T16:51:27.286192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:27.286005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:27.286005+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.286005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:56:00.366383+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.328962+0000","last_change":"2024-03-28T16:51:04.328962+0000","last_active":"2024-03-28T16:51:04.328962+0000","last_peered":"2024-03-28T16:51:04.328962+0000","last_clean":"2024-03-28T16:51:04.328962+0000","last_became_active":"2024-03-28T16:48:58.688497+0000","last_became_peered":"2024-03-28T16:48:58.688497+0000","last_unstale":"2024-03-28T16:51:04.328962+0000","last_undegraded":"2024-03-28T16:51:04.328962+0000","last_fullsized":"2024-03-28T16:51:04.328962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.328732+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.328732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:30.892979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"31'10","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.284276+0000","last_change":"2024-03-28T16:50:55.284276+0000","last_active":"2024-03-28T16:50:55.284276+0000","last_peered":"2024-03-28T16:50:55.284276+0000","last_clean":"2024-03-28T16:50:55.284276+0000","last_became_active":"2024-03-28T16:49:00.715923+0000","last_became_peered":"2024-03-28T16:49:00.715923+0000","last_unstale":"2024-03-28T16:50:55.284276+0000","last_undegraded":"2024-03-28T16:50:55.284276+0000","last_fullsized":"2024-03-28T16:50:55.284276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:50:55.283889+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.283889+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:07.952223+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.156607+0000","last_change":"2024-03-28T16:50:49.156607+0000","last_active":"2024-03-28T16:50:49.156607+0000","last_peered":"2024-03-28T16:50:49.156607+0000","last_clean":"2024-03-28T16:50:49.156607+0000","last_became_active":"2024-03-28T16:49:02.729550+0000","last_became_peered":"2024-03-28T16:49:02.729550+0000","last_unstale":"2024-03-28T16:50:49.156607+0000","last_undegraded":"2024-03-28T16:50:49.156607+0000","last_fullsized":"2024-03-28T16:50:49.156607+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.156374+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.156374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:19.754691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:46.532606+0000","last_change":"2024-03-28T16:51:46.532668+0000","last_active":"2024-03-28T16:51:46.532606+0000","last_peered":"2024-03-28T16:51:46.532606+0000","last_clean":"2024-03-28T16:51:46.532606+0000","last_became_active":"2024-03-28T16:49:04.742308+0000","last_became_peered":"2024-03-28T16:49:04.742308+0000","last_unstale":"2024-03-28T16:51:46.532606+0000","last_undegraded":"2024-03-28T16:51:46.532606+0000","last_fullsized":"2024-03-28T16:51:46.532606+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:46.532395+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:46.532395+0000","last_clean_scrub_stamp":"2024-03-28T16:51:46.532395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:10.071486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"31'8","reported_seq":68,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.147922+0000","last_change":"2024-03-28T16:51:30.147995+0000","last_active":"2024-03-28T16:51:30.147922+0000","last_peered":"2024-03-28T16:51:30.147922+0000","last_clean":"2024-03-28T16:51:30.147922+0000","last_became_active":"2024-03-28T16:48:56.675685+0000","last_became_peered":"2024-03-28T16:48:56.675685+0000","last_unstale":"2024-03-28T16:51:30.147922+0000","last_undegraded":"2024-03-28T16:51:30.147922+0000","last_fullsized":"2024-03-28T16:51:30.147922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:30.147724+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:30.147724+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.147724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:42:47.609297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:54.069947+0000","last_change":"2024-03-28T16:51:54.069999+0000","last_active":"2024-03-28T16:51:54.069947+0000","last_peered":"2024-03-28T16:51:54.069947+0000","last_clean":"2024-03-28T16:51:54.069947+0000","last_became_active":"2024-03-28T16:48:54.658931+0000","last_became_peered":"2024-03-28T16:48:54.658931+0000","last_unstale":"2024-03-28T16:51:54.069947+0000","last_undegraded":"2024-03-28T16:51:54.069947+0000","last_fullsized":"2024-03-28T16:51:54.069947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:54.069722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:54.069722+0000","last_clean_scrub_stamp":"2024-03-28T16:51:54.069722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:51:26.808328+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"31'6","reported_seq":216,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.337837+0000","last_change":"2024-03-28T16:51:08.337837+0000","last_active":"2024-03-28T16:51:08.337837+0000","last_peered":"2024-03-28T16:51:08.337837+0000","last_clean":"2024-03-28T16:51:08.337837+0000","last_became_active":"2024-03-28T16:48:58.686288+0000","last_became_peered":"2024-03-28T16:48:58.686288+0000","last_unstale":"2024-03-28T16:51:08.337837+0000","last_undegraded":"2024-03-28T16:51:08.337837+0000","last_fullsized":"2024-03-28T16:51:08.337837+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:08.337418+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.337418+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:50.313341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":921,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":270,"num_read_kb":269,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"31'16","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.357686+0000","last_change":"2024-03-28T16:50:59.357686+0000","last_active":"2024-03-28T16:50:59.357686+0000","last_peered":"2024-03-28T16:50:59.357686+0000","last_clean":"2024-03-28T16:50:59.357686+0000","last_became_active":"2024-03-28T16:49:00.702201+0000","last_became_peered":"2024-03-28T16:49:00.702201+0000","last_unstale":"2024-03-28T16:50:59.357686+0000","last_undegraded":"2024-03-28T16:50:59.357686+0000","last_fullsized":"2024-03-28T16:50:59.357686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'16","last_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_deep_scrub":"31'16","last_deep_scrub_stamp":"2024-03-28T16:50:59.357393+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.357393+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:54.820495+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"31'249","reported_seq":342,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.231850+0000","last_change":"2024-03-28T16:51:08.231850+0000","last_active":"2024-03-28T16:51:08.231850+0000","last_peered":"2024-03-28T16:51:08.231850+0000","last_clean":"2024-03-28T16:51:08.231850+0000","last_became_active":"2024-03-28T16:49:02.736982+0000","last_became_peered":"2024-03-28T16:49:02.736982+0000","last_unstale":"2024-03-28T16:51:08.231850+0000","last_undegraded":"2024-03-28T16:51:08.231850+0000","last_fullsized":"2024-03-28T16:51:08.231850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:08.231583+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.231583+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:26.966593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.350955+0000","last_change":"2024-03-28T16:51:04.350955+0000","last_active":"2024-03-28T16:51:04.350955+0000","last_peered":"2024-03-28T16:51:04.350955+0000","last_clean":"2024-03-28T16:51:04.350955+0000","last_became_active":"2024-03-28T16:49:04.739945+0000","last_became_peered":"2024-03-28T16:49:04.739945+0000","last_unstale":"2024-03-28T16:51:04.350955+0000","last_undegraded":"2024-03-28T16:51:04.350955+0000","last_fullsized":"2024-03-28T16:51:04.350955+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.350588+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.350588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:52.331897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.382333+0000","last_change":"2024-03-28T16:51:14.382333+0000","last_active":"2024-03-28T16:51:14.382333+0000","last_peered":"2024-03-28T16:51:14.382333+0000","last_clean":"2024-03-28T16:51:14.382333+0000","last_became_active":"2024-03-28T16:48:56.697045+0000","last_became_peered":"2024-03-28T16:48:56.697045+0000","last_unstale":"2024-03-28T16:51:14.382333+0000","last_undegraded":"2024-03-28T16:51:14.382333+0000","last_fullsized":"2024-03-28T16:51:14.382333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:14.382083+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.382083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:28.408578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.082705+0000","last_change":"2024-03-28T16:51:35.082806+0000","last_active":"2024-03-28T16:51:35.082705+0000","last_peered":"2024-03-28T16:51:35.082705+0000","last_clean":"2024-03-28T16:51:35.082705+0000","last_became_active":"2024-03-28T16:48:54.658659+0000","last_became_peered":"2024-03-28T16:48:54.658659+0000","last_unstale":"2024-03-28T16:51:35.082705+0000","last_undegraded":"2024-03-28T16:51:35.082705+0000","last_fullsized":"2024-03-28T16:51:35.082705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:35.082491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:35.082491+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.082491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:11.308208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.124265+0000","last_change":"2024-03-28T16:50:49.124265+0000","last_active":"2024-03-28T16:50:49.124265+0000","last_peered":"2024-03-28T16:50:49.124265+0000","last_clean":"2024-03-28T16:50:49.124265+0000","last_became_active":"2024-03-28T16:48:58.706198+0000","last_became_peered":"2024-03-28T16:48:58.706198+0000","last_unstale":"2024-03-28T16:50:49.124265+0000","last_undegraded":"2024-03-28T16:50:49.124265+0000","last_fullsized":"2024-03-28T16:50:49.124265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.124014+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.124014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.365150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.425199+0000","last_change":"2024-03-28T16:50:57.425199+0000","last_active":"2024-03-28T16:50:57.425199+0000","last_peered":"2024-03-28T16:50:57.425199+0000","last_clean":"2024-03-28T16:50:57.425199+0000","last_became_active":"2024-03-28T16:49:00.710542+0000","last_became_peered":"2024-03-28T16:49:00.710542+0000","last_unstale":"2024-03-28T16:50:57.425199+0000","last_undegraded":"2024-03-28T16:50:57.425199+0000","last_fullsized":"2024-03-28T16:50:57.425199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:57.424946+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.424946+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:19.309481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"31'249","reported_seq":357,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.296501+0000","last_change":"2024-03-28T16:51:09.296501+0000","last_active":"2024-03-28T16:51:09.296501+0000","last_peered":"2024-03-28T16:51:09.296501+0000","last_clean":"2024-03-28T16:51:09.296501+0000","last_became_active":"2024-03-28T16:49:02.733749+0000","last_became_peered":"2024-03-28T16:49:02.733749+0000","last_unstale":"2024-03-28T16:51:09.296501+0000","last_undegraded":"2024-03-28T16:51:09.296501+0000","last_fullsized":"2024-03-28T16:51:09.296501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:51:09.296245+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.296245+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:57.394089+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.285771+0000","last_change":"2024-03-28T16:51:06.285771+0000","last_active":"2024-03-28T16:51:06.285771+0000","last_peered":"2024-03-28T16:51:06.285771+0000","last_clean":"2024-03-28T16:51:06.285771+0000","last_became_active":"2024-03-28T16:48:56.693008+0000","last_became_peered":"2024-03-28T16:48:56.693008+0000","last_unstale":"2024-03-28T16:51:06.285771+0000","last_undegraded":"2024-03-28T16:51:06.285771+0000","last_fullsized":"2024-03-28T16:51:06.285771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.285504+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.285504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:11.465640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.389203+0000","last_change":"2024-03-28T16:51:17.389264+0000","last_active":"2024-03-28T16:51:17.389203+0000","last_peered":"2024-03-28T16:51:17.389203+0000","last_clean":"2024-03-28T16:51:17.389203+0000","last_became_active":"2024-03-28T16:48:54.671640+0000","last_became_peered":"2024-03-28T16:48:54.671640+0000","last_unstale":"2024-03-28T16:51:17.389203+0000","last_undegraded":"2024-03-28T16:51:17.389203+0000","last_fullsized":"2024-03-28T16:51:17.389203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.389016+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.389016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:59.364938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.302414+0000","last_change":"2024-03-28T16:51:00.302414+0000","last_active":"2024-03-28T16:51:00.302414+0000","last_peered":"2024-03-28T16:51:00.302414+0000","last_clean":"2024-03-28T16:51:00.302414+0000","last_became_active":"2024-03-28T16:48:58.687174+0000","last_became_peered":"2024-03-28T16:48:58.687174+0000","last_unstale":"2024-03-28T16:51:00.302414+0000","last_undegraded":"2024-03-28T16:51:00.302414+0000","last_fullsized":"2024-03-28T16:51:00.302414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.302183+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.302183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:46.487988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"31'49","reported_seq":109,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.444852+0000","last_change":"2024-03-28T16:50:55.444852+0000","last_active":"2024-03-28T16:50:55.444852+0000","last_peered":"2024-03-28T16:50:55.444852+0000","last_clean":"2024-03-28T16:50:55.444852+0000","last_became_active":"2024-03-28T16:49:00.712610+0000","last_became_peered":"2024-03-28T16:49:00.712610+0000","last_unstale":"2024-03-28T16:50:55.444852+0000","last_undegraded":"2024-03-28T16:50:55.444852+0000","last_fullsized":"2024-03-28T16:50:55.444852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'49","last_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_deep_scrub":"31'49","last_deep_scrub_stamp":"2024-03-28T16:50:55.444545+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.444545+0000","objects_scrubbed":9,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:56.707599+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.195172+0000","last_change":"2024-03-28T16:50:53.195172+0000","last_active":"2024-03-28T16:50:53.195172+0000","last_peered":"2024-03-28T16:50:53.195172+0000","last_clean":"2024-03-28T16:50:53.195172+0000","last_became_active":"2024-03-28T16:49:02.729981+0000","last_became_peered":"2024-03-28T16:49:02.729981+0000","last_unstale":"2024-03-28T16:50:53.195172+0000","last_undegraded":"2024-03-28T16:50:53.195172+0000","last_fullsized":"2024-03-28T16:50:53.195172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.194894+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.194894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:05.863228+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"31'3","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.397864+0000","last_change":"2024-03-28T16:50:56.397864+0000","last_active":"2024-03-28T16:50:56.397864+0000","last_peered":"2024-03-28T16:50:56.397864+0000","last_clean":"2024-03-28T16:50:56.397864+0000","last_became_active":"2024-03-28T16:49:04.727564+0000","last_became_peered":"2024-03-28T16:49:04.727564+0000","last_unstale":"2024-03-28T16:50:56.397864+0000","last_undegraded":"2024-03-28T16:50:56.397864+0000","last_fullsized":"2024-03-28T16:50:56.397864+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:56.397465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.397465+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:08:26.995782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'24","reported_seq":112,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.267353+0000","last_change":"2024-03-28T16:51:17.267421+0000","last_active":"2024-03-28T16:51:17.267353+0000","last_peered":"2024-03-28T16:51:17.267353+0000","last_clean":"2024-03-28T16:51:17.267353+0000","last_became_active":"2024-03-28T16:48:56.677507+0000","last_became_peered":"2024-03-28T16:48:56.677507+0000","last_unstale":"2024-03-28T16:51:17.267353+0000","last_undegraded":"2024-03-28T16:51:17.267353+0000","last_fullsized":"2024-03-28T16:51:17.267353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'24","last_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_deep_scrub":"31'24","last_deep_scrub_stamp":"2024-03-28T16:51:17.267112+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.267112+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:31.293607+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":98,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.358051+0000","last_change":"2024-03-28T16:51:15.358174+0000","last_active":"2024-03-28T16:51:15.358051+0000","last_peered":"2024-03-28T16:51:15.358051+0000","last_clean":"2024-03-28T16:51:15.358051+0000","last_became_active":"2024-03-28T16:48:54.671935+0000","last_became_peered":"2024-03-28T16:48:54.671935+0000","last_unstale":"2024-03-28T16:51:15.358051+0000","last_undegraded":"2024-03-28T16:51:15.358051+0000","last_fullsized":"2024-03-28T16:51:15.358051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:15.357818+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.357818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:41.892555+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.371282+0000","last_change":"2024-03-28T16:51:02.371282+0000","last_active":"2024-03-28T16:51:02.371282+0000","last_peered":"2024-03-28T16:51:02.371282+0000","last_clean":"2024-03-28T16:51:02.371282+0000","last_became_active":"2024-03-28T16:48:58.689576+0000","last_became_peered":"2024-03-28T16:48:58.689576+0000","last_unstale":"2024-03-28T16:51:02.371282+0000","last_undegraded":"2024-03-28T16:51:02.371282+0000","last_fullsized":"2024-03-28T16:51:02.371282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.371035+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.371035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:20.148230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.191604+0000","last_change":"2024-03-28T16:51:11.191604+0000","last_active":"2024-03-28T16:51:11.191604+0000","last_peered":"2024-03-28T16:51:11.191604+0000","last_clean":"2024-03-28T16:51:11.191604+0000","last_became_active":"2024-03-28T16:49:00.726565+0000","last_became_peered":"2024-03-28T16:49:00.726565+0000","last_unstale":"2024-03-28T16:51:11.191604+0000","last_undegraded":"2024-03-28T16:51:11.191604+0000","last_fullsized":"2024-03-28T16:51:11.191604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:11.191343+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.191343+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:47:25.401107+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.183048+0000","last_change":"2024-03-28T16:50:51.183048+0000","last_active":"2024-03-28T16:50:51.183048+0000","last_peered":"2024-03-28T16:50:51.183048+0000","last_clean":"2024-03-28T16:50:51.183048+0000","last_became_active":"2024-03-28T16:49:02.712736+0000","last_became_peered":"2024-03-28T16:49:02.712736+0000","last_unstale":"2024-03-28T16:50:51.183048+0000","last_undegraded":"2024-03-28T16:50:51.183048+0000","last_fullsized":"2024-03-28T16:50:51.183048+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.182678+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.182678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:48.071769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"31'4","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.320355+0000","last_change":"2024-03-28T16:51:05.320355+0000","last_active":"2024-03-28T16:51:05.320355+0000","last_peered":"2024-03-28T16:51:05.320355+0000","last_clean":"2024-03-28T16:51:05.320355+0000","last_became_active":"2024-03-28T16:49:04.737728+0000","last_became_peered":"2024-03-28T16:49:04.737728+0000","last_unstale":"2024-03-28T16:51:05.320355+0000","last_undegraded":"2024-03-28T16:51:05.320355+0000","last_fullsized":"2024-03-28T16:51:05.320355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:05.320076+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.320076+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:34.995213+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"31'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.314961+0000","last_change":"2024-03-28T16:51:14.315025+0000","last_active":"2024-03-28T16:51:14.314961+0000","last_peered":"2024-03-28T16:51:14.314961+0000","last_clean":"2024-03-28T16:51:14.314961+0000","last_became_active":"2024-03-28T16:48:56.680734+0000","last_became_peered":"2024-03-28T16:48:56.680734+0000","last_unstale":"2024-03-28T16:51:14.314961+0000","last_undegraded":"2024-03-28T16:51:14.314961+0000","last_fullsized":"2024-03-28T16:51:14.314961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:14.314772+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.314772+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:49.109332+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.126840+0000","last_change":"2024-03-28T16:51:38.126897+0000","last_active":"2024-03-28T16:51:38.126840+0000","last_peered":"2024-03-28T16:51:38.126840+0000","last_clean":"2024-03-28T16:51:38.126840+0000","last_became_active":"2024-03-28T16:48:54.676051+0000","last_became_peered":"2024-03-28T16:48:54.676051+0000","last_unstale":"2024-03-28T16:51:38.126840+0000","last_undegraded":"2024-03-28T16:51:38.126840+0000","last_fullsized":"2024-03-28T16:51:38.126840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.126656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.126656+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.126656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:49.967866+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:11.352892+0000","last_change":"2024-03-28T16:51:11.352892+0000","last_active":"2024-03-28T16:51:11.352892+0000","last_peered":"2024-03-28T16:51:11.352892+0000","last_clean":"2024-03-28T16:51:11.352892+0000","last_became_active":"2024-03-28T16:48:58.681621+0000","last_became_peered":"2024-03-28T16:48:58.681621+0000","last_unstale":"2024-03-28T16:51:11.352892+0000","last_undegraded":"2024-03-28T16:51:11.352892+0000","last_fullsized":"2024-03-28T16:51:11.352892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:11.352650+0000","last_clean_scrub_stamp":"2024-03-28T16:51:11.352650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:16.271950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.096269+0000","last_change":"2024-03-28T16:51:18.096365+0000","last_active":"2024-03-28T16:51:18.096269+0000","last_peered":"2024-03-28T16:51:18.096269+0000","last_clean":"2024-03-28T16:51:18.096269+0000","last_became_active":"2024-03-28T16:49:00.720095+0000","last_became_peered":"2024-03-28T16:49:00.720095+0000","last_unstale":"2024-03-28T16:51:18.096269+0000","last_undegraded":"2024-03-28T16:51:18.096269+0000","last_fullsized":"2024-03-28T16:51:18.096269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:18.096048+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.096048+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:25:10.105136+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"31'249","reported_seq":352,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.171801+0000","last_change":"2024-03-28T16:50:52.171801+0000","last_active":"2024-03-28T16:50:52.171801+0000","last_peered":"2024-03-28T16:50:52.171801+0000","last_clean":"2024-03-28T16:50:52.171801+0000","last_became_active":"2024-03-28T16:49:02.712729+0000","last_became_peered":"2024-03-28T16:49:02.712729+0000","last_unstale":"2024-03-28T16:50:52.171801+0000","last_undegraded":"2024-03-28T16:50:52.171801+0000","last_fullsized":"2024-03-28T16:50:52.171801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.171465+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.171465+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:19.659817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"31'2","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:45.557316+0000","last_change":"2024-03-28T16:51:45.557374+0000","last_active":"2024-03-28T16:51:45.557316+0000","last_peered":"2024-03-28T16:51:45.557316+0000","last_clean":"2024-03-28T16:51:45.557316+0000","last_became_active":"2024-03-28T16:49:04.742405+0000","last_became_peered":"2024-03-28T16:49:04.742405+0000","last_unstale":"2024-03-28T16:51:45.557316+0000","last_undegraded":"2024-03-28T16:51:45.557316+0000","last_fullsized":"2024-03-28T16:51:45.557316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:45.557138+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:45.557138+0000","last_clean_scrub_stamp":"2024-03-28T16:51:45.557138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:24:45.198431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"31'6","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.394496+0000","last_change":"2024-03-28T16:51:18.394559+0000","last_active":"2024-03-28T16:51:18.394496+0000","last_peered":"2024-03-28T16:51:18.394496+0000","last_clean":"2024-03-28T16:51:18.394496+0000","last_became_active":"2024-03-28T16:48:56.684687+0000","last_became_peered":"2024-03-28T16:48:56.684687+0000","last_unstale":"2024-03-28T16:51:18.394496+0000","last_undegraded":"2024-03-28T16:51:18.394496+0000","last_fullsized":"2024-03-28T16:51:18.394496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:18.394309+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.394309+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:06:06.492152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:01.981153+0000","last_change":"2024-03-28T16:52:01.981221+0000","last_active":"2024-03-28T16:52:01.981153+0000","last_peered":"2024-03-28T16:52:01.981153+0000","last_clean":"2024-03-28T16:52:01.981153+0000","last_became_active":"2024-03-28T16:48:54.657697+0000","last_became_peered":"2024-03-28T16:48:54.657697+0000","last_unstale":"2024-03-28T16:52:01.981153+0000","last_undegraded":"2024-03-28T16:52:01.981153+0000","last_fullsized":"2024-03-28T16:52:01.981153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:52:01.980962+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:52:01.980962+0000","last_clean_scrub_stamp":"2024-03-28T16:52:01.980962+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:45:01.171370+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.386277+0000","last_change":"2024-03-28T16:51:12.386390+0000","last_active":"2024-03-28T16:51:12.386277+0000","last_peered":"2024-03-28T16:51:12.386277+0000","last_clean":"2024-03-28T16:51:12.386277+0000","last_became_active":"2024-03-28T16:48:58.681426+0000","last_became_peered":"2024-03-28T16:48:58.681426+0000","last_unstale":"2024-03-28T16:51:12.386277+0000","last_undegraded":"2024-03-28T16:51:12.386277+0000","last_fullsized":"2024-03-28T16:51:12.386277+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:12.386042+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.386042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:21.605218+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"31'61","reported_seq":134,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.103088+0000","last_change":"2024-03-28T16:51:14.103172+0000","last_active":"2024-03-28T16:51:14.103088+0000","last_peered":"2024-03-28T16:51:14.103088+0000","last_clean":"2024-03-28T16:51:14.103088+0000","last_became_active":"2024-03-28T16:49:00.725145+0000","last_became_peered":"2024-03-28T16:49:00.725145+0000","last_unstale":"2024-03-28T16:51:14.103088+0000","last_undegraded":"2024-03-28T16:51:14.103088+0000","last_fullsized":"2024-03-28T16:51:14.103088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'61","last_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_deep_scrub":"31'61","last_deep_scrub_stamp":"2024-03-28T16:51:14.102888+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.102888+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:13:35.860974+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.181042+0000","last_change":"2024-03-28T16:50:54.181042+0000","last_active":"2024-03-28T16:50:54.181042+0000","last_peered":"2024-03-28T16:50:54.181042+0000","last_clean":"2024-03-28T16:50:54.181042+0000","last_became_active":"2024-03-28T16:49:02.711308+0000","last_became_peered":"2024-03-28T16:49:02.711308+0000","last_unstale":"2024-03-28T16:50:54.181042+0000","last_undegraded":"2024-03-28T16:50:54.181042+0000","last_fullsized":"2024-03-28T16:50:54.181042+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.180810+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.180810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:17.080003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"31'4","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.393450+0000","last_change":"2024-03-28T16:51:13.393450+0000","last_active":"2024-03-28T16:51:13.393450+0000","last_peered":"2024-03-28T16:51:13.393450+0000","last_clean":"2024-03-28T16:51:13.393450+0000","last_became_active":"2024-03-28T16:49:04.722533+0000","last_became_peered":"2024-03-28T16:49:04.722533+0000","last_unstale":"2024-03-28T16:51:13.393450+0000","last_undegraded":"2024-03-28T16:51:13.393450+0000","last_fullsized":"2024-03-28T16:51:13.393450+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:13.393212+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.393212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:22.165871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"31'8","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.203924+0000","last_change":"2024-03-28T16:51:27.203985+0000","last_active":"2024-03-28T16:51:27.203924+0000","last_peered":"2024-03-28T16:51:27.203924+0000","last_clean":"2024-03-28T16:51:27.203924+0000","last_became_active":"2024-03-28T16:48:56.681286+0000","last_became_peered":"2024-03-28T16:48:56.681286+0000","last_unstale":"2024-03-28T16:51:27.203924+0000","last_undegraded":"2024-03-28T16:51:27.203924+0000","last_fullsized":"2024-03-28T16:51:27.203924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:27.203737+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:27.203737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.203737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:24:14.186013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:28.551323+0000","last_change":"2024-03-28T16:51:28.551377+0000","last_active":"2024-03-28T16:51:28.551323+0000","last_peered":"2024-03-28T16:51:28.551323+0000","last_clean":"2024-03-28T16:51:28.551323+0000","last_became_active":"2024-03-28T16:48:54.664245+0000","last_became_peered":"2024-03-28T16:48:54.664245+0000","last_unstale":"2024-03-28T16:51:28.551323+0000","last_undegraded":"2024-03-28T16:51:28.551323+0000","last_fullsized":"2024-03-28T16:51:28.551323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:28.551113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:28.551113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:28.551113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:24.518811+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.365035+0000","last_change":"2024-03-28T16:51:05.365035+0000","last_active":"2024-03-28T16:51:05.365035+0000","last_peered":"2024-03-28T16:51:05.365035+0000","last_clean":"2024-03-28T16:51:05.365035+0000","last_became_active":"2024-03-28T16:48:58.690938+0000","last_became_peered":"2024-03-28T16:48:58.690938+0000","last_unstale":"2024-03-28T16:51:05.365035+0000","last_undegraded":"2024-03-28T16:51:05.365035+0000","last_fullsized":"2024-03-28T16:51:05.365035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.364765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.364765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:50.121796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"31'37","reported_seq":93,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.221214+0000","last_change":"2024-03-28T16:50:51.221214+0000","last_active":"2024-03-28T16:50:51.221214+0000","last_peered":"2024-03-28T16:50:51.221214+0000","last_clean":"2024-03-28T16:50:51.221214+0000","last_became_active":"2024-03-28T16:49:00.721128+0000","last_became_peered":"2024-03-28T16:49:00.721128+0000","last_unstale":"2024-03-28T16:50:51.221214+0000","last_undegraded":"2024-03-28T16:50:51.221214+0000","last_fullsized":"2024-03-28T16:50:51.221214+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'37","last_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_deep_scrub":"31'37","last_deep_scrub_stamp":"2024-03-28T16:50:51.220960+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.220960+0000","objects_scrubbed":5,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:47.323997+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.275109+0000","last_change":"2024-03-28T16:51:07.275109+0000","last_active":"2024-03-28T16:51:07.275109+0000","last_peered":"2024-03-28T16:51:07.275109+0000","last_clean":"2024-03-28T16:51:07.275109+0000","last_became_active":"2024-03-28T16:49:02.737217+0000","last_became_peered":"2024-03-28T16:49:02.737217+0000","last_unstale":"2024-03-28T16:51:07.275109+0000","last_undegraded":"2024-03-28T16:51:07.275109+0000","last_fullsized":"2024-03-28T16:51:07.275109+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.274868+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.274868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:51:03.242566+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.076599+0000","last_change":"2024-03-28T16:51:37.076651+0000","last_active":"2024-03-28T16:51:37.076599+0000","last_peered":"2024-03-28T16:51:37.076599+0000","last_clean":"2024-03-28T16:51:37.076599+0000","last_became_active":"2024-03-28T16:49:04.747952+0000","last_became_peered":"2024-03-28T16:49:04.747952+0000","last_unstale":"2024-03-28T16:51:37.076599+0000","last_undegraded":"2024-03-28T16:51:37.076599+0000","last_fullsized":"2024-03-28T16:51:37.076599+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.076411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.076411+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.076411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:35:47.198210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.193692+0000","last_change":"2024-03-28T16:51:00.193692+0000","last_active":"2024-03-28T16:51:00.193692+0000","last_peered":"2024-03-28T16:51:00.193692+0000","last_clean":"2024-03-28T16:51:00.193692+0000","last_became_active":"2024-03-28T16:48:56.695439+0000","last_became_peered":"2024-03-28T16:48:56.695439+0000","last_unstale":"2024-03-28T16:51:00.193692+0000","last_undegraded":"2024-03-28T16:51:00.193692+0000","last_fullsized":"2024-03-28T16:51:00.193692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.193439+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.193439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:35.074661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"29'2","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:49.006497+0000","last_change":"2024-03-28T16:51:49.006621+0000","last_active":"2024-03-28T16:51:49.006497+0000","last_peered":"2024-03-28T16:51:49.006497+0000","last_clean":"2024-03-28T16:51:49.006497+0000","last_became_active":"2024-03-28T16:48:54.672858+0000","last_became_peered":"2024-03-28T16:48:54.672858+0000","last_unstale":"2024-03-28T16:51:49.006497+0000","last_undegraded":"2024-03-28T16:51:49.006497+0000","last_fullsized":"2024-03-28T16:51:49.006497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:49.006205+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:49.006205+0000","last_clean_scrub_stamp":"2024-03-28T16:51:49.006205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:42.130598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.345768+0000","last_change":"2024-03-28T16:51:10.345768+0000","last_active":"2024-03-28T16:51:10.345768+0000","last_peered":"2024-03-28T16:51:10.345768+0000","last_clean":"2024-03-28T16:51:10.345768+0000","last_became_active":"2024-03-28T16:48:58.683116+0000","last_became_peered":"2024-03-28T16:48:58.683116+0000","last_unstale":"2024-03-28T16:51:10.345768+0000","last_undegraded":"2024-03-28T16:51:10.345768+0000","last_fullsized":"2024-03-28T16:51:10.345768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:10.345522+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.345522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:45.930075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"31'44","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.389452+0000","last_change":"2024-03-28T16:51:01.389452+0000","last_active":"2024-03-28T16:51:01.389452+0000","last_peered":"2024-03-28T16:51:01.389452+0000","last_clean":"2024-03-28T16:51:01.389452+0000","last_became_active":"2024-03-28T16:49:00.697239+0000","last_became_peered":"2024-03-28T16:49:00.697239+0000","last_unstale":"2024-03-28T16:51:01.389452+0000","last_undegraded":"2024-03-28T16:51:01.389452+0000","last_fullsized":"2024-03-28T16:51:01.389452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'44","last_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_deep_scrub":"31'44","last_deep_scrub_stamp":"2024-03-28T16:51:01.389173+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.389173+0000","objects_scrubbed":7,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:28.877525+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"31'249","reported_seq":335,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.428834+0000","last_change":"2024-03-28T16:50:47.428834+0000","last_active":"2024-03-28T16:50:47.428834+0000","last_peered":"2024-03-28T16:50:47.428834+0000","last_clean":"2024-03-28T16:50:47.428834+0000","last_became_active":"2024-03-28T16:49:02.720287+0000","last_became_peered":"2024-03-28T16:49:02.720287+0000","last_unstale":"2024-03-28T16:50:47.428834+0000","last_undegraded":"2024-03-28T16:50:47.428834+0000","last_fullsized":"2024-03-28T16:50:47.428834+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:47.428456+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.428456+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:01.404074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"31'14","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:47.490733+0000","last_change":"2024-03-28T16:51:47.490803+0000","last_active":"2024-03-28T16:51:47.490733+0000","last_peered":"2024-03-28T16:51:47.490733+0000","last_clean":"2024-03-28T16:51:47.490733+0000","last_became_active":"2024-03-28T16:49:04.742490+0000","last_became_peered":"2024-03-28T16:49:04.742490+0000","last_unstale":"2024-03-28T16:51:47.490733+0000","last_undegraded":"2024-03-28T16:51:47.490733+0000","last_fullsized":"2024-03-28T16:51:47.490733+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'14","last_scrub_stamp":"2024-03-28T16:51:47.490545+0000","last_deep_scrub":"31'14","last_deep_scrub_stamp":"2024-03-28T16:51:47.490545+0000","last_clean_scrub_stamp":"2024-03-28T16:51:47.490545+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:29.772619+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":170,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.337241+0000","last_change":"2024-03-28T16:51:22.337362+0000","last_active":"2024-03-28T16:51:22.337241+0000","last_peered":"2024-03-28T16:51:22.337241+0000","last_clean":"2024-03-28T16:51:22.337241+0000","last_became_active":"2024-03-28T16:48:56.689837+0000","last_became_peered":"2024-03-28T16:48:56.689837+0000","last_unstale":"2024-03-28T16:51:22.337241+0000","last_undegraded":"2024-03-28T16:51:22.337241+0000","last_fullsized":"2024-03-28T16:51:22.337241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.336956+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.336956+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:31.109615+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:44.014911+0000","last_change":"2024-03-28T16:51:44.014970+0000","last_active":"2024-03-28T16:51:44.014911+0000","last_peered":"2024-03-28T16:51:44.014911+0000","last_clean":"2024-03-28T16:51:44.014911+0000","last_became_active":"2024-03-28T16:48:54.664191+0000","last_became_peered":"2024-03-28T16:48:54.664191+0000","last_unstale":"2024-03-28T16:51:44.014911+0000","last_undegraded":"2024-03-28T16:51:44.014911+0000","last_fullsized":"2024-03-28T16:51:44.014911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:44.014723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:44.014723+0000","last_clean_scrub_stamp":"2024-03-28T16:51:44.014723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:58:04.858719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'5","reported_seq":394,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:46.100936+0000","last_change":"2024-03-28T16:51:46.101089+0000","last_active":"2024-03-28T16:51:46.100936+0000","last_peered":"2024-03-28T16:51:46.100936+0000","last_clean":"2024-03-28T16:51:46.100936+0000","last_became_active":"2024-03-28T16:49:37.611940+0000","last_became_peered":"2024-03-28T16:49:37.611940+0000","last_unstale":"2024-03-28T16:51:46.100936+0000","last_undegraded":"2024-03-28T16:51:46.100936+0000","last_fullsized":"2024-03-28T16:51:46.100936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'5","last_scrub_stamp":"2024-03-28T16:51:46.100668+0000","last_deep_scrub":"31'5","last_deep_scrub_stamp":"2024-03-28T16:51:46.100668+0000","last_clean_scrub_stamp":"2024-03-28T16:51:46.100668+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:54.818277+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":340,"num_read_kb":340,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.144955+0000","last_change":"2024-03-28T16:51:12.144955+0000","last_active":"2024-03-28T16:51:12.144955+0000","last_peered":"2024-03-28T16:51:12.144955+0000","last_clean":"2024-03-28T16:51:12.144955+0000","last_became_active":"2024-03-28T16:49:00.725209+0000","last_became_peered":"2024-03-28T16:49:00.725209+0000","last_unstale":"2024-03-28T16:51:12.144955+0000","last_undegraded":"2024-03-28T16:51:12.144955+0000","last_fullsized":"2024-03-28T16:51:12.144955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:12.144489+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.144489+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:29:36.042267+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"31'249","reported_seq":358,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.109690+0000","last_change":"2024-03-28T16:50:48.109690+0000","last_active":"2024-03-28T16:50:48.109690+0000","last_peered":"2024-03-28T16:50:48.109690+0000","last_clean":"2024-03-28T16:50:48.109690+0000","last_became_active":"2024-03-28T16:49:02.714348+0000","last_became_peered":"2024-03-28T16:49:02.714348+0000","last_unstale":"2024-03-28T16:50:48.109690+0000","last_undegraded":"2024-03-28T16:50:48.109690+0000","last_fullsized":"2024-03-28T16:50:48.109690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:48.109433+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.109433+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:32:49.372487+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:08.412356+0000","last_change":"2024-03-28T16:51:08.412356+0000","last_active":"2024-03-28T16:51:08.412356+0000","last_peered":"2024-03-28T16:51:08.412356+0000","last_clean":"2024-03-28T16:51:08.412356+0000","last_became_active":"2024-03-28T16:49:04.745846+0000","last_became_peered":"2024-03-28T16:49:04.745846+0000","last_unstale":"2024-03-28T16:51:08.412356+0000","last_undegraded":"2024-03-28T16:51:08.412356+0000","last_fullsized":"2024-03-28T16:51:08.412356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:08.412104+0000","last_clean_scrub_stamp":"2024-03-28T16:51:08.412104+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:37.251504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"31'6","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.269854+0000","last_change":"2024-03-28T16:51:25.269969+0000","last_active":"2024-03-28T16:51:25.269854+0000","last_peered":"2024-03-28T16:51:25.269854+0000","last_clean":"2024-03-28T16:51:25.269854+0000","last_became_active":"2024-03-28T16:48:56.680778+0000","last_became_peered":"2024-03-28T16:48:56.680778+0000","last_unstale":"2024-03-28T16:51:25.269854+0000","last_undegraded":"2024-03-28T16:51:25.269854+0000","last_fullsized":"2024-03-28T16:51:25.269854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:25.269642+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.269642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:53:12.659384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:03.003289+0000","last_change":"2024-03-28T16:52:03.003438+0000","last_active":"2024-03-28T16:52:03.003289+0000","last_peered":"2024-03-28T16:52:03.003289+0000","last_clean":"2024-03-28T16:52:03.003289+0000","last_became_active":"2024-03-28T16:48:54.676099+0000","last_became_peered":"2024-03-28T16:48:54.676099+0000","last_unstale":"2024-03-28T16:52:03.003289+0000","last_undegraded":"2024-03-28T16:52:03.003289+0000","last_fullsized":"2024-03-28T16:52:03.003289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:03.003033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:03.003033+0000","last_clean_scrub_stamp":"2024-03-28T16:52:03.003033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:49:50.012165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.165481+0000","last_change":"2024-03-28T16:50:48.165481+0000","last_active":"2024-03-28T16:50:48.165481+0000","last_peered":"2024-03-28T16:50:48.165481+0000","last_clean":"2024-03-28T16:50:48.165481+0000","last_became_active":"2024-03-28T16:48:58.703260+0000","last_became_peered":"2024-03-28T16:48:58.703260+0000","last_unstale":"2024-03-28T16:50:48.165481+0000","last_undegraded":"2024-03-28T16:50:48.165481+0000","last_fullsized":"2024-03-28T16:50:48.165481+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.165239+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.165239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:40:14.898173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"31'46","reported_seq":105,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.272646+0000","last_change":"2024-03-28T16:50:56.272646+0000","last_active":"2024-03-28T16:50:56.272646+0000","last_peered":"2024-03-28T16:50:56.272646+0000","last_clean":"2024-03-28T16:50:56.272646+0000","last_became_active":"2024-03-28T16:49:00.710953+0000","last_became_peered":"2024-03-28T16:49:00.710953+0000","last_unstale":"2024-03-28T16:50:56.272646+0000","last_undegraded":"2024-03-28T16:50:56.272646+0000","last_fullsized":"2024-03-28T16:50:56.272646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'46","last_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_deep_scrub":"31'46","last_deep_scrub_stamp":"2024-03-28T16:50:56.272353+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.272353+0000","objects_scrubbed":8,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:19.171546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.431537+0000","last_change":"2024-03-28T16:50:53.431537+0000","last_active":"2024-03-28T16:50:53.431537+0000","last_peered":"2024-03-28T16:50:53.431537+0000","last_clean":"2024-03-28T16:50:53.431537+0000","last_became_active":"2024-03-28T16:49:02.719672+0000","last_became_peered":"2024-03-28T16:49:02.719672+0000","last_unstale":"2024-03-28T16:50:53.431537+0000","last_undegraded":"2024-03-28T16:50:53.431537+0000","last_fullsized":"2024-03-28T16:50:53.431537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.431293+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.431293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:04:35.681957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"31'2","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.348998+0000","last_change":"2024-03-28T16:51:01.348998+0000","last_active":"2024-03-28T16:51:01.348998+0000","last_peered":"2024-03-28T16:51:01.348998+0000","last_clean":"2024-03-28T16:51:01.348998+0000","last_became_active":"2024-03-28T16:49:04.730049+0000","last_became_peered":"2024-03-28T16:49:04.730049+0000","last_unstale":"2024-03-28T16:51:01.348998+0000","last_undegraded":"2024-03-28T16:51:01.348998+0000","last_fullsized":"2024-03-28T16:51:01.348998+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:01.348719+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.348719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:30.188118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.197537+0000","last_change":"2024-03-28T16:51:03.197596+0000","last_active":"2024-03-28T16:51:03.197537+0000","last_peered":"2024-03-28T16:51:03.197537+0000","last_clean":"2024-03-28T16:51:03.197537+0000","last_became_active":"2024-03-28T16:48:56.693612+0000","last_became_peered":"2024-03-28T16:48:56.693612+0000","last_unstale":"2024-03-28T16:51:03.197537+0000","last_undegraded":"2024-03-28T16:51:03.197537+0000","last_fullsized":"2024-03-28T16:51:03.197537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:03.197346+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.197346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:04:49.417548+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"31'119","reported_seq":210,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.205423+0000","last_change":"2024-03-28T16:51:31.205497+0000","last_active":"2024-03-28T16:51:31.205423+0000","last_peered":"2024-03-28T16:51:31.205423+0000","last_clean":"2024-03-28T16:51:31.205423+0000","last_became_active":"2024-03-28T16:48:44.462729+0000","last_became_peered":"2024-03-28T16:48:44.462729+0000","last_unstale":"2024-03-28T16:51:31.205423+0000","last_undegraded":"2024-03-28T16:51:31.205423+0000","last_fullsized":"2024-03-28T16:51:31.205423+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'119","last_scrub_stamp":"2024-03-28T16:51:31.205230+0000","last_deep_scrub":"31'119","last_deep_scrub_stamp":"2024-03-28T16:51:31.205230+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.205230+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:47:05.795250+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:57.981295+0000","last_change":"2024-03-28T16:51:57.981352+0000","last_active":"2024-03-28T16:51:57.981295+0000","last_peered":"2024-03-28T16:51:57.981295+0000","last_clean":"2024-03-28T16:51:57.981295+0000","last_became_active":"2024-03-28T16:48:48.625016+0000","last_became_peered":"2024-03-28T16:48:48.625016+0000","last_unstale":"2024-03-28T16:51:57.981295+0000","last_undegraded":"2024-03-28T16:51:57.981295+0000","last_fullsized":"2024-03-28T16:51:57.981295+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:57.981069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:57.981069+0000","last_clean_scrub_stamp":"2024-03-28T16:51:57.981069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:37.245129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.064738+0000","last_change":"2024-03-28T16:51:41.064875+0000","last_active":"2024-03-28T16:51:41.064738+0000","last_peered":"2024-03-28T16:51:41.064738+0000","last_clean":"2024-03-28T16:51:41.064738+0000","last_became_active":"2024-03-28T16:48:54.665739+0000","last_became_peered":"2024-03-28T16:48:54.665739+0000","last_unstale":"2024-03-28T16:51:41.064738+0000","last_undegraded":"2024-03-28T16:51:41.064738+0000","last_fullsized":"2024-03-28T16:51:41.064738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:41.064504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:41.064504+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.064504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:26:51.380462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.295412+0000","last_change":"2024-03-28T16:51:03.295412+0000","last_active":"2024-03-28T16:51:03.295412+0000","last_peered":"2024-03-28T16:51:03.295412+0000","last_clean":"2024-03-28T16:51:03.295412+0000","last_became_active":"2024-03-28T16:48:58.682855+0000","last_became_peered":"2024-03-28T16:48:58.682855+0000","last_unstale":"2024-03-28T16:51:03.295412+0000","last_undegraded":"2024-03-28T16:51:03.295412+0000","last_fullsized":"2024-03-28T16:51:03.295412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.295139+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.295139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:08.548996+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"31'81","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.160163+0000","last_change":"2024-03-28T16:50:53.160163+0000","last_active":"2024-03-28T16:50:53.160163+0000","last_peered":"2024-03-28T16:50:53.160163+0000","last_clean":"2024-03-28T16:50:53.160163+0000","last_became_active":"2024-03-28T16:49:00.714562+0000","last_became_peered":"2024-03-28T16:49:00.714562+0000","last_unstale":"2024-03-28T16:50:53.160163+0000","last_undegraded":"2024-03-28T16:50:53.160163+0000","last_fullsized":"2024-03-28T16:50:53.160163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'81","last_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_deep_scrub":"31'81","last_deep_scrub_stamp":"2024-03-28T16:50:53.159787+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.159787+0000","objects_scrubbed":5,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:31:32.543735+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.290671+0000","last_change":"2024-03-28T16:50:57.290671+0000","last_active":"2024-03-28T16:50:57.290671+0000","last_peered":"2024-03-28T16:50:57.290671+0000","last_clean":"2024-03-28T16:50:57.290671+0000","last_became_active":"2024-03-28T16:49:02.730885+0000","last_became_peered":"2024-03-28T16:49:02.730885+0000","last_unstale":"2024-03-28T16:50:57.290671+0000","last_undegraded":"2024-03-28T16:50:57.290671+0000","last_fullsized":"2024-03-28T16:50:57.290671+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:57.290416+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.290416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:32.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"31'53","reported_seq":125,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.247075+0000","last_change":"2024-03-28T16:51:35.247207+0000","last_active":"2024-03-28T16:51:35.247075+0000","last_peered":"2024-03-28T16:51:35.247075+0000","last_clean":"2024-03-28T16:51:35.247075+0000","last_became_active":"2024-03-28T16:49:04.751558+0000","last_became_peered":"2024-03-28T16:49:04.751558+0000","last_unstale":"2024-03-28T16:51:35.247075+0000","last_undegraded":"2024-03-28T16:51:35.247075+0000","last_fullsized":"2024-03-28T16:51:35.247075+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'53","last_scrub_stamp":"2024-03-28T16:51:35.246864+0000","last_deep_scrub":"31'53","last_deep_scrub_stamp":"2024-03-28T16:51:35.246864+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.246864+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:53.626119+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":148,"num_read_kb":146,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"31'12","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.204617+0000","last_change":"2024-03-28T16:51:40.204617+0000","last_active":"2024-03-28T16:51:40.204617+0000","last_peered":"2024-03-28T16:51:40.204617+0000","last_clean":"2024-03-28T16:51:40.204617+0000","last_became_active":"2024-03-28T16:49:37.601016+0000","last_became_peered":"2024-03-28T16:49:37.601016+0000","last_unstale":"2024-03-28T16:51:40.204617+0000","last_undegraded":"2024-03-28T16:51:40.204617+0000","last_fullsized":"2024-03-28T16:51:40.204617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:40.204359+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:40.204359+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.204359+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:32.212954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:55.983074+0000","last_change":"2024-03-28T16:51:55.983136+0000","last_active":"2024-03-28T16:51:55.983074+0000","last_peered":"2024-03-28T16:51:55.983074+0000","last_clean":"2024-03-28T16:51:55.983074+0000","last_became_active":"2024-03-28T16:48:48.623654+0000","last_became_peered":"2024-03-28T16:48:48.623654+0000","last_unstale":"2024-03-28T16:51:55.983074+0000","last_undegraded":"2024-03-28T16:51:55.983074+0000","last_fullsized":"2024-03-28T16:51:55.983074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:55.982888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:55.982888+0000","last_clean_scrub_stamp":"2024-03-28T16:51:55.982888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:23:08.339948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.432079+0000","last_change":"2024-03-28T16:51:41.432131+0000","last_active":"2024-03-28T16:51:41.432079+0000","last_peered":"2024-03-28T16:51:41.432079+0000","last_clean":"2024-03-28T16:51:41.432079+0000","last_became_active":"2024-03-28T16:48:54.661831+0000","last_became_peered":"2024-03-28T16:48:54.661831+0000","last_unstale":"2024-03-28T16:51:41.432079+0000","last_undegraded":"2024-03-28T16:51:41.432079+0000","last_fullsized":"2024-03-28T16:51:41.432079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:41.431882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:41.431882+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.431882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:48.943852+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.191091+0000","last_change":"2024-03-28T16:50:47.191091+0000","last_active":"2024-03-28T16:50:47.191091+0000","last_peered":"2024-03-28T16:50:47.191091+0000","last_clean":"2024-03-28T16:50:47.191091+0000","last_became_active":"2024-03-28T16:48:58.705668+0000","last_became_peered":"2024-03-28T16:48:58.705668+0000","last_unstale":"2024-03-28T16:50:47.191091+0000","last_undegraded":"2024-03-28T16:50:47.191091+0000","last_fullsized":"2024-03-28T16:50:47.191091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.190844+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.190844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:09.946965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"31'260","reported_seq":455,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.259946+0000","last_change":"2024-03-28T16:50:54.259946+0000","last_active":"2024-03-28T16:50:54.259946+0000","last_peered":"2024-03-28T16:50:54.259946+0000","last_clean":"2024-03-28T16:50:54.259946+0000","last_became_active":"2024-03-28T16:49:00.717581+0000","last_became_peered":"2024-03-28T16:49:00.717581+0000","last_unstale":"2024-03-28T16:50:54.259946+0000","last_undegraded":"2024-03-28T16:50:54.259946+0000","last_fullsized":"2024-03-28T16:50:54.259946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'260","last_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_deep_scrub":"31'260","last_deep_scrub_stamp":"2024-03-28T16:50:54.259667+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.259667+0000","objects_scrubbed":10,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:21.748018+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":342,"num_read_kb":327,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.065783+0000","last_change":"2024-03-28T16:51:21.065839+0000","last_active":"2024-03-28T16:51:21.065783+0000","last_peered":"2024-03-28T16:51:21.065783+0000","last_clean":"2024-03-28T16:51:21.065783+0000","last_became_active":"2024-03-28T16:49:02.739334+0000","last_became_peered":"2024-03-28T16:49:02.739334+0000","last_unstale":"2024-03-28T16:51:21.065783+0000","last_undegraded":"2024-03-28T16:51:21.065783+0000","last_fullsized":"2024-03-28T16:51:21.065783+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.065552+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.065552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:21.706323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"31'5","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:42.068714+0000","last_change":"2024-03-28T16:51:42.068792+0000","last_active":"2024-03-28T16:51:42.068714+0000","last_peered":"2024-03-28T16:51:42.068714+0000","last_clean":"2024-03-28T16:51:42.068714+0000","last_became_active":"2024-03-28T16:49:04.744106+0000","last_became_peered":"2024-03-28T16:49:04.744106+0000","last_unstale":"2024-03-28T16:51:42.068714+0000","last_undegraded":"2024-03-28T16:51:42.068714+0000","last_fullsized":"2024-03-28T16:51:42.068714+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'5","last_scrub_stamp":"2024-03-28T16:51:42.068482+0000","last_deep_scrub":"31'5","last_deep_scrub_stamp":"2024-03-28T16:51:42.068482+0000","last_clean_scrub_stamp":"2024-03-28T16:51:42.068482+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:11.682779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:43.514905+0000","last_change":"2024-03-28T16:51:43.514968+0000","last_active":"2024-03-28T16:51:43.514905+0000","last_peered":"2024-03-28T16:51:43.514905+0000","last_clean":"2024-03-28T16:51:43.514905+0000","last_became_active":"2024-03-28T16:48:56.689099+0000","last_became_peered":"2024-03-28T16:48:56.689099+0000","last_unstale":"2024-03-28T16:51:43.514905+0000","last_undegraded":"2024-03-28T16:51:43.514905+0000","last_fullsized":"2024-03-28T16:51:43.514905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:43.514692+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:43.514692+0000","last_clean_scrub_stamp":"2024-03-28T16:51:43.514692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:47.685665+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.236189+0000","last_change":"2024-03-28T16:51:26.236274+0000","last_active":"2024-03-28T16:51:26.236189+0000","last_peered":"2024-03-28T16:51:26.236189+0000","last_clean":"2024-03-28T16:51:26.236189+0000","last_became_active":"2024-03-28T16:48:48.619726+0000","last_became_peered":"2024-03-28T16:48:48.619726+0000","last_unstale":"2024-03-28T16:51:26.236189+0000","last_undegraded":"2024-03-28T16:51:26.236189+0000","last_fullsized":"2024-03-28T16:51:26.236189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:26.235994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:26.235994+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.235994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:43.654590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:42.463543+0000","last_change":"2024-03-28T16:51:42.463599+0000","last_active":"2024-03-28T16:51:42.463543+0000","last_peered":"2024-03-28T16:51:42.463543+0000","last_clean":"2024-03-28T16:51:42.463543+0000","last_became_active":"2024-03-28T16:48:54.662170+0000","last_became_peered":"2024-03-28T16:48:54.662170+0000","last_unstale":"2024-03-28T16:51:42.463543+0000","last_undegraded":"2024-03-28T16:51:42.463543+0000","last_fullsized":"2024-03-28T16:51:42.463543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:42.463354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:42.463354+0000","last_clean_scrub_stamp":"2024-03-28T16:51:42.463354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:57:05.874302+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"31'2","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.414766+0000","last_change":"2024-03-28T16:51:07.414766+0000","last_active":"2024-03-28T16:51:07.414766+0000","last_peered":"2024-03-28T16:51:07.414766+0000","last_clean":"2024-03-28T16:51:07.414766+0000","last_became_active":"2024-03-28T16:48:58.685879+0000","last_became_peered":"2024-03-28T16:48:58.685879+0000","last_unstale":"2024-03-28T16:51:07.414766+0000","last_undegraded":"2024-03-28T16:51:07.414766+0000","last_fullsized":"2024-03-28T16:51:07.414766+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:07.414513+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.414513+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:54:53.600318+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"31'23","reported_seq":82,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.409006+0000","last_change":"2024-03-28T16:50:54.409006+0000","last_active":"2024-03-28T16:50:54.409006+0000","last_peered":"2024-03-28T16:50:54.409006+0000","last_clean":"2024-03-28T16:50:54.409006+0000","last_became_active":"2024-03-28T16:49:00.712920+0000","last_became_peered":"2024-03-28T16:49:00.712920+0000","last_unstale":"2024-03-28T16:50:54.409006+0000","last_undegraded":"2024-03-28T16:50:54.409006+0000","last_fullsized":"2024-03-28T16:50:54.409006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'23","last_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_deep_scrub":"31'23","last_deep_scrub_stamp":"2024-03-28T16:50:54.408718+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.408718+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:56.786363+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.341170+0000","last_change":"2024-03-28T16:51:01.341170+0000","last_active":"2024-03-28T16:51:01.341170+0000","last_peered":"2024-03-28T16:51:01.341170+0000","last_clean":"2024-03-28T16:51:01.341170+0000","last_became_active":"2024-03-28T16:49:02.731428+0000","last_became_peered":"2024-03-28T16:49:02.731428+0000","last_unstale":"2024-03-28T16:51:01.341170+0000","last_undegraded":"2024-03-28T16:51:01.341170+0000","last_fullsized":"2024-03-28T16:51:01.341170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:01.340925+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.340925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:06.594781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:06.941591+0000","last_change":"2024-03-28T16:52:06.941651+0000","last_active":"2024-03-28T16:52:06.941591+0000","last_peered":"2024-03-28T16:52:06.941591+0000","last_clean":"2024-03-28T16:52:06.941591+0000","last_became_active":"2024-03-28T16:49:04.756877+0000","last_became_peered":"2024-03-28T16:49:04.756877+0000","last_unstale":"2024-03-28T16:52:06.941591+0000","last_undegraded":"2024-03-28T16:52:06.941591+0000","last_fullsized":"2024-03-28T16:52:06.941591+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:06.941400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:06.941400+0000","last_clean_scrub_stamp":"2024-03-28T16:52:06.941400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:08.761464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.501412+0000","last_change":"2024-03-28T16:51:33.501547+0000","last_active":"2024-03-28T16:51:33.501412+0000","last_peered":"2024-03-28T16:51:33.501412+0000","last_clean":"2024-03-28T16:51:33.501412+0000","last_became_active":"2024-03-28T16:48:56.682832+0000","last_became_peered":"2024-03-28T16:48:56.682832+0000","last_unstale":"2024-03-28T16:51:33.501412+0000","last_undegraded":"2024-03-28T16:51:33.501412+0000","last_fullsized":"2024-03-28T16:51:33.501412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.501165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.501165+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.501165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:09.726882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:55.020918+0000","last_change":"2024-03-28T16:51:55.020972+0000","last_active":"2024-03-28T16:51:55.020918+0000","last_peered":"2024-03-28T16:51:55.020918+0000","last_clean":"2024-03-28T16:51:55.020918+0000","last_became_active":"2024-03-28T16:48:48.623491+0000","last_became_peered":"2024-03-28T16:48:48.623491+0000","last_unstale":"2024-03-28T16:51:55.020918+0000","last_undegraded":"2024-03-28T16:51:55.020918+0000","last_fullsized":"2024-03-28T16:51:55.020918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:55.020730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:55.020730+0000","last_clean_scrub_stamp":"2024-03-28T16:51:55.020730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:49:39.696603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.385902+0000","last_change":"2024-03-28T16:51:19.386018+0000","last_active":"2024-03-28T16:51:19.385902+0000","last_peered":"2024-03-28T16:51:19.385902+0000","last_clean":"2024-03-28T16:51:19.385902+0000","last_became_active":"2024-03-28T16:48:54.665915+0000","last_became_peered":"2024-03-28T16:48:54.665915+0000","last_unstale":"2024-03-28T16:51:19.385902+0000","last_undegraded":"2024-03-28T16:51:19.385902+0000","last_fullsized":"2024-03-28T16:51:19.385902+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:19.385614+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.385614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:54.970167+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.375959+0000","last_change":"2024-03-28T16:51:06.375959+0000","last_active":"2024-03-28T16:51:06.375959+0000","last_peered":"2024-03-28T16:51:06.375959+0000","last_clean":"2024-03-28T16:51:06.375959+0000","last_became_active":"2024-03-28T16:48:58.686866+0000","last_became_peered":"2024-03-28T16:48:58.686866+0000","last_unstale":"2024-03-28T16:51:06.375959+0000","last_undegraded":"2024-03-28T16:51:06.375959+0000","last_fullsized":"2024-03-28T16:51:06.375959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:06.375714+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.375714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:32.910452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"31'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.195780+0000","last_change":"2024-03-28T16:50:51.195780+0000","last_active":"2024-03-28T16:50:51.195780+0000","last_peered":"2024-03-28T16:50:51.195780+0000","last_clean":"2024-03-28T16:50:51.195780+0000","last_became_active":"2024-03-28T16:49:00.713517+0000","last_became_peered":"2024-03-28T16:49:00.713517+0000","last_unstale":"2024-03-28T16:50:51.195780+0000","last_undegraded":"2024-03-28T16:50:51.195780+0000","last_fullsized":"2024-03-28T16:50:51.195780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'19","last_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_deep_scrub":"31'19","last_deep_scrub_stamp":"2024-03-28T16:50:51.195522+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.195522+0000","objects_scrubbed":7,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:16.598840+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":19,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.126759+0000","last_change":"2024-03-28T16:51:25.126813+0000","last_active":"2024-03-28T16:51:25.126759+0000","last_peered":"2024-03-28T16:51:25.126759+0000","last_clean":"2024-03-28T16:51:25.126759+0000","last_became_active":"2024-03-28T16:49:02.740173+0000","last_became_peered":"2024-03-28T16:49:02.740173+0000","last_unstale":"2024-03-28T16:51:25.126759+0000","last_undegraded":"2024-03-28T16:51:25.126759+0000","last_fullsized":"2024-03-28T16:51:25.126759+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:25.126557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:25.126557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.126557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:42.133827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.302529+0000","last_change":"2024-03-28T16:51:13.302529+0000","last_active":"2024-03-28T16:51:13.302529+0000","last_peered":"2024-03-28T16:51:13.302529+0000","last_clean":"2024-03-28T16:51:13.302529+0000","last_became_active":"2024-03-28T16:49:04.739206+0000","last_became_peered":"2024-03-28T16:49:04.739206+0000","last_unstale":"2024-03-28T16:51:13.302529+0000","last_undegraded":"2024-03-28T16:51:13.302529+0000","last_fullsized":"2024-03-28T16:51:13.302529+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:13.302200+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.302200+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:42:35.072558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"31'6","reported_seq":68,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:12.328005+0000","last_change":"2024-03-28T16:51:12.328075+0000","last_active":"2024-03-28T16:51:12.328005+0000","last_peered":"2024-03-28T16:51:12.328005+0000","last_clean":"2024-03-28T16:51:12.328005+0000","last_became_active":"2024-03-28T16:48:56.674447+0000","last_became_peered":"2024-03-28T16:48:56.674447+0000","last_unstale":"2024-03-28T16:51:12.328005+0000","last_undegraded":"2024-03-28T16:51:12.328005+0000","last_fullsized":"2024-03-28T16:51:12.328005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:12.327822+0000","last_clean_scrub_stamp":"2024-03-28T16:51:12.327822+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:38:47.912375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.230294+0000","last_change":"2024-03-28T16:51:30.230347+0000","last_active":"2024-03-28T16:51:30.230294+0000","last_peered":"2024-03-28T16:51:30.230294+0000","last_clean":"2024-03-28T16:51:30.230294+0000","last_became_active":"2024-03-28T16:48:54.656496+0000","last_became_peered":"2024-03-28T16:48:54.656496+0000","last_unstale":"2024-03-28T16:51:30.230294+0000","last_undegraded":"2024-03-28T16:51:30.230294+0000","last_fullsized":"2024-03-28T16:51:30.230294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.230107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.230107+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.230107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:06:05.111329+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.444458+0000","last_change":"2024-03-28T16:51:36.444515+0000","last_active":"2024-03-28T16:51:36.444458+0000","last_peered":"2024-03-28T16:51:36.444458+0000","last_clean":"2024-03-28T16:51:36.444458+0000","last_became_active":"2024-03-28T16:48:48.622627+0000","last_became_peered":"2024-03-28T16:48:48.622627+0000","last_unstale":"2024-03-28T16:51:36.444458+0000","last_undegraded":"2024-03-28T16:51:36.444458+0000","last_fullsized":"2024-03-28T16:51:36.444458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.444268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.444268+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.444268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:33.466795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.355429+0000","last_change":"2024-03-28T16:51:03.355429+0000","last_active":"2024-03-28T16:51:03.355429+0000","last_peered":"2024-03-28T16:51:03.355429+0000","last_clean":"2024-03-28T16:51:03.355429+0000","last_became_active":"2024-03-28T16:48:58.689270+0000","last_became_peered":"2024-03-28T16:48:58.689270+0000","last_unstale":"2024-03-28T16:51:03.355429+0000","last_undegraded":"2024-03-28T16:51:03.355429+0000","last_fullsized":"2024-03-28T16:51:03.355429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.355165+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.355165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:33.030302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"31'69","reported_seq":145,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.306941+0000","last_change":"2024-03-28T16:50:46.306941+0000","last_active":"2024-03-28T16:50:46.306941+0000","last_peered":"2024-03-28T16:50:46.306941+0000","last_clean":"2024-03-28T16:50:46.306941+0000","last_became_active":"2024-03-28T16:49:00.706088+0000","last_became_peered":"2024-03-28T16:49:00.706088+0000","last_unstale":"2024-03-28T16:50:46.306941+0000","last_undegraded":"2024-03-28T16:50:46.306941+0000","last_fullsized":"2024-03-28T16:50:46.306941+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'69","last_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_deep_scrub":"31'69","last_deep_scrub_stamp":"2024-03-28T16:50:46.306710+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.306710+0000","objects_scrubbed":8,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:04:28.604401+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:51.441291+0000","last_change":"2024-03-28T16:50:51.441291+0000","last_active":"2024-03-28T16:50:51.441291+0000","last_peered":"2024-03-28T16:50:51.441291+0000","last_clean":"2024-03-28T16:50:51.441291+0000","last_became_active":"2024-03-28T16:49:02.720020+0000","last_became_peered":"2024-03-28T16:49:02.720020+0000","last_unstale":"2024-03-28T16:50:51.441291+0000","last_undegraded":"2024-03-28T16:50:51.441291+0000","last_fullsized":"2024-03-28T16:50:51.441291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:51.441051+0000","last_clean_scrub_stamp":"2024-03-28T16:50:51.441051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:13:01.864029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"31'11","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:44.568909+0000","last_change":"2024-03-28T16:51:44.568983+0000","last_active":"2024-03-28T16:51:44.568909+0000","last_peered":"2024-03-28T16:51:44.568909+0000","last_clean":"2024-03-28T16:51:44.568909+0000","last_became_active":"2024-03-28T16:49:04.740713+0000","last_became_peered":"2024-03-28T16:49:04.740713+0000","last_unstale":"2024-03-28T16:51:44.568909+0000","last_undegraded":"2024-03-28T16:51:44.568909+0000","last_fullsized":"2024-03-28T16:51:44.568909+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'11","last_scrub_stamp":"2024-03-28T16:51:44.568723+0000","last_deep_scrub":"31'11","last_deep_scrub_stamp":"2024-03-28T16:51:44.568723+0000","last_clean_scrub_stamp":"2024-03-28T16:51:44.568723+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:53.286332+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:39.006930+0000","last_change":"2024-03-28T16:51:39.006995+0000","last_active":"2024-03-28T16:51:39.006930+0000","last_peered":"2024-03-28T16:51:39.006930+0000","last_clean":"2024-03-28T16:51:39.006930+0000","last_became_active":"2024-03-28T16:48:56.683153+0000","last_became_peered":"2024-03-28T16:48:56.683153+0000","last_unstale":"2024-03-28T16:51:39.006930+0000","last_undegraded":"2024-03-28T16:51:39.006930+0000","last_fullsized":"2024-03-28T16:51:39.006930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:39.006741+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:39.006741+0000","last_clean_scrub_stamp":"2024-03-28T16:51:39.006741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:46.985028+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.491771+0000","last_change":"2024-03-28T16:51:34.491822+0000","last_active":"2024-03-28T16:51:34.491771+0000","last_peered":"2024-03-28T16:51:34.491771+0000","last_clean":"2024-03-28T16:51:34.491771+0000","last_became_active":"2024-03-28T16:48:54.662120+0000","last_became_peered":"2024-03-28T16:48:54.662120+0000","last_unstale":"2024-03-28T16:51:34.491771+0000","last_undegraded":"2024-03-28T16:51:34.491771+0000","last_fullsized":"2024-03-28T16:51:34.491771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:34.491575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:34.491575+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.491575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:45:59.766359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:56.935354+0000","last_change":"2024-03-28T16:51:56.935407+0000","last_active":"2024-03-28T16:51:56.935354+0000","last_peered":"2024-03-28T16:51:56.935354+0000","last_clean":"2024-03-28T16:51:56.935354+0000","last_became_active":"2024-03-28T16:48:48.623908+0000","last_became_peered":"2024-03-28T16:48:48.623908+0000","last_unstale":"2024-03-28T16:51:56.935354+0000","last_undegraded":"2024-03-28T16:51:56.935354+0000","last_fullsized":"2024-03-28T16:51:56.935354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:56.935166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:56.935166+0000","last_clean_scrub_stamp":"2024-03-28T16:51:56.935166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:00:15.278636+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.159039+0000","last_change":"2024-03-28T16:50:55.159039+0000","last_active":"2024-03-28T16:50:55.159039+0000","last_peered":"2024-03-28T16:50:55.159039+0000","last_clean":"2024-03-28T16:50:55.159039+0000","last_became_active":"2024-03-28T16:48:58.682361+0000","last_became_peered":"2024-03-28T16:48:58.682361+0000","last_unstale":"2024-03-28T16:50:55.159039+0000","last_undegraded":"2024-03-28T16:50:55.159039+0000","last_fullsized":"2024-03-28T16:50:55.159039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:55.158776+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.158776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:53:10.542165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"31'21","reported_seq":62,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.252537+0000","last_change":"2024-03-28T16:50:52.252537+0000","last_active":"2024-03-28T16:50:52.252537+0000","last_peered":"2024-03-28T16:50:52.252537+0000","last_clean":"2024-03-28T16:50:52.252537+0000","last_became_active":"2024-03-28T16:49:00.719873+0000","last_became_peered":"2024-03-28T16:49:00.719873+0000","last_unstale":"2024-03-28T16:50:52.252537+0000","last_undegraded":"2024-03-28T16:50:52.252537+0000","last_fullsized":"2024-03-28T16:50:52.252537+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'21","last_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_deep_scrub":"31'21","last_deep_scrub_stamp":"2024-03-28T16:50:52.252240+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.252240+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:47.715342+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"31'249","reported_seq":304,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:46.104945+0000","last_change":"2024-03-28T16:50:46.104945+0000","last_active":"2024-03-28T16:50:46.104945+0000","last_peered":"2024-03-28T16:50:46.104945+0000","last_clean":"2024-03-28T16:50:46.104945+0000","last_became_active":"2024-03-28T16:49:02.715064+0000","last_became_peered":"2024-03-28T16:49:02.715064+0000","last_unstale":"2024-03-28T16:50:46.104945+0000","last_undegraded":"2024-03-28T16:50:46.104945+0000","last_fullsized":"2024-03-28T16:50:46.104945+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:46.104655+0000","last_clean_scrub_stamp":"2024-03-28T16:50:46.104655+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:39.533970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"31'2","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:09.884032+0000","last_change":"2024-03-28T16:52:09.884099+0000","last_active":"2024-03-28T16:52:09.884032+0000","last_peered":"2024-03-28T16:52:09.884032+0000","last_clean":"2024-03-28T16:52:09.884032+0000","last_became_active":"2024-03-28T16:49:04.762988+0000","last_became_peered":"2024-03-28T16:49:04.762988+0000","last_unstale":"2024-03-28T16:52:09.884032+0000","last_undegraded":"2024-03-28T16:52:09.884032+0000","last_fullsized":"2024-03-28T16:52:09.884032+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:52:09.883836+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:52:09.883836+0000","last_clean_scrub_stamp":"2024-03-28T16:52:09.883836+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:07.187017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.222398+0000","last_change":"2024-03-28T16:51:04.222527+0000","last_active":"2024-03-28T16:51:04.222398+0000","last_peered":"2024-03-28T16:51:04.222398+0000","last_clean":"2024-03-28T16:51:04.222398+0000","last_became_active":"2024-03-28T16:48:56.693035+0000","last_became_peered":"2024-03-28T16:48:56.693035+0000","last_unstale":"2024-03-28T16:51:04.222398+0000","last_undegraded":"2024-03-28T16:51:04.222398+0000","last_fullsized":"2024-03-28T16:51:04.222398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:04.222151+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.222151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:44:13.382250+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:50.030518+0000","last_change":"2024-03-28T16:51:50.030630+0000","last_active":"2024-03-28T16:51:50.030518+0000","last_peered":"2024-03-28T16:51:50.030518+0000","last_clean":"2024-03-28T16:51:50.030518+0000","last_became_active":"2024-03-28T16:48:54.661861+0000","last_became_peered":"2024-03-28T16:48:54.661861+0000","last_unstale":"2024-03-28T16:51:50.030518+0000","last_undegraded":"2024-03-28T16:51:50.030518+0000","last_fullsized":"2024-03-28T16:51:50.030518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:50.030315+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:50.030315+0000","last_clean_scrub_stamp":"2024-03-28T16:51:50.030315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:56:37.849059+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.449142+0000","last_change":"2024-03-28T16:51:35.449195+0000","last_active":"2024-03-28T16:51:35.449142+0000","last_peered":"2024-03-28T16:51:35.449142+0000","last_clean":"2024-03-28T16:51:35.449142+0000","last_became_active":"2024-03-28T16:48:48.621748+0000","last_became_peered":"2024-03-28T16:48:48.621748+0000","last_unstale":"2024-03-28T16:51:35.449142+0000","last_undegraded":"2024-03-28T16:51:35.449142+0000","last_fullsized":"2024-03-28T16:51:35.449142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:35.448941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:35.448941+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.448941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:12.528913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:54.163965+0000","last_change":"2024-03-28T16:50:54.163965+0000","last_active":"2024-03-28T16:50:54.163965+0000","last_peered":"2024-03-28T16:50:54.163965+0000","last_clean":"2024-03-28T16:50:54.163965+0000","last_became_active":"2024-03-28T16:48:58.683654+0000","last_became_peered":"2024-03-28T16:48:58.683654+0000","last_unstale":"2024-03-28T16:50:54.163965+0000","last_undegraded":"2024-03-28T16:50:54.163965+0000","last_fullsized":"2024-03-28T16:50:54.163965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:54.163592+0000","last_clean_scrub_stamp":"2024-03-28T16:50:54.163592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:03:27.253015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.038093+0000","last_change":"2024-03-28T16:51:16.038179+0000","last_active":"2024-03-28T16:51:16.038093+0000","last_peered":"2024-03-28T16:51:16.038093+0000","last_clean":"2024-03-28T16:51:16.038093+0000","last_became_active":"2024-03-28T16:49:00.720064+0000","last_became_peered":"2024-03-28T16:49:00.720064+0000","last_unstale":"2024-03-28T16:51:16.038093+0000","last_undegraded":"2024-03-28T16:51:16.038093+0000","last_fullsized":"2024-03-28T16:51:16.038093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:16.037881+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.037881+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:09:44.064566+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"31'249","reported_seq":337,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.233041+0000","last_change":"2024-03-28T16:50:49.233041+0000","last_active":"2024-03-28T16:50:49.233041+0000","last_peered":"2024-03-28T16:50:49.233041+0000","last_clean":"2024-03-28T16:50:49.233041+0000","last_became_active":"2024-03-28T16:49:02.729612+0000","last_became_peered":"2024-03-28T16:49:02.729612+0000","last_unstale":"2024-03-28T16:50:49.233041+0000","last_undegraded":"2024-03-28T16:50:49.233041+0000","last_fullsized":"2024-03-28T16:50:49.233041+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:49.232798+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.232798+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:57.227518+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'3","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.267309+0000","last_change":"2024-03-28T16:51:09.267309+0000","last_active":"2024-03-28T16:51:09.267309+0000","last_peered":"2024-03-28T16:51:09.267309+0000","last_clean":"2024-03-28T16:51:09.267309+0000","last_became_active":"2024-03-28T16:49:04.738417+0000","last_became_peered":"2024-03-28T16:49:04.738417+0000","last_unstale":"2024-03-28T16:51:09.267309+0000","last_undegraded":"2024-03-28T16:51:09.267309+0000","last_fullsized":"2024-03-28T16:51:09.267309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-03-28T16:51:09.267056+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.267056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:58.701007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"31'12","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.363436+0000","last_change":"2024-03-28T16:51:16.363501+0000","last_active":"2024-03-28T16:51:16.363436+0000","last_peered":"2024-03-28T16:51:16.363436+0000","last_clean":"2024-03-28T16:51:16.363436+0000","last_became_active":"2024-03-28T16:48:56.682030+0000","last_became_peered":"2024-03-28T16:48:56.682030+0000","last_unstale":"2024-03-28T16:51:16.363436+0000","last_undegraded":"2024-03-28T16:51:16.363436+0000","last_fullsized":"2024-03-28T16:51:16.363436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:16.363254+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.363254+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:07:05.797204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":44,"num_read_kb":40,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"31'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.287919+0000","last_change":"2024-03-28T16:51:20.287986+0000","last_active":"2024-03-28T16:51:20.287919+0000","last_peered":"2024-03-28T16:51:20.287919+0000","last_clean":"2024-03-28T16:51:20.287919+0000","last_became_active":"2024-03-28T16:48:54.673618+0000","last_became_peered":"2024-03-28T16:48:54.673618+0000","last_unstale":"2024-03-28T16:51:20.287919+0000","last_undegraded":"2024-03-28T16:51:20.287919+0000","last_fullsized":"2024-03-28T16:51:20.287919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:20.287729+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.287729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:53.368108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:58.945626+0000","last_change":"2024-03-28T16:51:58.945678+0000","last_active":"2024-03-28T16:51:58.945626+0000","last_peered":"2024-03-28T16:51:58.945626+0000","last_clean":"2024-03-28T16:51:58.945626+0000","last_became_active":"2024-03-28T16:48:49.260854+0000","last_became_peered":"2024-03-28T16:48:49.260854+0000","last_unstale":"2024-03-28T16:51:58.945626+0000","last_undegraded":"2024-03-28T16:51:58.945626+0000","last_fullsized":"2024-03-28T16:51:58.945626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:58.945435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:58.945435+0000","last_clean_scrub_stamp":"2024-03-28T16:51:58.945435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:30:27.534851+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.166685+0000","last_change":"2024-03-28T16:50:56.166685+0000","last_active":"2024-03-28T16:50:56.166685+0000","last_peered":"2024-03-28T16:50:56.166685+0000","last_clean":"2024-03-28T16:50:56.166685+0000","last_became_active":"2024-03-28T16:48:58.680496+0000","last_became_peered":"2024-03-28T16:48:58.680496+0000","last_unstale":"2024-03-28T16:50:56.166685+0000","last_undegraded":"2024-03-28T16:50:56.166685+0000","last_fullsized":"2024-03-28T16:50:56.166685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:56.166428+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.166428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:06.567829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"31'90","reported_seq":183,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.384285+0000","last_change":"2024-03-28T16:50:58.384285+0000","last_active":"2024-03-28T16:50:58.384285+0000","last_peered":"2024-03-28T16:50:58.384285+0000","last_clean":"2024-03-28T16:50:58.384285+0000","last_became_active":"2024-03-28T16:49:00.708312+0000","last_became_peered":"2024-03-28T16:49:00.708312+0000","last_unstale":"2024-03-28T16:50:58.384285+0000","last_undegraded":"2024-03-28T16:50:58.384285+0000","last_fullsized":"2024-03-28T16:50:58.384285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'90","last_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_deep_scrub":"31'90","last_deep_scrub_stamp":"2024-03-28T16:50:58.383995+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.383995+0000","objects_scrubbed":8,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:20:54.487031+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.285325+0000","last_change":"2024-03-28T16:50:47.285325+0000","last_active":"2024-03-28T16:50:47.285325+0000","last_peered":"2024-03-28T16:50:47.285325+0000","last_clean":"2024-03-28T16:50:47.285325+0000","last_became_active":"2024-03-28T16:49:02.714352+0000","last_became_peered":"2024-03-28T16:49:02.714352+0000","last_unstale":"2024-03-28T16:50:47.285325+0000","last_undegraded":"2024-03-28T16:50:47.285325+0000","last_fullsized":"2024-03-28T16:50:47.285325+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.285202+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.285202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:46:10.170506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.228376+0000","last_change":"2024-03-28T16:51:36.228433+0000","last_active":"2024-03-28T16:51:36.228376+0000","last_peered":"2024-03-28T16:51:36.228376+0000","last_clean":"2024-03-28T16:51:36.228376+0000","last_became_active":"2024-03-28T16:49:04.751541+0000","last_became_peered":"2024-03-28T16:49:04.751541+0000","last_unstale":"2024-03-28T16:51:36.228376+0000","last_undegraded":"2024-03-28T16:51:36.228376+0000","last_fullsized":"2024-03-28T16:51:36.228376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.228185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.228185+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.228185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:41.408321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.344614+0000","last_change":"2024-03-28T16:51:00.344614+0000","last_active":"2024-03-28T16:51:00.344614+0000","last_peered":"2024-03-28T16:51:00.344614+0000","last_clean":"2024-03-28T16:51:00.344614+0000","last_became_active":"2024-03-28T16:49:04.740213+0000","last_became_peered":"2024-03-28T16:49:04.740213+0000","last_unstale":"2024-03-28T16:51:00.344614+0000","last_undegraded":"2024-03-28T16:51:00.344614+0000","last_fullsized":"2024-03-28T16:51:00.344614+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:00.344363+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.344363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:57.233454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'6","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.364821+0000","last_change":"2024-03-28T16:51:10.364821+0000","last_active":"2024-03-28T16:51:10.364821+0000","last_peered":"2024-03-28T16:51:10.364821+0000","last_clean":"2024-03-28T16:51:10.364821+0000","last_became_active":"2024-03-28T16:48:56.680314+0000","last_became_peered":"2024-03-28T16:48:56.680314+0000","last_unstale":"2024-03-28T16:51:10.364821+0000","last_undegraded":"2024-03-28T16:51:10.364821+0000","last_fullsized":"2024-03-28T16:51:10.364821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:10.364556+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.364556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:03:15.618413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:46.043748+0000","last_change":"2024-03-28T16:51:46.043800+0000","last_active":"2024-03-28T16:51:46.043748+0000","last_peered":"2024-03-28T16:51:46.043748+0000","last_clean":"2024-03-28T16:51:46.043748+0000","last_became_active":"2024-03-28T16:48:54.662705+0000","last_became_peered":"2024-03-28T16:48:54.662705+0000","last_unstale":"2024-03-28T16:51:46.043748+0000","last_undegraded":"2024-03-28T16:51:46.043748+0000","last_fullsized":"2024-03-28T16:51:46.043748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:46.043568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:46.043568+0000","last_clean_scrub_stamp":"2024-03-28T16:51:46.043568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:33:51.512636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:45.081449+0000","last_change":"2024-03-28T16:51:45.081503+0000","last_active":"2024-03-28T16:51:45.081449+0000","last_peered":"2024-03-28T16:51:45.081449+0000","last_clean":"2024-03-28T16:51:45.081449+0000","last_became_active":"2024-03-28T16:49:37.606075+0000","last_became_peered":"2024-03-28T16:49:37.606075+0000","last_unstale":"2024-03-28T16:51:45.081449+0000","last_undegraded":"2024-03-28T16:51:45.081449+0000","last_fullsized":"2024-03-28T16:51:45.081449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:45.081273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:45.081273+0000","last_clean_scrub_stamp":"2024-03-28T16:51:45.081273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:05:49.252246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.084532+0000","last_change":"2024-03-28T16:51:15.084532+0000","last_active":"2024-03-28T16:51:15.084532+0000","last_peered":"2024-03-28T16:51:15.084532+0000","last_clean":"2024-03-28T16:51:15.084532+0000","last_became_active":"2024-03-28T16:49:00.724954+0000","last_became_peered":"2024-03-28T16:49:00.724954+0000","last_unstale":"2024-03-28T16:51:15.084532+0000","last_undegraded":"2024-03-28T16:51:15.084532+0000","last_fullsized":"2024-03-28T16:51:15.084532+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:51:15.084235+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.084235+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:01:39.553948+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.246355+0000","last_change":"2024-03-28T16:51:09.246355+0000","last_active":"2024-03-28T16:51:09.246355+0000","last_peered":"2024-03-28T16:51:09.246355+0000","last_clean":"2024-03-28T16:51:09.246355+0000","last_became_active":"2024-03-28T16:49:02.736971+0000","last_became_peered":"2024-03-28T16:49:02.736971+0000","last_unstale":"2024-03-28T16:51:09.246355+0000","last_undegraded":"2024-03-28T16:51:09.246355+0000","last_fullsized":"2024-03-28T16:51:09.246355+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.246098+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.246098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:55:30.745667+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"31'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.442625+0000","last_change":"2024-03-28T16:51:17.442696+0000","last_active":"2024-03-28T16:51:17.442625+0000","last_peered":"2024-03-28T16:51:17.442625+0000","last_clean":"2024-03-28T16:51:17.442625+0000","last_became_active":"2024-03-28T16:49:04.722728+0000","last_became_peered":"2024-03-28T16:49:04.722728+0000","last_unstale":"2024-03-28T16:51:17.442625+0000","last_undegraded":"2024-03-28T16:51:17.442625+0000","last_fullsized":"2024-03-28T16:51:17.442625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:17.442424+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.442424+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:00:34.861019+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:44.063414+0000","last_change":"2024-03-28T16:51:44.063470+0000","last_active":"2024-03-28T16:51:44.063414+0000","last_peered":"2024-03-28T16:51:44.063414+0000","last_clean":"2024-03-28T16:51:44.063414+0000","last_became_active":"2024-03-28T16:49:37.602800+0000","last_became_peered":"2024-03-28T16:49:37.602800+0000","last_unstale":"2024-03-28T16:51:44.063414+0000","last_undegraded":"2024-03-28T16:51:44.063414+0000","last_fullsized":"2024-03-28T16:51:44.063414+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:44.063240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:44.063240+0000","last_clean_scrub_stamp":"2024-03-28T16:51:44.063240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:57:07.474188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.199918+0000","last_change":"2024-03-28T16:51:31.200010+0000","last_active":"2024-03-28T16:51:31.199918+0000","last_peered":"2024-03-28T16:51:31.199918+0000","last_clean":"2024-03-28T16:51:31.199918+0000","last_became_active":"2024-03-28T16:48:54.657149+0000","last_became_peered":"2024-03-28T16:48:54.657149+0000","last_unstale":"2024-03-28T16:51:31.199918+0000","last_undegraded":"2024-03-28T16:51:31.199918+0000","last_fullsized":"2024-03-28T16:51:31.199918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:31.199694+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:31.199694+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.199694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:43.004903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.178814+0000","last_change":"2024-03-28T16:50:52.178814+0000","last_active":"2024-03-28T16:50:52.178814+0000","last_peered":"2024-03-28T16:50:52.178814+0000","last_clean":"2024-03-28T16:50:52.178814+0000","last_became_active":"2024-03-28T16:48:58.695730+0000","last_became_peered":"2024-03-28T16:48:58.695730+0000","last_unstale":"2024-03-28T16:50:52.178814+0000","last_undegraded":"2024-03-28T16:50:52.178814+0000","last_fullsized":"2024-03-28T16:50:52.178814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:52.178561+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.178561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:00.951220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"31'8","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:00.353835+0000","last_change":"2024-03-28T16:51:00.353835+0000","last_active":"2024-03-28T16:51:00.353835+0000","last_peered":"2024-03-28T16:51:00.353835+0000","last_clean":"2024-03-28T16:51:00.353835+0000","last_became_active":"2024-03-28T16:49:00.700022+0000","last_became_peered":"2024-03-28T16:49:00.700022+0000","last_unstale":"2024-03-28T16:51:00.353835+0000","last_undegraded":"2024-03-28T16:51:00.353835+0000","last_fullsized":"2024-03-28T16:51:00.353835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'8","last_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_deep_scrub":"31'8","last_deep_scrub_stamp":"2024-03-28T16:51:00.353557+0000","last_clean_scrub_stamp":"2024-03-28T16:51:00.353557+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:18.400713+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:49.443394+0000","last_change":"2024-03-28T16:50:49.443394+0000","last_active":"2024-03-28T16:50:49.443394+0000","last_peered":"2024-03-28T16:50:49.443394+0000","last_clean":"2024-03-28T16:50:49.443394+0000","last_became_active":"2024-03-28T16:49:02.726000+0000","last_became_peered":"2024-03-28T16:49:02.726000+0000","last_unstale":"2024-03-28T16:50:49.443394+0000","last_undegraded":"2024-03-28T16:50:49.443394+0000","last_fullsized":"2024-03-28T16:50:49.443394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:49.443035+0000","last_clean_scrub_stamp":"2024-03-28T16:50:49.443035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:33:21.896718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:48.480062+0000","last_change":"2024-03-28T16:51:48.480176+0000","last_active":"2024-03-28T16:51:48.480062+0000","last_peered":"2024-03-28T16:51:48.480062+0000","last_clean":"2024-03-28T16:51:48.480062+0000","last_became_active":"2024-03-28T16:49:04.743424+0000","last_became_peered":"2024-03-28T16:49:04.743424+0000","last_unstale":"2024-03-28T16:51:48.480062+0000","last_undegraded":"2024-03-28T16:51:48.480062+0000","last_fullsized":"2024-03-28T16:51:48.480062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:48.479806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:48.479806+0000","last_clean_scrub_stamp":"2024-03-28T16:51:48.479806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:10:13.930593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.486418+0000","last_change":"2024-03-28T16:51:19.486556+0000","last_active":"2024-03-28T16:51:19.486418+0000","last_peered":"2024-03-28T16:51:19.486418+0000","last_clean":"2024-03-28T16:51:19.486418+0000","last_became_active":"2024-03-28T16:48:56.675974+0000","last_became_peered":"2024-03-28T16:48:56.675974+0000","last_unstale":"2024-03-28T16:51:19.486418+0000","last_undegraded":"2024-03-28T16:51:19.486418+0000","last_fullsized":"2024-03-28T16:51:19.486418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.486207+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.486207+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:42.867614+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.236966+0000","last_change":"2024-03-28T16:51:33.237031+0000","last_active":"2024-03-28T16:51:33.236966+0000","last_peered":"2024-03-28T16:51:33.236966+0000","last_clean":"2024-03-28T16:51:33.236966+0000","last_became_active":"2024-03-28T16:48:54.658696+0000","last_became_peered":"2024-03-28T16:48:54.658696+0000","last_unstale":"2024-03-28T16:51:33.236966+0000","last_undegraded":"2024-03-28T16:51:33.236966+0000","last_fullsized":"2024-03-28T16:51:33.236966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:33.236770+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:33.236770+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.236770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:05:19.456972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"31'3","reported_seq":209,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.379662+0000","last_change":"2024-03-28T16:51:06.379662+0000","last_active":"2024-03-28T16:51:06.379662+0000","last_peered":"2024-03-28T16:51:06.379662+0000","last_clean":"2024-03-28T16:51:06.379662+0000","last_became_active":"2024-03-28T16:48:58.689275+0000","last_became_peered":"2024-03-28T16:48:58.689275+0000","last_unstale":"2024-03-28T16:51:06.379662+0000","last_undegraded":"2024-03-28T16:51:06.379662+0000","last_fullsized":"2024-03-28T16:51:06.379662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:51:06.379283+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.379283+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:41.522575+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":256,"num_read_kb":255,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"31'42","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:55.233580+0000","last_change":"2024-03-28T16:50:55.233580+0000","last_active":"2024-03-28T16:50:55.233580+0000","last_peered":"2024-03-28T16:50:55.233580+0000","last_clean":"2024-03-28T16:50:55.233580+0000","last_became_active":"2024-03-28T16:49:00.717886+0000","last_became_peered":"2024-03-28T16:49:00.717886+0000","last_unstale":"2024-03-28T16:50:55.233580+0000","last_undegraded":"2024-03-28T16:50:55.233580+0000","last_fullsized":"2024-03-28T16:50:55.233580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'42","last_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_deep_scrub":"31'42","last_deep_scrub_stamp":"2024-03-28T16:50:55.233264+0000","last_clean_scrub_stamp":"2024-03-28T16:50:55.233264+0000","objects_scrubbed":6,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:03:43.214573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":56,"num_write":38,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:47.129943+0000","last_change":"2024-03-28T16:50:47.129943+0000","last_active":"2024-03-28T16:50:47.129943+0000","last_peered":"2024-03-28T16:50:47.129943+0000","last_clean":"2024-03-28T16:50:47.129943+0000","last_became_active":"2024-03-28T16:49:02.714305+0000","last_became_peered":"2024-03-28T16:49:02.714305+0000","last_unstale":"2024-03-28T16:50:47.129943+0000","last_undegraded":"2024-03-28T16:50:47.129943+0000","last_fullsized":"2024-03-28T16:50:47.129943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:47.129698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:47.129698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:49.507343+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:04.024275+0000","last_change":"2024-03-28T16:52:04.024329+0000","last_active":"2024-03-28T16:52:04.024275+0000","last_peered":"2024-03-28T16:52:04.024275+0000","last_clean":"2024-03-28T16:52:04.024275+0000","last_became_active":"2024-03-28T16:49:04.749702+0000","last_became_peered":"2024-03-28T16:49:04.749702+0000","last_unstale":"2024-03-28T16:52:04.024275+0000","last_undegraded":"2024-03-28T16:52:04.024275+0000","last_fullsized":"2024-03-28T16:52:04.024275+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:04.024095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:04.024095+0000","last_clean_scrub_stamp":"2024-03-28T16:52:04.024095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:41:48.078302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'6","reported_seq":58,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.093412+0000","last_change":"2024-03-28T16:51:26.093486+0000","last_active":"2024-03-28T16:51:26.093412+0000","last_peered":"2024-03-28T16:51:26.093412+0000","last_clean":"2024-03-28T16:51:26.093412+0000","last_became_active":"2024-03-28T16:48:56.682795+0000","last_became_peered":"2024-03-28T16:48:56.682795+0000","last_unstale":"2024-03-28T16:51:26.093412+0000","last_undegraded":"2024-03-28T16:51:26.093412+0000","last_fullsized":"2024-03-28T16:51:26.093412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:26.093216+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:26.093216+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.093216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:08.375290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.237688+0000","last_change":"2024-03-28T16:51:26.237758+0000","last_active":"2024-03-28T16:51:26.237688+0000","last_peered":"2024-03-28T16:51:26.237688+0000","last_clean":"2024-03-28T16:51:26.237688+0000","last_became_active":"2024-03-28T16:48:54.673216+0000","last_became_peered":"2024-03-28T16:48:54.673216+0000","last_unstale":"2024-03-28T16:51:26.237688+0000","last_undegraded":"2024-03-28T16:51:26.237688+0000","last_fullsized":"2024-03-28T16:51:26.237688+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:26.237512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:26.237512+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.237512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:57.241670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.166465+0000","last_change":"2024-03-28T16:50:58.166465+0000","last_active":"2024-03-28T16:50:58.166465+0000","last_peered":"2024-03-28T16:50:58.166465+0000","last_clean":"2024-03-28T16:50:58.166465+0000","last_became_active":"2024-03-28T16:48:58.679497+0000","last_became_peered":"2024-03-28T16:48:58.679497+0000","last_unstale":"2024-03-28T16:50:58.166465+0000","last_undegraded":"2024-03-28T16:50:58.166465+0000","last_fullsized":"2024-03-28T16:50:58.166465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:58.166221+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.166221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:45:21.547629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.370703+0000","last_change":"2024-03-28T16:51:02.370703+0000","last_active":"2024-03-28T16:51:02.370703+0000","last_peered":"2024-03-28T16:51:02.370703+0000","last_clean":"2024-03-28T16:51:02.370703+0000","last_became_active":"2024-03-28T16:49:00.696142+0000","last_became_peered":"2024-03-28T16:49:00.696142+0000","last_unstale":"2024-03-28T16:51:02.370703+0000","last_undegraded":"2024-03-28T16:51:02.370703+0000","last_fullsized":"2024-03-28T16:51:02.370703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:51:02.370450+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.370450+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:40:15.038783+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.448158+0000","last_change":"2024-03-28T16:50:50.448158+0000","last_active":"2024-03-28T16:50:50.448158+0000","last_peered":"2024-03-28T16:50:50.448158+0000","last_clean":"2024-03-28T16:50:50.448158+0000","last_became_active":"2024-03-28T16:49:02.725529+0000","last_became_peered":"2024-03-28T16:49:02.725529+0000","last_unstale":"2024-03-28T16:50:50.448158+0000","last_undegraded":"2024-03-28T16:50:50.448158+0000","last_fullsized":"2024-03-28T16:50:50.448158+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.447911+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.447911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:02:45.521899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.151593+0000","last_change":"2024-03-28T16:51:31.151647+0000","last_active":"2024-03-28T16:51:31.151593+0000","last_peered":"2024-03-28T16:51:31.151593+0000","last_clean":"2024-03-28T16:51:31.151593+0000","last_became_active":"2024-03-28T16:48:56.675060+0000","last_became_peered":"2024-03-28T16:48:56.675060+0000","last_unstale":"2024-03-28T16:51:31.151593+0000","last_undegraded":"2024-03-28T16:51:31.151593+0000","last_fullsized":"2024-03-28T16:51:31.151593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:31.151371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:31.151371+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.151371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:46.085551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:50.460481+0000","last_change":"2024-03-28T16:51:50.460481+0000","last_active":"2024-03-28T16:51:50.460481+0000","last_peered":"2024-03-28T16:51:50.460481+0000","last_clean":"2024-03-28T16:51:50.460481+0000","last_became_active":"2024-03-28T16:49:37.599647+0000","last_became_peered":"2024-03-28T16:49:37.599647+0000","last_unstale":"2024-03-28T16:51:50.460481+0000","last_undegraded":"2024-03-28T16:51:50.460481+0000","last_fullsized":"2024-03-28T16:51:50.460481+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:50.460232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:50.460232+0000","last_clean_scrub_stamp":"2024-03-28T16:51:50.460232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:34.292013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:07.369809+0000","last_change":"2024-03-28T16:51:07.369837+0000","last_active":"2024-03-28T16:51:07.369809+0000","last_peered":"2024-03-28T16:51:07.369809+0000","last_clean":"2024-03-28T16:51:07.369809+0000","last_became_active":"2024-03-28T16:48:58.686480+0000","last_became_peered":"2024-03-28T16:48:58.686480+0000","last_unstale":"2024-03-28T16:51:07.369809+0000","last_undegraded":"2024-03-28T16:51:07.369809+0000","last_fullsized":"2024-03-28T16:51:07.369809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:07.369707+0000","last_clean_scrub_stamp":"2024-03-28T16:51:07.369707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:56.803658+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"31'60","reported_seq":133,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:56.252559+0000","last_change":"2024-03-28T16:50:56.252559+0000","last_active":"2024-03-28T16:50:56.252559+0000","last_peered":"2024-03-28T16:50:56.252559+0000","last_clean":"2024-03-28T16:50:56.252559+0000","last_became_active":"2024-03-28T16:49:00.715342+0000","last_became_peered":"2024-03-28T16:49:00.715342+0000","last_unstale":"2024-03-28T16:50:56.252559+0000","last_undegraded":"2024-03-28T16:50:56.252559+0000","last_fullsized":"2024-03-28T16:50:56.252559+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'60","last_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_deep_scrub":"31'60","last_deep_scrub_stamp":"2024-03-28T16:50:56.252305+0000","last_clean_scrub_stamp":"2024-03-28T16:50:56.252305+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:41.009336+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.444559+0000","last_change":"2024-03-28T16:50:48.444559+0000","last_active":"2024-03-28T16:50:48.444559+0000","last_peered":"2024-03-28T16:50:48.444559+0000","last_clean":"2024-03-28T16:50:48.444559+0000","last_became_active":"2024-03-28T16:49:02.727925+0000","last_became_peered":"2024-03-28T16:49:02.727925+0000","last_unstale":"2024-03-28T16:50:48.444559+0000","last_undegraded":"2024-03-28T16:50:48.444559+0000","last_fullsized":"2024-03-28T16:50:48.444559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.444297+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.444297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:24:42.861356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"31'12","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:49.443162+0000","last_change":"2024-03-28T16:51:49.443231+0000","last_active":"2024-03-28T16:51:49.443162+0000","last_peered":"2024-03-28T16:51:49.443162+0000","last_clean":"2024-03-28T16:51:49.443162+0000","last_became_active":"2024-03-28T16:49:04.751988+0000","last_became_peered":"2024-03-28T16:49:04.751988+0000","last_unstale":"2024-03-28T16:51:49.443162+0000","last_undegraded":"2024-03-28T16:51:49.443162+0000","last_fullsized":"2024-03-28T16:51:49.443162+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:49.442980+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:49.442980+0000","last_clean_scrub_stamp":"2024-03-28T16:51:49.442980+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:18:33.519156+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:15.399302+0000","last_change":"2024-03-28T16:51:15.399354+0000","last_active":"2024-03-28T16:51:15.399302+0000","last_peered":"2024-03-28T16:51:15.399302+0000","last_clean":"2024-03-28T16:51:15.399302+0000","last_became_active":"2024-03-28T16:48:56.696836+0000","last_became_peered":"2024-03-28T16:48:56.696836+0000","last_unstale":"2024-03-28T16:51:15.399302+0000","last_undegraded":"2024-03-28T16:51:15.399302+0000","last_fullsized":"2024-03-28T16:51:15.399302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:15.399113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:15.399113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:12.560237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.016129+0000","last_change":"2024-03-28T16:51:37.016182+0000","last_active":"2024-03-28T16:51:37.016129+0000","last_peered":"2024-03-28T16:51:37.016129+0000","last_clean":"2024-03-28T16:51:37.016129+0000","last_became_active":"2024-03-28T16:48:54.667665+0000","last_became_peered":"2024-03-28T16:48:54.667665+0000","last_unstale":"2024-03-28T16:51:37.016129+0000","last_undegraded":"2024-03-28T16:51:37.016129+0000","last_fullsized":"2024-03-28T16:51:37.016129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.015919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.015919+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.015919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:14.095891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:09.409622+0000","last_change":"2024-03-28T16:51:09.409674+0000","last_active":"2024-03-28T16:51:09.409622+0000","last_peered":"2024-03-28T16:51:09.409622+0000","last_clean":"2024-03-28T16:51:09.409622+0000","last_became_active":"2024-03-28T16:48:58.682126+0000","last_became_peered":"2024-03-28T16:48:58.682126+0000","last_unstale":"2024-03-28T16:51:09.409622+0000","last_undegraded":"2024-03-28T16:51:09.409622+0000","last_fullsized":"2024-03-28T16:51:09.409622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:09.409427+0000","last_clean_scrub_stamp":"2024-03-28T16:51:09.409427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:27.456583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"31'43","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:57.182861+0000","last_change":"2024-03-28T16:50:57.182861+0000","last_active":"2024-03-28T16:50:57.182861+0000","last_peered":"2024-03-28T16:50:57.182861+0000","last_clean":"2024-03-28T16:50:57.182861+0000","last_became_active":"2024-03-28T16:49:00.715242+0000","last_became_peered":"2024-03-28T16:49:00.715242+0000","last_unstale":"2024-03-28T16:50:57.182861+0000","last_undegraded":"2024-03-28T16:50:57.182861+0000","last_fullsized":"2024-03-28T16:50:57.182861+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:57.182585+0000","last_clean_scrub_stamp":"2024-03-28T16:50:57.182585+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.262964+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:53.238776+0000","last_change":"2024-03-28T16:50:53.238776+0000","last_active":"2024-03-28T16:50:53.238776+0000","last_peered":"2024-03-28T16:50:53.238776+0000","last_clean":"2024-03-28T16:50:53.238776+0000","last_became_active":"2024-03-28T16:49:02.730913+0000","last_became_peered":"2024-03-28T16:49:02.730913+0000","last_unstale":"2024-03-28T16:50:53.238776+0000","last_undegraded":"2024-03-28T16:50:53.238776+0000","last_fullsized":"2024-03-28T16:50:53.238776+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:53.238528+0000","last_clean_scrub_stamp":"2024-03-28T16:50:53.238528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:38:50.127619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"31'10","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:05.941415+0000","last_change":"2024-03-28T16:52:05.941484+0000","last_active":"2024-03-28T16:52:05.941415+0000","last_peered":"2024-03-28T16:52:05.941415+0000","last_clean":"2024-03-28T16:52:05.941415+0000","last_became_active":"2024-03-28T16:49:04.753809+0000","last_became_peered":"2024-03-28T16:49:04.753809+0000","last_unstale":"2024-03-28T16:52:05.941415+0000","last_undegraded":"2024-03-28T16:52:05.941415+0000","last_fullsized":"2024-03-28T16:52:05.941415+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:52:05.941209+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:52:05.941209+0000","last_clean_scrub_stamp":"2024-03-28T16:52:05.941209+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:38:27.650509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":60,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.175511+0000","last_change":"2024-03-28T16:51:02.175511+0000","last_active":"2024-03-28T16:51:02.175511+0000","last_peered":"2024-03-28T16:51:02.175511+0000","last_clean":"2024-03-28T16:51:02.175511+0000","last_became_active":"2024-03-28T16:48:56.694619+0000","last_became_peered":"2024-03-28T16:48:56.694619+0000","last_unstale":"2024-03-28T16:51:02.175511+0000","last_undegraded":"2024-03-28T16:51:02.175511+0000","last_fullsized":"2024-03-28T16:51:02.175511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:02.175255+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.175255+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:28.260113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.155413+0000","last_change":"2024-03-28T16:51:32.155474+0000","last_active":"2024-03-28T16:51:32.155413+0000","last_peered":"2024-03-28T16:51:32.155413+0000","last_clean":"2024-03-28T16:51:32.155413+0000","last_became_active":"2024-03-28T16:48:54.653526+0000","last_became_peered":"2024-03-28T16:48:54.653526+0000","last_unstale":"2024-03-28T16:51:32.155413+0000","last_undegraded":"2024-03-28T16:51:32.155413+0000","last_fullsized":"2024-03-28T16:51:32.155413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.155231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.155231+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.155231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:47:35.685453+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.228518+0000","last_change":"2024-03-28T16:51:38.228577+0000","last_active":"2024-03-28T16:51:38.228518+0000","last_peered":"2024-03-28T16:51:38.228518+0000","last_clean":"2024-03-28T16:51:38.228518+0000","last_became_active":"2024-03-28T16:49:37.609889+0000","last_became_peered":"2024-03-28T16:49:37.609889+0000","last_unstale":"2024-03-28T16:51:38.228518+0000","last_undegraded":"2024-03-28T16:51:38.228518+0000","last_fullsized":"2024-03-28T16:51:38.228518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.228325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.228325+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.228325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:56.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"31'39","reported_seq":95,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.342055+0000","last_change":"2024-03-28T16:50:58.342055+0000","last_active":"2024-03-28T16:50:58.342055+0000","last_peered":"2024-03-28T16:50:58.342055+0000","last_clean":"2024-03-28T16:50:58.342055+0000","last_became_active":"2024-03-28T16:49:00.706220+0000","last_became_peered":"2024-03-28T16:49:00.706220+0000","last_unstale":"2024-03-28T16:50:58.342055+0000","last_undegraded":"2024-03-28T16:50:58.342055+0000","last_fullsized":"2024-03-28T16:50:58.342055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'39","last_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_deep_scrub":"31'39","last_deep_scrub_stamp":"2024-03-28T16:50:58.341815+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.341815+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:35.862752+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.375363+0000","last_change":"2024-03-28T16:51:05.375363+0000","last_active":"2024-03-28T16:51:05.375363+0000","last_peered":"2024-03-28T16:51:05.375363+0000","last_clean":"2024-03-28T16:51:05.375363+0000","last_became_active":"2024-03-28T16:49:02.733574+0000","last_became_peered":"2024-03-28T16:49:02.733574+0000","last_unstale":"2024-03-28T16:51:05.375363+0000","last_undegraded":"2024-03-28T16:51:05.375363+0000","last_fullsized":"2024-03-28T16:51:05.375363+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:05.375113+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.375113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:59:40.169672+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"31'2","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.118142+0000","last_change":"2024-03-28T16:51:29.118208+0000","last_active":"2024-03-28T16:51:29.118142+0000","last_peered":"2024-03-28T16:51:29.118142+0000","last_clean":"2024-03-28T16:51:29.118142+0000","last_became_active":"2024-03-28T16:49:04.746951+0000","last_became_peered":"2024-03-28T16:49:04.746951+0000","last_unstale":"2024-03-28T16:51:29.118142+0000","last_undegraded":"2024-03-28T16:51:29.118142+0000","last_fullsized":"2024-03-28T16:51:29.118142+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:29.117944+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:29.117944+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.117944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:12.949724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.562342+0000","last_change":"2024-03-28T16:51:29.562487+0000","last_active":"2024-03-28T16:51:29.562342+0000","last_peered":"2024-03-28T16:51:29.562342+0000","last_clean":"2024-03-28T16:51:29.562342+0000","last_became_active":"2024-03-28T16:48:56.681440+0000","last_became_peered":"2024-03-28T16:48:56.681440+0000","last_unstale":"2024-03-28T16:51:29.562342+0000","last_undegraded":"2024-03-28T16:51:29.562342+0000","last_fullsized":"2024-03-28T16:51:29.562342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:29.562080+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:29.562080+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.562080+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:14:48.297089+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:53.057583+0000","last_change":"2024-03-28T16:51:53.057636+0000","last_active":"2024-03-28T16:51:53.057583+0000","last_peered":"2024-03-28T16:51:53.057583+0000","last_clean":"2024-03-28T16:51:53.057583+0000","last_became_active":"2024-03-28T16:48:54.671954+0000","last_became_peered":"2024-03-28T16:48:54.671954+0000","last_unstale":"2024-03-28T16:51:53.057583+0000","last_undegraded":"2024-03-28T16:51:53.057583+0000","last_fullsized":"2024-03-28T16:51:53.057583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:53.057385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:53.057385+0000","last_clean_scrub_stamp":"2024-03-28T16:51:53.057385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:22:56.914997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:04.303980+0000","last_change":"2024-03-28T16:51:04.303980+0000","last_active":"2024-03-28T16:51:04.303980+0000","last_peered":"2024-03-28T16:51:04.303980+0000","last_clean":"2024-03-28T16:51:04.303980+0000","last_became_active":"2024-03-28T16:48:58.681112+0000","last_became_peered":"2024-03-28T16:48:58.681112+0000","last_unstale":"2024-03-28T16:51:04.303980+0000","last_undegraded":"2024-03-28T16:51:04.303980+0000","last_fullsized":"2024-03-28T16:51:04.303980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:04.303737+0000","last_clean_scrub_stamp":"2024-03-28T16:51:04.303737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:00:22.080932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'15","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:01.229100+0000","last_change":"2024-03-28T16:51:01.229100+0000","last_active":"2024-03-28T16:51:01.229100+0000","last_peered":"2024-03-28T16:51:01.229100+0000","last_clean":"2024-03-28T16:51:01.229100+0000","last_became_active":"2024-03-28T16:49:00.719573+0000","last_became_peered":"2024-03-28T16:49:00.719573+0000","last_unstale":"2024-03-28T16:51:01.229100+0000","last_undegraded":"2024-03-28T16:51:01.229100+0000","last_fullsized":"2024-03-28T16:51:01.229100+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'15","last_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_deep_scrub":"31'15","last_deep_scrub_stamp":"2024-03-28T16:51:01.228821+0000","last_clean_scrub_stamp":"2024-03-28T16:51:01.228821+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:05:13.034030+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"31'249","reported_seq":310,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:52.440393+0000","last_change":"2024-03-28T16:50:52.440393+0000","last_active":"2024-03-28T16:50:52.440393+0000","last_peered":"2024-03-28T16:50:52.440393+0000","last_clean":"2024-03-28T16:50:52.440393+0000","last_became_active":"2024-03-28T16:49:02.734131+0000","last_became_peered":"2024-03-28T16:49:02.734131+0000","last_unstale":"2024-03-28T16:50:52.440393+0000","last_undegraded":"2024-03-28T16:50:52.440393+0000","last_fullsized":"2024-03-28T16:50:52.440393+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'249","last_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_deep_scrub":"31'249","last_deep_scrub_stamp":"2024-03-28T16:50:52.440130+0000","last_clean_scrub_stamp":"2024-03-28T16:50:52.440130+0000","objects_scrubbed":1,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:54:49.215852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:33.089330+0000","last_change":"2024-03-28T16:51:33.089387+0000","last_active":"2024-03-28T16:51:33.089330+0000","last_peered":"2024-03-28T16:51:33.089330+0000","last_clean":"2024-03-28T16:51:33.089330+0000","last_became_active":"2024-03-28T16:49:04.747356+0000","last_became_peered":"2024-03-28T16:49:04.747356+0000","last_unstale":"2024-03-28T16:51:33.089330+0000","last_undegraded":"2024-03-28T16:51:33.089330+0000","last_fullsized":"2024-03-28T16:51:33.089330+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:33.089137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:33.089137+0000","last_clean_scrub_stamp":"2024-03-28T16:51:33.089137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:01:20.214432+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"31'6","reported_seq":57,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:20.352050+0000","last_change":"2024-03-28T16:51:20.352050+0000","last_active":"2024-03-28T16:51:20.352050+0000","last_peered":"2024-03-28T16:51:20.352050+0000","last_clean":"2024-03-28T16:51:20.352050+0000","last_became_active":"2024-03-28T16:48:56.684784+0000","last_became_peered":"2024-03-28T16:48:56.684784+0000","last_unstale":"2024-03-28T16:51:20.352050+0000","last_undegraded":"2024-03-28T16:51:20.352050+0000","last_fullsized":"2024-03-28T16:51:20.352050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:20.351803+0000","last_clean_scrub_stamp":"2024-03-28T16:51:20.351803+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:59:25.271104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.050474+0000","last_change":"2024-03-28T16:51:36.050529+0000","last_active":"2024-03-28T16:51:36.050474+0000","last_peered":"2024-03-28T16:51:36.050474+0000","last_clean":"2024-03-28T16:51:36.050474+0000","last_became_active":"2024-03-28T16:48:54.666526+0000","last_became_peered":"2024-03-28T16:48:54.666526+0000","last_unstale":"2024-03-28T16:51:36.050474+0000","last_undegraded":"2024-03-28T16:51:36.050474+0000","last_fullsized":"2024-03-28T16:51:36.050474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.050281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.050281+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.050281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:46:01.325065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:02.331654+0000","last_change":"2024-03-28T16:51:02.331654+0000","last_active":"2024-03-28T16:51:02.331654+0000","last_peered":"2024-03-28T16:51:02.331654+0000","last_clean":"2024-03-28T16:51:02.331654+0000","last_became_active":"2024-03-28T16:48:58.684177+0000","last_became_peered":"2024-03-28T16:48:58.684177+0000","last_unstale":"2024-03-28T16:51:02.331654+0000","last_undegraded":"2024-03-28T16:51:02.331654+0000","last_fullsized":"2024-03-28T16:51:02.331654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:02.331416+0000","last_clean_scrub_stamp":"2024-03-28T16:51:02.331416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:20.378572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"31'43","reported_seq":101,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.202390+0000","last_change":"2024-03-28T16:50:50.202390+0000","last_active":"2024-03-28T16:50:50.202390+0000","last_peered":"2024-03-28T16:50:50.202390+0000","last_clean":"2024-03-28T16:50:50.202390+0000","last_became_active":"2024-03-28T16:49:00.721346+0000","last_became_peered":"2024-03-28T16:49:00.721346+0000","last_unstale":"2024-03-28T16:50:50.202390+0000","last_undegraded":"2024-03-28T16:50:50.202390+0000","last_fullsized":"2024-03-28T16:50:50.202390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'43","last_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_deep_scrub":"31'43","last_deep_scrub_stamp":"2024-03-28T16:50:50.202141+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.202141+0000","objects_scrubbed":7,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:12.086676+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:48.276938+0000","last_change":"2024-03-28T16:50:48.276938+0000","last_active":"2024-03-28T16:50:48.276938+0000","last_peered":"2024-03-28T16:50:48.276938+0000","last_clean":"2024-03-28T16:50:48.276938+0000","last_became_active":"2024-03-28T16:49:02.713019+0000","last_became_peered":"2024-03-28T16:49:02.713019+0000","last_unstale":"2024-03-28T16:50:48.276938+0000","last_undegraded":"2024-03-28T16:50:48.276938+0000","last_fullsized":"2024-03-28T16:50:48.276938+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:48.276698+0000","last_clean_scrub_stamp":"2024-03-28T16:50:48.276698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:29.219774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:43.038235+0000","last_change":"2024-03-28T16:51:43.038288+0000","last_active":"2024-03-28T16:51:43.038235+0000","last_peered":"2024-03-28T16:51:43.038235+0000","last_clean":"2024-03-28T16:51:43.038235+0000","last_became_active":"2024-03-28T16:49:04.744580+0000","last_became_peered":"2024-03-28T16:49:04.744580+0000","last_unstale":"2024-03-28T16:51:43.038235+0000","last_undegraded":"2024-03-28T16:51:43.038235+0000","last_fullsized":"2024-03-28T16:51:43.038235+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:43.038045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:43.038045+0000","last_clean_scrub_stamp":"2024-03-28T16:51:43.038045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:19:50.550015+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"31'4","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.310265+0000","last_change":"2024-03-28T16:51:24.310265+0000","last_active":"2024-03-28T16:51:24.310265+0000","last_peered":"2024-03-28T16:51:24.310265+0000","last_clean":"2024-03-28T16:51:24.310265+0000","last_became_active":"2024-03-28T16:48:56.689926+0000","last_became_peered":"2024-03-28T16:48:56.689926+0000","last_unstale":"2024-03-28T16:51:24.310265+0000","last_undegraded":"2024-03-28T16:51:24.310265+0000","last_fullsized":"2024-03-28T16:51:24.310265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:24.310025+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.310025+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:21.471148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:43.004934+0000","last_change":"2024-03-28T16:51:43.004997+0000","last_active":"2024-03-28T16:51:43.004934+0000","last_peered":"2024-03-28T16:51:43.004934+0000","last_clean":"2024-03-28T16:51:43.004934+0000","last_became_active":"2024-03-28T16:48:54.669508+0000","last_became_peered":"2024-03-28T16:48:54.669508+0000","last_unstale":"2024-03-28T16:51:43.004934+0000","last_undegraded":"2024-03-28T16:51:43.004934+0000","last_fullsized":"2024-03-28T16:51:43.004934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:43.004709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:43.004709+0000","last_clean_scrub_stamp":"2024-03-28T16:51:43.004709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:28:48.991119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:03.396495+0000","last_change":"2024-03-28T16:51:03.396495+0000","last_active":"2024-03-28T16:51:03.396495+0000","last_peered":"2024-03-28T16:51:03.396495+0000","last_clean":"2024-03-28T16:51:03.396495+0000","last_became_active":"2024-03-28T16:48:58.691104+0000","last_became_peered":"2024-03-28T16:48:58.691104+0000","last_unstale":"2024-03-28T16:51:03.396495+0000","last_undegraded":"2024-03-28T16:51:03.396495+0000","last_fullsized":"2024-03-28T16:51:03.396495+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:03.396253+0000","last_clean_scrub_stamp":"2024-03-28T16:51:03.396253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:52:26.295446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"31'34","reported_seq":89,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.307759+0000","last_change":"2024-03-28T16:50:59.307759+0000","last_active":"2024-03-28T16:50:59.307759+0000","last_peered":"2024-03-28T16:50:59.307759+0000","last_clean":"2024-03-28T16:50:59.307759+0000","last_became_active":"2024-03-28T16:49:00.705931+0000","last_became_peered":"2024-03-28T16:49:00.705931+0000","last_unstale":"2024-03-28T16:50:59.307759+0000","last_undegraded":"2024-03-28T16:50:59.307759+0000","last_fullsized":"2024-03-28T16:50:59.307759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'34","last_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_deep_scrub":"31'34","last_deep_scrub_stamp":"2024-03-28T16:50:59.307494+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.307494+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:28.146893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.090891+0000","last_change":"2024-03-28T16:51:13.090891+0000","last_active":"2024-03-28T16:51:13.090891+0000","last_peered":"2024-03-28T16:51:13.090891+0000","last_clean":"2024-03-28T16:51:13.090891+0000","last_became_active":"2024-03-28T16:49:02.737825+0000","last_became_peered":"2024-03-28T16:49:02.737825+0000","last_unstale":"2024-03-28T16:51:13.090891+0000","last_undegraded":"2024-03-28T16:51:13.090891+0000","last_fullsized":"2024-03-28T16:51:13.090891+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:13.090537+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.090537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:44:11.869557+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:08.894376+0000","last_change":"2024-03-28T16:52:08.894448+0000","last_active":"2024-03-28T16:52:08.894376+0000","last_peered":"2024-03-28T16:52:08.894376+0000","last_clean":"2024-03-28T16:52:08.894376+0000","last_became_active":"2024-03-28T16:49:04.761016+0000","last_became_peered":"2024-03-28T16:49:04.761016+0000","last_unstale":"2024-03-28T16:52:08.894376+0000","last_undegraded":"2024-03-28T16:52:08.894376+0000","last_fullsized":"2024-03-28T16:52:08.894376+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:08.894150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:08.894150+0000","last_clean_scrub_stamp":"2024-03-28T16:52:08.894150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:23:06.083725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:35.064823+0000","last_change":"2024-03-28T16:51:35.064895+0000","last_active":"2024-03-28T16:51:35.064823+0000","last_peered":"2024-03-28T16:51:35.064823+0000","last_clean":"2024-03-28T16:51:35.064823+0000","last_became_active":"2024-03-28T16:48:56.665877+0000","last_became_peered":"2024-03-28T16:48:56.665877+0000","last_unstale":"2024-03-28T16:51:35.064823+0000","last_undegraded":"2024-03-28T16:51:35.064823+0000","last_fullsized":"2024-03-28T16:51:35.064823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:35.064632+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:35.064632+0000","last_clean_scrub_stamp":"2024-03-28T16:51:35.064632+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:17:41.776672+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.598680+0000","last_change":"2024-03-28T16:51:24.598733+0000","last_active":"2024-03-28T16:51:24.598680+0000","last_peered":"2024-03-28T16:51:24.598680+0000","last_clean":"2024-03-28T16:51:24.598680+0000","last_became_active":"2024-03-28T16:48:54.665345+0000","last_became_peered":"2024-03-28T16:48:54.665345+0000","last_unstale":"2024-03-28T16:51:24.598680+0000","last_undegraded":"2024-03-28T16:51:24.598680+0000","last_fullsized":"2024-03-28T16:51:24.598680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:24.598488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:24.598488+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.598488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:05:10.818690+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.256599+0000","last_change":"2024-03-28T16:50:59.256599+0000","last_active":"2024-03-28T16:50:59.256599+0000","last_peered":"2024-03-28T16:50:59.256599+0000","last_clean":"2024-03-28T16:50:59.256599+0000","last_became_active":"2024-03-28T16:48:58.687306+0000","last_became_peered":"2024-03-28T16:48:58.687306+0000","last_unstale":"2024-03-28T16:50:59.256599+0000","last_undegraded":"2024-03-28T16:50:59.256599+0000","last_fullsized":"2024-03-28T16:50:59.256599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:59.256359+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.256359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:34.399651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"31'28","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:59.213439+0000","last_change":"2024-03-28T16:50:59.213439+0000","last_active":"2024-03-28T16:50:59.213439+0000","last_peered":"2024-03-28T16:50:59.213439+0000","last_clean":"2024-03-28T16:50:59.213439+0000","last_became_active":"2024-03-28T16:49:00.714587+0000","last_became_peered":"2024-03-28T16:49:00.714587+0000","last_unstale":"2024-03-28T16:50:59.213439+0000","last_undegraded":"2024-03-28T16:50:59.213439+0000","last_fullsized":"2024-03-28T16:50:59.213439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'28","last_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_deep_scrub":"31'28","last_deep_scrub_stamp":"2024-03-28T16:50:59.213187+0000","last_clean_scrub_stamp":"2024-03-28T16:50:59.213187+0000","objects_scrubbed":2,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:38.889215+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:17.080305+0000","last_change":"2024-03-28T16:51:17.080305+0000","last_active":"2024-03-28T16:51:17.080305+0000","last_peered":"2024-03-28T16:51:17.080305+0000","last_clean":"2024-03-28T16:51:17.080305+0000","last_became_active":"2024-03-28T16:49:02.737748+0000","last_became_peered":"2024-03-28T16:49:02.737748+0000","last_unstale":"2024-03-28T16:51:17.080305+0000","last_undegraded":"2024-03-28T16:51:17.080305+0000","last_fullsized":"2024-03-28T16:51:17.080305+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:17.080049+0000","last_clean_scrub_stamp":"2024-03-28T16:51:17.080049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:12.040592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'2","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:07.933322+0000","last_change":"2024-03-28T16:52:07.933389+0000","last_active":"2024-03-28T16:52:07.933322+0000","last_peered":"2024-03-28T16:52:07.933322+0000","last_clean":"2024-03-28T16:52:07.933322+0000","last_became_active":"2024-03-28T16:49:04.760227+0000","last_became_peered":"2024-03-28T16:49:04.760227+0000","last_unstale":"2024-03-28T16:52:07.933322+0000","last_undegraded":"2024-03-28T16:52:07.933322+0000","last_fullsized":"2024-03-28T16:52:07.933322+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:52:07.933085+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:52:07.933085+0000","last_clean_scrub_stamp":"2024-03-28T16:52:07.933085+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:26:54.429022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"31'12","reported_seq":66,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:13.347902+0000","last_change":"2024-03-28T16:51:13.347902+0000","last_active":"2024-03-28T16:51:13.347902+0000","last_peered":"2024-03-28T16:51:13.347902+0000","last_clean":"2024-03-28T16:51:13.347902+0000","last_became_active":"2024-03-28T16:48:56.677785+0000","last_became_peered":"2024-03-28T16:48:56.677785+0000","last_unstale":"2024-03-28T16:51:13.347902+0000","last_undegraded":"2024-03-28T16:51:13.347902+0000","last_fullsized":"2024-03-28T16:51:13.347902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'12","last_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_deep_scrub":"31'12","last_deep_scrub_stamp":"2024-03-28T16:51:13.347663+0000","last_clean_scrub_stamp":"2024-03-28T16:51:13.347663+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:39.911911+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:34.128855+0000","last_change":"2024-03-28T16:51:34.128912+0000","last_active":"2024-03-28T16:51:34.128855+0000","last_peered":"2024-03-28T16:51:34.128855+0000","last_clean":"2024-03-28T16:51:34.128855+0000","last_became_active":"2024-03-28T16:48:54.654236+0000","last_became_peered":"2024-03-28T16:48:54.654236+0000","last_unstale":"2024-03-28T16:51:34.128855+0000","last_undegraded":"2024-03-28T16:51:34.128855+0000","last_fullsized":"2024-03-28T16:51:34.128855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:34.128651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:34.128651+0000","last_clean_scrub_stamp":"2024-03-28T16:51:34.128651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:18.350401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"31'3","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:58.215895+0000","last_change":"2024-03-28T16:50:58.215895+0000","last_active":"2024-03-28T16:50:58.215895+0000","last_peered":"2024-03-28T16:50:58.215895+0000","last_clean":"2024-03-28T16:50:58.215895+0000","last_became_active":"2024-03-28T16:48:58.696875+0000","last_became_peered":"2024-03-28T16:48:58.696875+0000","last_unstale":"2024-03-28T16:50:58.215895+0000","last_undegraded":"2024-03-28T16:50:58.215895+0000","last_fullsized":"2024-03-28T16:50:58.215895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-03-28T16:50:58.215651+0000","last_clean_scrub_stamp":"2024-03-28T16:50:58.215651+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:42.972682+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"31'35","reported_seq":90,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:05.272061+0000","last_change":"2024-03-28T16:51:05.272061+0000","last_active":"2024-03-28T16:51:05.272061+0000","last_peered":"2024-03-28T16:51:05.272061+0000","last_clean":"2024-03-28T16:51:05.272061+0000","last_became_active":"2024-03-28T16:49:00.719562+0000","last_became_peered":"2024-03-28T16:49:00.719562+0000","last_unstale":"2024-03-28T16:51:05.272061+0000","last_undegraded":"2024-03-28T16:51:05.272061+0000","last_fullsized":"2024-03-28T16:51:05.272061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'35","last_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_deep_scrub":"31'35","last_deep_scrub_stamp":"2024-03-28T16:51:05.271785+0000","last_clean_scrub_stamp":"2024-03-28T16:51:05.271785+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:01.679261+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1150,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:50:50.209342+0000","last_change":"2024-03-28T16:50:50.209342+0000","last_active":"2024-03-28T16:50:50.209342+0000","last_peered":"2024-03-28T16:50:50.209342+0000","last_clean":"2024-03-28T16:50:50.209342+0000","last_became_active":"2024-03-28T16:49:02.712424+0000","last_became_peered":"2024-03-28T16:49:02.712424+0000","last_unstale":"2024-03-28T16:50:50.209342+0000","last_undegraded":"2024-03-28T16:50:50.209342+0000","last_fullsized":"2024-03-28T16:50:50.209342+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:50:50.209010+0000","last_clean_scrub_stamp":"2024-03-28T16:50:50.209010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:57:45.672112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":26,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:04.977128+0000","last_change":"2024-03-28T16:52:04.977234+0000","last_active":"2024-03-28T16:52:04.977128+0000","last_peered":"2024-03-28T16:52:04.977128+0000","last_clean":"2024-03-28T16:52:04.977128+0000","last_became_active":"2024-03-28T16:49:04.750653+0000","last_became_peered":"2024-03-28T16:49:04.750653+0000","last_unstale":"2024-03-28T16:52:04.977128+0000","last_undegraded":"2024-03-28T16:52:04.977128+0000","last_fullsized":"2024-03-28T16:52:04.977128+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:52:04.976919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:52:04.976919+0000","last_clean_scrub_stamp":"2024-03-28T16:52:04.976919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:15:49.081133+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"29'2","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:41.212741+0000","last_change":"2024-03-28T16:51:41.212741+0000","last_active":"2024-03-28T16:51:41.212741+0000","last_peered":"2024-03-28T16:51:41.212741+0000","last_clean":"2024-03-28T16:51:41.212741+0000","last_became_active":"2024-03-28T16:49:37.609599+0000","last_became_peered":"2024-03-28T16:49:37.609599+0000","last_unstale":"2024-03-28T16:51:41.212741+0000","last_undegraded":"2024-03-28T16:51:41.212741+0000","last_fullsized":"2024-03-28T16:51:41.212741+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:41.212399+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:41.212399+0000","last_clean_scrub_stamp":"2024-03-28T16:51:41.212399+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:25.434150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.209561+0000","last_change":"2024-03-28T16:51:32.209624+0000","last_active":"2024-03-28T16:51:32.209561+0000","last_peered":"2024-03-28T16:51:32.209561+0000","last_clean":"2024-03-28T16:51:32.209561+0000","last_became_active":"2024-03-28T16:48:54.657516+0000","last_became_peered":"2024-03-28T16:48:54.657516+0000","last_unstale":"2024-03-28T16:51:32.209561+0000","last_undegraded":"2024-03-28T16:51:32.209561+0000","last_fullsized":"2024-03-28T16:51:32.209561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.209381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.209381+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.209381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:58.294239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"31'4","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.553584+0000","last_change":"2024-03-28T16:51:27.553649+0000","last_active":"2024-03-28T16:51:27.553584+0000","last_peered":"2024-03-28T16:51:27.553584+0000","last_clean":"2024-03-28T16:51:27.553584+0000","last_became_active":"2024-03-28T16:48:56.681540+0000","last_became_peered":"2024-03-28T16:48:56.681540+0000","last_unstale":"2024-03-28T16:51:27.553584+0000","last_undegraded":"2024-03-28T16:51:27.553584+0000","last_fullsized":"2024-03-28T16:51:27.553584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:27.553385+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:27.553385+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.553385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:24:32.733520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.448938+0000","last_change":"2024-03-28T16:51:37.449001+0000","last_active":"2024-03-28T16:51:37.448938+0000","last_peered":"2024-03-28T16:51:37.448938+0000","last_clean":"2024-03-28T16:51:37.448938+0000","last_became_active":"2024-03-28T16:48:54.654494+0000","last_became_peered":"2024-03-28T16:48:54.654494+0000","last_unstale":"2024-03-28T16:51:37.448938+0000","last_undegraded":"2024-03-28T16:51:37.448938+0000","last_fullsized":"2024-03-28T16:51:37.448938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:37.448740+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:37.448740+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.448740+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:27:45.427028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"31'4","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.141067+0000","last_change":"2024-03-28T16:51:32.141133+0000","last_active":"2024-03-28T16:51:32.141067+0000","last_peered":"2024-03-28T16:51:32.141067+0000","last_clean":"2024-03-28T16:51:32.141067+0000","last_became_active":"2024-03-28T16:48:56.671088+0000","last_became_peered":"2024-03-28T16:48:56.671088+0000","last_unstale":"2024-03-28T16:51:32.141067+0000","last_undegraded":"2024-03-28T16:51:32.141067+0000","last_fullsized":"2024-03-28T16:51:32.141067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:32.140882+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:32.140882+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.140882+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:30.700161+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:59.956831+0000","last_change":"2024-03-28T16:51:59.956889+0000","last_active":"2024-03-28T16:51:59.956831+0000","last_peered":"2024-03-28T16:51:59.956831+0000","last_clean":"2024-03-28T16:51:59.956831+0000","last_became_active":"2024-03-28T16:48:54.656463+0000","last_became_peered":"2024-03-28T16:48:54.656463+0000","last_unstale":"2024-03-28T16:51:59.956831+0000","last_undegraded":"2024-03-28T16:51:59.956831+0000","last_fullsized":"2024-03-28T16:51:59.956831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:59.956600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:59.956600+0000","last_clean_scrub_stamp":"2024-03-28T16:51:59.956600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:55:19.674683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:31.497847+0000","last_change":"2024-03-28T16:51:31.497957+0000","last_active":"2024-03-28T16:51:31.497847+0000","last_peered":"2024-03-28T16:51:31.497847+0000","last_clean":"2024-03-28T16:51:31.497847+0000","last_became_active":"2024-03-28T16:48:56.681596+0000","last_became_peered":"2024-03-28T16:48:56.681596+0000","last_unstale":"2024-03-28T16:51:31.497847+0000","last_undegraded":"2024-03-28T16:51:31.497847+0000","last_fullsized":"2024-03-28T16:51:31.497847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:31.497632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:31.497632+0000","last_clean_scrub_stamp":"2024-03-28T16:51:31.497632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:23.506227+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"31'2","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:26.564274+0000","last_change":"2024-03-28T16:51:26.564336+0000","last_active":"2024-03-28T16:51:26.564274+0000","last_peered":"2024-03-28T16:51:26.564274+0000","last_clean":"2024-03-28T16:51:26.564274+0000","last_became_active":"2024-03-28T16:48:54.665175+0000","last_became_peered":"2024-03-28T16:48:54.665175+0000","last_unstale":"2024-03-28T16:51:26.564274+0000","last_undegraded":"2024-03-28T16:51:26.564274+0000","last_fullsized":"2024-03-28T16:51:26.564274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:26.564092+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:26.564092+0000","last_clean_scrub_stamp":"2024-03-28T16:51:26.564092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:44.943347+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.523436+0000","last_change":"2024-03-28T16:51:21.523575+0000","last_active":"2024-03-28T16:51:21.523436+0000","last_peered":"2024-03-28T16:51:21.523436+0000","last_clean":"2024-03-28T16:51:21.523436+0000","last_became_active":"2024-03-28T16:48:56.676726+0000","last_became_peered":"2024-03-28T16:48:56.676726+0000","last_unstale":"2024-03-28T16:51:21.523436+0000","last_undegraded":"2024-03-28T16:51:21.523436+0000","last_fullsized":"2024-03-28T16:51:21.523436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:21.523213+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.523213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:56.404435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:38.467825+0000","last_change":"2024-03-28T16:51:38.467878+0000","last_active":"2024-03-28T16:51:38.467825+0000","last_peered":"2024-03-28T16:51:38.467825+0000","last_clean":"2024-03-28T16:51:38.467825+0000","last_became_active":"2024-03-28T16:48:54.654620+0000","last_became_peered":"2024-03-28T16:48:54.654620+0000","last_unstale":"2024-03-28T16:51:38.467825+0000","last_undegraded":"2024-03-28T16:51:38.467825+0000","last_fullsized":"2024-03-28T16:51:38.467825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:38.467639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:38.467639+0000","last_clean_scrub_stamp":"2024-03-28T16:51:38.467639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:55:12.097451+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"31'6","reported_seq":71,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:19.278285+0000","last_change":"2024-03-28T16:51:19.278345+0000","last_active":"2024-03-28T16:51:19.278285+0000","last_peered":"2024-03-28T16:51:19.278285+0000","last_clean":"2024-03-28T16:51:19.278285+0000","last_became_active":"2024-03-28T16:48:56.673730+0000","last_became_peered":"2024-03-28T16:48:56.673730+0000","last_unstale":"2024-03-28T16:51:19.278285+0000","last_undegraded":"2024-03-28T16:51:19.278285+0000","last_fullsized":"2024-03-28T16:51:19.278285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:19.278071+0000","last_clean_scrub_stamp":"2024-03-28T16:51:19.278071+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:29.679472+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:37.988224+0000","last_change":"2024-03-28T16:51:37.988292+0000","last_active":"2024-03-28T16:51:37.988224+0000","last_peered":"2024-03-28T16:51:37.988224+0000","last_clean":"2024-03-28T16:51:37.988224+0000","last_became_active":"2024-03-28T16:48:54.668579+0000","last_became_peered":"2024-03-28T16:48:54.668579+0000","last_unstale":"2024-03-28T16:51:37.988224+0000","last_undegraded":"2024-03-28T16:51:37.988224+0000","last_fullsized":"2024-03-28T16:51:37.988224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:37.988041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:37.988041+0000","last_clean_scrub_stamp":"2024-03-28T16:51:37.988041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:25:35.010568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.446697+0000","last_change":"2024-03-28T16:51:18.446757+0000","last_active":"2024-03-28T16:51:18.446697+0000","last_peered":"2024-03-28T16:51:18.446697+0000","last_clean":"2024-03-28T16:51:18.446697+0000","last_became_active":"2024-03-28T16:48:56.689063+0000","last_became_peered":"2024-03-28T16:48:56.689063+0000","last_unstale":"2024-03-28T16:51:18.446697+0000","last_undegraded":"2024-03-28T16:51:18.446697+0000","last_fullsized":"2024-03-28T16:51:18.446697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:18.446501+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.446501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:51.526879+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:25.283525+0000","last_change":"2024-03-28T16:51:25.283581+0000","last_active":"2024-03-28T16:51:25.283525+0000","last_peered":"2024-03-28T16:51:25.283525+0000","last_clean":"2024-03-28T16:51:25.283525+0000","last_became_active":"2024-03-28T16:48:54.660654+0000","last_became_peered":"2024-03-28T16:48:54.660654+0000","last_unstale":"2024-03-28T16:51:25.283525+0000","last_undegraded":"2024-03-28T16:51:25.283525+0000","last_fullsized":"2024-03-28T16:51:25.283525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:25.283348+0000","last_clean_scrub_stamp":"2024-03-28T16:51:25.283348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:28.047908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:21.316649+0000","last_change":"2024-03-28T16:51:21.316736+0000","last_active":"2024-03-28T16:51:21.316649+0000","last_peered":"2024-03-28T16:51:21.316649+0000","last_clean":"2024-03-28T16:51:21.316649+0000","last_became_active":"2024-03-28T16:48:56.673400+0000","last_became_peered":"2024-03-28T16:48:56.673400+0000","last_unstale":"2024-03-28T16:51:21.316649+0000","last_undegraded":"2024-03-28T16:51:21.316649+0000","last_fullsized":"2024-03-28T16:51:21.316649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:21.316476+0000","last_clean_scrub_stamp":"2024-03-28T16:51:21.316476+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:10:20.931612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.420032+0000","last_change":"2024-03-28T16:51:40.420091+0000","last_active":"2024-03-28T16:51:40.420032+0000","last_peered":"2024-03-28T16:51:40.420032+0000","last_clean":"2024-03-28T16:51:40.420032+0000","last_became_active":"2024-03-28T16:48:54.659131+0000","last_became_peered":"2024-03-28T16:48:54.659131+0000","last_unstale":"2024-03-28T16:51:40.420032+0000","last_undegraded":"2024-03-28T16:51:40.420032+0000","last_fullsized":"2024-03-28T16:51:40.420032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:40.419842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:40.419842+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.419842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:08:10.034139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:10.289210+0000","last_change":"2024-03-28T16:51:10.289210+0000","last_active":"2024-03-28T16:51:10.289210+0000","last_peered":"2024-03-28T16:51:10.289210+0000","last_clean":"2024-03-28T16:51:10.289210+0000","last_became_active":"2024-03-28T16:48:56.683625+0000","last_became_peered":"2024-03-28T16:48:56.683625+0000","last_unstale":"2024-03-28T16:51:10.289210+0000","last_undegraded":"2024-03-28T16:51:10.289210+0000","last_fullsized":"2024-03-28T16:51:10.289210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:10.288939+0000","last_clean_scrub_stamp":"2024-03-28T16:51:10.288939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:23:52.264862+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"31'4","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:52:00.975491+0000","last_change":"2024-03-28T16:52:00.975620+0000","last_active":"2024-03-28T16:52:00.975491+0000","last_peered":"2024-03-28T16:52:00.975491+0000","last_clean":"2024-03-28T16:52:00.975491+0000","last_became_active":"2024-03-28T16:48:54.656383+0000","last_became_peered":"2024-03-28T16:48:54.656383+0000","last_unstale":"2024-03-28T16:52:00.975491+0000","last_undegraded":"2024-03-28T16:52:00.975491+0000","last_fullsized":"2024-03-28T16:52:00.975491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:52:00.975275+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:52:00.975275+0000","last_clean_scrub_stamp":"2024-03-28T16:52:00.975275+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:02:12.413542+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"31'2","reported_seq":47,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:27.144122+0000","last_change":"2024-03-28T16:51:27.144188+0000","last_active":"2024-03-28T16:51:27.144122+0000","last_peered":"2024-03-28T16:51:27.144122+0000","last_clean":"2024-03-28T16:51:27.144122+0000","last_became_active":"2024-03-28T16:48:56.682502+0000","last_became_peered":"2024-03-28T16:48:56.682502+0000","last_unstale":"2024-03-28T16:51:27.144122+0000","last_undegraded":"2024-03-28T16:51:27.144122+0000","last_fullsized":"2024-03-28T16:51:27.144122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:27.143940+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:27.143940+0000","last_clean_scrub_stamp":"2024-03-28T16:51:27.143940+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:09:52.594727+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:32.482752+0000","last_change":"2024-03-28T16:51:32.482808+0000","last_active":"2024-03-28T16:51:32.482752+0000","last_peered":"2024-03-28T16:51:32.482752+0000","last_clean":"2024-03-28T16:51:32.482752+0000","last_became_active":"2024-03-28T16:48:54.661859+0000","last_became_peered":"2024-03-28T16:48:54.661859+0000","last_unstale":"2024-03-28T16:51:32.482752+0000","last_undegraded":"2024-03-28T16:51:32.482752+0000","last_fullsized":"2024-03-28T16:51:32.482752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:32.482535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:32.482535+0000","last_clean_scrub_stamp":"2024-03-28T16:51:32.482535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:16.704285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.557168+0000","last_change":"2024-03-28T16:51:23.557231+0000","last_active":"2024-03-28T16:51:23.557168+0000","last_peered":"2024-03-28T16:51:23.557168+0000","last_clean":"2024-03-28T16:51:23.557168+0000","last_became_active":"2024-03-28T16:48:56.677215+0000","last_became_peered":"2024-03-28T16:48:56.677215+0000","last_unstale":"2024-03-28T16:51:23.557168+0000","last_undegraded":"2024-03-28T16:51:23.557168+0000","last_fullsized":"2024-03-28T16:51:23.557168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:23.556975+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:23.556975+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.556975+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:49.641832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'2","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:22.301018+0000","last_change":"2024-03-28T16:51:22.301084+0000","last_active":"2024-03-28T16:51:22.301018+0000","last_peered":"2024-03-28T16:51:22.301018+0000","last_clean":"2024-03-28T16:51:22.301018+0000","last_became_active":"2024-03-28T16:48:54.673588+0000","last_became_peered":"2024-03-28T16:48:54.673588+0000","last_unstale":"2024-03-28T16:51:22.301018+0000","last_undegraded":"2024-03-28T16:51:22.301018+0000","last_fullsized":"2024-03-28T16:51:22.301018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-03-28T16:51:22.300834+0000","last_clean_scrub_stamp":"2024-03-28T16:51:22.300834+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:39.676855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"31'4","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:18.261019+0000","last_change":"2024-03-28T16:51:18.261152+0000","last_active":"2024-03-28T16:51:18.261019+0000","last_peered":"2024-03-28T16:51:18.261019+0000","last_clean":"2024-03-28T16:51:18.261019+0000","last_became_active":"2024-03-28T16:48:56.670414+0000","last_became_peered":"2024-03-28T16:48:56.670414+0000","last_unstale":"2024-03-28T16:51:18.261019+0000","last_undegraded":"2024-03-28T16:51:18.261019+0000","last_fullsized":"2024-03-28T16:51:18.261019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:18.260778+0000","last_clean_scrub_stamp":"2024-03-28T16:51:18.260778+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:15.421901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:24.256951+0000","last_change":"2024-03-28T16:51:24.257006+0000","last_active":"2024-03-28T16:51:24.256951+0000","last_peered":"2024-03-28T16:51:24.256951+0000","last_clean":"2024-03-28T16:51:24.256951+0000","last_became_active":"2024-03-28T16:48:54.673241+0000","last_became_peered":"2024-03-28T16:48:54.673241+0000","last_unstale":"2024-03-28T16:51:24.256951+0000","last_undegraded":"2024-03-28T16:51:24.256951+0000","last_fullsized":"2024-03-28T16:51:24.256951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:24.256765+0000","last_clean_scrub_stamp":"2024-03-28T16:51:24.256765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:47.235830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"31'6","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:14.319730+0000","last_change":"2024-03-28T16:51:14.319730+0000","last_active":"2024-03-28T16:51:14.319730+0000","last_peered":"2024-03-28T16:51:14.319730+0000","last_clean":"2024-03-28T16:51:14.319730+0000","last_became_active":"2024-03-28T16:48:56.683357+0000","last_became_peered":"2024-03-28T16:48:56.683357+0000","last_unstale":"2024-03-28T16:51:14.319730+0000","last_undegraded":"2024-03-28T16:51:14.319730+0000","last_fullsized":"2024-03-28T16:51:14.319730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:14.319444+0000","last_clean_scrub_stamp":"2024-03-28T16:51:14.319444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:31:39.722762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:36.080203+0000","last_change":"2024-03-28T16:51:36.080262+0000","last_active":"2024-03-28T16:51:36.080203+0000","last_peered":"2024-03-28T16:51:36.080203+0000","last_clean":"2024-03-28T16:51:36.080203+0000","last_became_active":"2024-03-28T16:48:54.676227+0000","last_became_peered":"2024-03-28T16:48:54.676227+0000","last_unstale":"2024-03-28T16:51:36.080203+0000","last_undegraded":"2024-03-28T16:51:36.080203+0000","last_fullsized":"2024-03-28T16:51:36.080203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:36.080017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:36.080017+0000","last_clean_scrub_stamp":"2024-03-28T16:51:36.080017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:31:46.963031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"31'6","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:06.281665+0000","last_change":"2024-03-28T16:51:06.281665+0000","last_active":"2024-03-28T16:51:06.281665+0000","last_peered":"2024-03-28T16:51:06.281665+0000","last_clean":"2024-03-28T16:51:06.281665+0000","last_became_active":"2024-03-28T16:48:56.692738+0000","last_became_peered":"2024-03-28T16:48:56.692738+0000","last_unstale":"2024-03-28T16:51:06.281665+0000","last_undegraded":"2024-03-28T16:51:06.281665+0000","last_fullsized":"2024-03-28T16:51:06.281665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:06.281413+0000","last_clean_scrub_stamp":"2024-03-28T16:51:06.281413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:07:32.845661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:48.035856+0000","last_change":"2024-03-28T16:51:48.035917+0000","last_active":"2024-03-28T16:51:48.035856+0000","last_peered":"2024-03-28T16:51:48.035856+0000","last_clean":"2024-03-28T16:51:48.035856+0000","last_became_active":"2024-03-28T16:48:54.661608+0000","last_became_peered":"2024-03-28T16:48:54.661608+0000","last_unstale":"2024-03-28T16:51:48.035856+0000","last_undegraded":"2024-03-28T16:51:48.035856+0000","last_fullsized":"2024-03-28T16:51:48.035856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:48.035665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:48.035665+0000","last_clean_scrub_stamp":"2024-03-28T16:51:48.035665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:20.222230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"31'6","reported_seq":74,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.107093+0000","last_change":"2024-03-28T16:51:23.107159+0000","last_active":"2024-03-28T16:51:23.107093+0000","last_peered":"2024-03-28T16:51:23.107093+0000","last_clean":"2024-03-28T16:51:23.107093+0000","last_became_active":"2024-03-28T16:48:56.685330+0000","last_became_peered":"2024-03-28T16:48:56.685330+0000","last_unstale":"2024-03-28T16:51:23.107093+0000","last_undegraded":"2024-03-28T16:51:23.107093+0000","last_fullsized":"2024-03-28T16:51:23.107093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'6","last_scrub_stamp":"2024-03-28T16:51:23.106864+0000","last_deep_scrub":"31'6","last_deep_scrub_stamp":"2024-03-28T16:51:23.106864+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.106864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:05:31.824473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.338591+0000","last_change":"2024-03-28T16:51:23.338657+0000","last_active":"2024-03-28T16:51:23.338591+0000","last_peered":"2024-03-28T16:51:23.338591+0000","last_clean":"2024-03-28T16:51:23.338591+0000","last_became_active":"2024-03-28T16:48:54.661126+0000","last_became_peered":"2024-03-28T16:48:54.661126+0000","last_unstale":"2024-03-28T16:51:23.338591+0000","last_undegraded":"2024-03-28T16:51:23.338591+0000","last_fullsized":"2024-03-28T16:51:23.338591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:23.338388+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:37.364884+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"31'10","reported_seq":69,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:23.275847+0000","last_change":"2024-03-28T16:51:23.275918+0000","last_active":"2024-03-28T16:51:23.275847+0000","last_peered":"2024-03-28T16:51:23.275847+0000","last_clean":"2024-03-28T16:51:23.275847+0000","last_became_active":"2024-03-28T16:48:56.679813+0000","last_became_peered":"2024-03-28T16:48:56.679813+0000","last_unstale":"2024-03-28T16:51:23.275847+0000","last_undegraded":"2024-03-28T16:51:23.275847+0000","last_fullsized":"2024-03-28T16:51:23.275847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'10","last_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_deep_scrub":"31'10","last_deep_scrub_stamp":"2024-03-28T16:51:23.275672+0000","last_clean_scrub_stamp":"2024-03-28T16:51:23.275672+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:58.156894+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:40.083211+0000","last_change":"2024-03-28T16:51:40.083323+0000","last_active":"2024-03-28T16:51:40.083211+0000","last_peered":"2024-03-28T16:51:40.083211+0000","last_clean":"2024-03-28T16:51:40.083211+0000","last_became_active":"2024-03-28T16:48:54.664716+0000","last_became_peered":"2024-03-28T16:48:54.664716+0000","last_unstale":"2024-03-28T16:51:40.083211+0000","last_undegraded":"2024-03-28T16:51:40.083211+0000","last_fullsized":"2024-03-28T16:51:40.083211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:40.082968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:40.082968+0000","last_clean_scrub_stamp":"2024-03-28T16:51:40.082968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T02:49:31.036900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:42.046958+0000","last_change":"2024-03-28T16:51:42.047018+0000","last_active":"2024-03-28T16:51:42.046958+0000","last_peered":"2024-03-28T16:51:42.046958+0000","last_clean":"2024-03-28T16:51:42.046958+0000","last_became_active":"2024-03-28T16:48:54.664908+0000","last_became_peered":"2024-03-28T16:48:54.664908+0000","last_unstale":"2024-03-28T16:51:42.046958+0000","last_undegraded":"2024-03-28T16:51:42.046958+0000","last_fullsized":"2024-03-28T16:51:42.046958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:42.046771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:42.046771+0000","last_clean_scrub_stamp":"2024-03-28T16:51:42.046771+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T04:14:36.194510+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"31'4","reported_seq":52,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:16.440088+0000","last_change":"2024-03-28T16:51:16.440088+0000","last_active":"2024-03-28T16:51:16.440088+0000","last_peered":"2024-03-28T16:51:16.440088+0000","last_clean":"2024-03-28T16:51:16.440088+0000","last_became_active":"2024-03-28T16:48:56.690091+0000","last_became_peered":"2024-03-28T16:48:56.690091+0000","last_unstale":"2024-03-28T16:51:16.440088+0000","last_undegraded":"2024-03-28T16:51:16.440088+0000","last_fullsized":"2024-03-28T16:51:16.440088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'4","last_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_deep_scrub":"31'4","last_deep_scrub_stamp":"2024-03-28T16:51:16.439837+0000","last_clean_scrub_stamp":"2024-03-28T16:51:16.439837+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:32:19.204397+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:30.539510+0000","last_change":"2024-03-28T16:51:30.539564+0000","last_active":"2024-03-28T16:51:30.539510+0000","last_peered":"2024-03-28T16:51:30.539510+0000","last_clean":"2024-03-28T16:51:30.539510+0000","last_became_active":"2024-03-28T16:48:54.662771+0000","last_became_peered":"2024-03-28T16:48:54.662771+0000","last_unstale":"2024-03-28T16:51:30.539510+0000","last_undegraded":"2024-03-28T16:51:30.539510+0000","last_fullsized":"2024-03-28T16:51:30.539510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T16:51:30.539307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T16:51:30.539307+0000","last_clean_scrub_stamp":"2024-03-28T16:51:30.539307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:55:52.038876+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"31'2","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-28T16:51:29.253966+0000","last_change":"2024-03-28T16:51:29.254037+0000","last_active":"2024-03-28T16:51:29.253966+0000","last_peered":"2024-03-28T16:51:29.253966+0000","last_clean":"2024-03-28T16:51:29.253966+0000","last_became_active":"2024-03-28T16:48:56.682957+0000","last_became_peered":"2024-03-28T16:48:56.682957+0000","last_unstale":"2024-03-28T16:51:29.253966+0000","last_undegraded":"2024-03-28T16:51:29.253966+0000","last_fullsized":"2024-03-28T16:51:29.253966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-03-28T16:51:29.253788+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-03-28T16:51:29.253788+0000","last_clean_scrub_stamp":"2024-03-28T16:51:29.253788+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T03:02:14.899395+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1746,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":476,"num_read_kb":444,"num_write":314,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":339,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1992,"ondisk_log_size":1992,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":7688,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":1539,"num_read_kb":1325,"num_write":581,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1750,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1422,"ondisk_log_size":1422,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1526,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":866,"num_read_kb":864,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":956,"num_read_kb":900,"num_write":304,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":209,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":163,"num_write_kb":29309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44076,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327764,"statfs":{"total":96636764160,"available":96591630336,"internally_reserved":0,"allocated":430080,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67000000000000004,"15min":0.66800000000000004},"min":{"1min":0.58999999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.87,"5min":1.246,"15min":1.246},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82399999999999995,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66500000000000004}]},{"osd":1,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.47399999999999998,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.82299999999999995,"5min":1.9159999999999999,"15min":1.9159999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.82599999999999996,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64300000000000002,"15min":0.63900000000000001},"min":{"1min":0.51600000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.874,"5min":1.671,"15min":1.671},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.53100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94399999999999995,"5min":1.331,"15min":1.331},"last":0.69099999999999995}]}]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":124,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40472,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331368,"statfs":{"total":96636764160,"available":96595320832,"internally_reserved":0,"allocated":999424,"data_stored":721358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.64100000000000001},"min":{"1min":0.57699999999999996,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":2.98,"15min":2.98},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.64800000000000002,"15min":0.629},"min":{"1min":0.58199999999999996,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82999999999999996,"5min":1.01,"15min":1.01},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.433,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.75900000000000001,"5min":2.6280000000000001,"15min":2.6280000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57299999999999995,"15min":0.54400000000000004},"min":{"1min":0.47499999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.95899999999999996,"5min":3.0800000000000001,"15min":3.0800000000000001},"last":0.82999999999999996}]},{"osd":3,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.48099999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.76100000000000001,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.498,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.78600000000000003,"5min":2.7589999999999999,"15min":2.7589999999999999},"last":0.76400000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705712,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40484,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331356,"statfs":{"total":96636764160,"available":96595308544,"internally_reserved":0,"allocated":1011712,"data_stored":723336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.54900000000000004,"5min":0.309,"15min":0.309},"max":{"1min":0.75900000000000001,"5min":0.878,"15min":0.878},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65800000000000003,"15min":0.65100000000000002},"min":{"1min":0.57099999999999995,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.91400000000000003,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.46400000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69899999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.623},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52800000000000002,"15min":0.505},"min":{"1min":0.437,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.79600000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.496}]},{"osd":3,"last update":"Thu Mar 28 16:51:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.56999999999999995},"min":{"1min":0.47499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.76500000000000001,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66500000000000004,"15min":0.65300000000000002},"min":{"1min":0.54900000000000004,"5min":0.246,"15min":0.246},"max":{"1min":0.82399999999999995,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.81799999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705712,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39896,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331944,"statfs":{"total":96636764160,"available":96595910656,"internally_reserved":0,"allocated":409600,"data_stored":132503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 28 16:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70999999999999996,"15min":0.70899999999999996},"min":{"1min":0.503,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.88400000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.65800000000000003},"min":{"1min":0.58299999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Mar 28 16:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.53200000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58499999999999996,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Mar 28 16:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.75900000000000001},"min":{"1min":0.62,"5min":0.372,"15min":0.372},"max":{"1min":0.94199999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.61499999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89400000000000002,"5min":1.117,"15min":1.117},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T16:52:29.276 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-28T16:52:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.423+0000 7f29376e4640 1 Processor -- start 2024-03-28T16:52:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.423+0000 7f29376e4640 1 -- start start 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2930099770 0x7f2930098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2930098fa0 con 0x7f2930097f70 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2930099130 con 0x7f2930099770 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2935480640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2930097f70 0x7f2930098350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52690/0 (socket says 172.21.15.119:52690) 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2935480640 1 -- 172.21.15.119:0/2746796665 learned_addr learned my addr 172.21.15.119:0/2746796665 (peer_addr_for_me v1:172.21.15.119:0/0) 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2934c7f640 1 --2- 172.21.15.119:0/2746796665 >> v2:172.21.15.119:3300/0 conn(0x7f2930099770 0x7f2930098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2927fff640 1 -- 172.21.15.119:0/2746796665 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382343362 0 0) 0x7f2930098fa0 con 0x7f2930097f70 2024-03-28T16:52:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2927fff640 1 -- 172.21.15.119:0/2746796665 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f291c003530 con 0x7f2930097f70 2024-03-28T16:52:29.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2934c7f640 1 -- 172.21.15.119:0/2746796665 >> v1:172.21.15.119:6789/0 conn(0x7f2930097f70 legacy=0x7f2930098350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.430 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2934c7f640 1 -- 172.21.15.119:0/2746796665 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2930134520 con 0x7f2930099770 2024-03-28T16:52:29.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2934c7f640 1 --2- 172.21.15.119:0/2746796665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930099770 0x7f2930098a60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2920009740 tx=0x7f292001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05a5a5b965ea937 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:29.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2927fff640 1 -- 172.21.15.119:0/2746796665 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29200ab530 con 0x7f2930099770 2024-03-28T16:52:29.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2927fff640 1 -- 172.21.15.119:0/2746796665 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29200a7050 con 0x7f2930099770 2024-03-28T16:52:29.431 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f2927fff640 1 -- 172.21.15.119:0/2746796665 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29200ab7e0 con 0x7f2930099770 2024-03-28T16:52:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 -- 172.21.15.119:0/2746796665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930099770 msgr2=0x7f2930098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 --2- 172.21.15.119:0/2746796665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930099770 0x7f2930098a60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2920009740 tx=0x7f292001c370 comp rx=0 tx=0).stop 2024-03-28T16:52:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 -- 172.21.15.119:0/2746796665 shutdown_connections 2024-03-28T16:52:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 --2- 172.21.15.119:0/2746796665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930099770 0x7f2930098a60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 -- 172.21.15.119:0/2746796665 >> 172.21.15.119:0/2746796665 conn(0x7f293009e690 msgr2=0x7f293009eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:52:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 -- 172.21.15.119:0/2746796665 shutdown_connections 2024-03-28T16:52:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.427+0000 7f29376e4640 1 -- 172.21.15.119:0/2746796665 wait complete. 2024-03-28T16:52:29.434 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f29376e4640 1 Processor -- start 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f29376e4640 1 -- start start 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f29376e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930097f70 0x7f293008fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f29376e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29300920c0 con 0x7f2930097f70 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2935480640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930097f70 0x7f293008fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2935480640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930097f70 0x7f293008fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59788/0 (socket says 172.21.15.119:59788) 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2935480640 1 -- 172.21.15.119:0/84066390 learned_addr learned my addr 172.21.15.119:0/84066390 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2935480640 1 -- 172.21.15.119:0/84066390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920009420 con 0x7f2930097f70 2024-03-28T16:52:29.435 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2935480640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930097f70 0x7f293008fe40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c450 tx=0x7f292c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f292c005c20 con 0x7f2930097f70 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c005d80 con 0x7f2930097f70 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2930091ba0 con 0x7f2930097f70 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f292c0071a0 con 0x7f2930097f70 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2930096370 con 0x7f2930097f70 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2930091690 con 0x7f2930097f70 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43535+0+0 (secure 0 0 0) 0x7f292c007300 con 0x7f2930097f70 2024-03-28T16:52:29.436 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2925ffb640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2914035a10 0x7f2914037ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T16:52:29.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f292c047c10 con 0x7f2930097f70 2024-03-28T16:52:29.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.431+0000 7f2934c7f640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2914035a10 0x7f2914037ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T16:52:29.437 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.435+0000 7f2934c7f640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2914035a10 0x7f2914037ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f29200ad000 tx=0x7f2920008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T16:52:29.438 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.435+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f292c012400 con 0x7f2930097f70 2024-03-28T16:52:29.586 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.583+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f2930062860 con 0x7f2930097f70 2024-03-28T16:52:29.587 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.583+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f292c017030 con 0x7f2930097f70 2024-03-28T16:52:29.588 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f2925ffb640 1 -- 172.21.15.119:0/84066390 <== mon.0 v2:172.21.15.119:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f292c012400 con 0x7f2930097f70 2024-03-28T16:52:29.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2914035a10 msgr2=0x7f2914037ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2914035a10 0x7f2914037ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f29200ad000 tx=0x7f2920008040 comp rx=0 tx=0).stop 2024-03-28T16:52:29.590 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930097f70 msgr2=0x7f293008fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T16:52:29.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930097f70 0x7f293008fe40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c450 tx=0x7f292c00c910 comp rx=0 tx=0).stop 2024-03-28T16:52:29.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 shutdown_connections 2024-03-28T16:52:29.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.170:6824/317743614,v1:172.21.15.170:6825/317743614] conn(0x7f2914035a10 0x7f2914037ed0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:29.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 --2- 172.21.15.119:0/84066390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2930097f70 0x7f293008fe40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T16:52:29.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 >> 172.21.15.119:0/84066390 conn(0x7f293009e690 msgr2=0x7f2930132ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T16:52:29.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 shutdown_connections 2024-03-28T16:52:29.591 INFO:teuthology.orchestra.run.smithi119.stderr:2024-03-28T16:52:29.587+0000 7f29376e4640 1 -- 172.21.15.119:0/84066390 wait complete. 2024-03-28T16:52:29.607 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-28T16:52:29.608 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-28T16:52:29.608 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-28T16:52:29.608 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T16:52:29.695 INFO:tasks.ceph.osd.0:Stopped 2024-03-28T16:52:29.696 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-28T16:52:29.696 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T16:52:29.782 INFO:tasks.ceph.osd.1:Stopped 2024-03-28T16:52:29.782 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-28T16:52:29.783 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T16:52:29.859 INFO:tasks.ceph.osd.2:Stopped 2024-03-28T16:52:29.859 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-28T16:52:29.859 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T16:52:29.957 INFO:tasks.ceph.osd.3:Stopped 2024-03-28T16:52:29.957 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-28T16:52:29.957 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-28T16:52:29.958 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T16:52:30.007 INFO:tasks.ceph.mgr.x:Stopped 2024-03-28T16:52:30.007 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-28T16:52:30.008 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-28T16:52:30.008 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T16:52:30.028 INFO:tasks.ceph.mon.a:Stopped 2024-03-28T16:52:30.029 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-28T16:52:30.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-03-28T16:52:30.086 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T16:52:30.086 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-28T16:52:30.324 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:52:30.324 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-28T16:52:30.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:52:30.531 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-28T16:52:30.714 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:52:30.714 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-28T16:52:30.873 INFO:tasks.ceph:Archiving mon data... 2024-03-28T16:52:30.874 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/data/mon.a.tgz 2024-03-28T16:52:30.874 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2024-03-28T16:52:30.880 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.uT5y43Izki 2024-03-28T16:52:30.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uT5y43Izki 2024-03-28T16:52:30.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.uT5y43Izki 2024-03-28T16:52:31.053 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.uT5y43Izki is 240KB 2024-03-28T16:52:31.113 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.uT5y43Izki 2024-03-28T16:52:31.119 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-28T16:52:31.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-28T16:52:31.165 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-28T16:52:31.224 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-28T16:52:31.230 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-28T16:52:31.230 INFO:tasks.ceph:Archiving crash dumps... 2024-03-28T16:52:31.232 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/remote/smithi119/crash 2024-03-28T16:52:31.233 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-28T16:52:31.281 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/remote/smithi170/crash 2024-03-28T16:52:31.282 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-28T16:52:31.295 INFO:tasks.ceph:Compressing logs... 2024-03-28T16:52:31.296 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T16:52:31.329 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T16:52:31.347 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24382.log 2024-03-28T16:52:31.347 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23702.log 2024-03-28T16:52:31.348 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24458.log 2024-03-28T16:52:31.348 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24122.log 2024-03-28T16:52:31.348 INFO:teuthology.orchestra.run.smithi119.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.24382.log.gz 2024-03-28T16:52:31.349 INFO:teuthology.orchestra.run.smithi119.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.23702.log.gz 2024-03-28T16:52:31.350 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24458.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.24458.log.gz 2024-03-28T16:52:31.350 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23992.log 2024-03-28T16:52:31.350 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21486.log 2024-03-28T16:52:31.350 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24122.log: /var/log/ceph/ceph-client.admin.23992.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-28T16:52:31.351 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.21486.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.23992.log.gz 2024-03-28T16:52:31.351 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.21486.log.gz 2024-03-28T16:52:31.351 INFO:teuthology.orchestra.run.smithi119.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.24122.log.gz 2024-03-28T16:52:31.351 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23900.log 2024-03-28T16:52:31.354 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24096.log 2024-03-28T16:52:31.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.23900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23598.log 2024-03-28T16:52:31.354 INFO:teuthology.orchestra.run.smithi119.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.23900.log.gz 2024-03-28T16:52:31.354 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23562.log 2024-03-28T16:52:31.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24096.log: /var/log/ceph/ceph-client.admin.23598.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.24096.log.gz 2024-03-28T16:52:31.354 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20232.log 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.23598.log.gz 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23952.log 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.23562.log.gz 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24252.log 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.tmp-client.admin.20232.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20232.log.gz 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24408.log 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.23952.log.gz 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23696.log 2024-03-28T16:52:31.355 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.24252.log.gz 2024-03-28T16:52:31.356 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23808.log 2024-03-28T16:52:31.356 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24408.log: /var/log/ceph/ceph-client.admin.23696.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.24408.log.gz 2024-03-28T16:52:31.356 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-28T16:52:31.359 INFO:teuthology.orchestra.run.smithi119.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.23696.log.gz 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23458.log 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.23808.log.gz 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23510.log 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.23458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24174.log 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.23458.log.gz 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24356.log 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24174.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.23510.log.gz 2024-03-28T16:52:31.360 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21556.log 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.24174.log.gz 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23595.log 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.24356.log.gz 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.21556.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.21556.log.gz 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24018.log 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23807.log 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24018.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.23595.log.gz 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- 86.0% /var/log/ceph/ceph-client.admin.24434.log 2024-03-28T16:52:31.361 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.24018.log.gz 2024-03-28T16:52:31.362 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-28T16:52:31.362 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23807.log: /var/log/ceph/ceph-client.admin.24434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23536.log 2024-03-28T16:52:31.362 INFO:teuthology.orchestra.run.smithi119.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.23807.log.gz 2024-03-28T16:52:31.362 INFO:teuthology.orchestra.run.smithi119.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.24434.log.gz 2024-03-28T16:52:31.362 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23484.log 2024-03-28T16:52:31.362 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24278.log 2024-03-28T16:52:31.374 INFO:teuthology.orchestra.run.smithi119.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.23536.log.gz 2024-03-28T16:52:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24330.log 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24278.log: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.23484.log.gz 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23926.log 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.24278.log.gz 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23600.log 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24304.log 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.24330.log.gz 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.23926.log.gz 2024-03-28T16:52:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23600.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr:gzip 87.1% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.23600.log.gz 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.24070.log 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24304.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23433.log 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.24304.log.gz 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24070.log: 88.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23693.log 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.24070.log.gz 2024-03-28T16:52:31.376 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23704.log 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23433.log: /var/log/ceph/ceph-client.admin.23693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23599.log 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.23433.log.gz 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.23693.log.gz 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24148.log 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23599.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.23704.log.gz 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21599.log 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.23599.log.gz 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24044.log 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.21599.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.24148.log.gz 2024-03-28T16:52:31.377 INFO:teuthology.orchestra.run.smithi119.stderr: 85.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.21599.log.gz 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.23800.log 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.24044.log: /var/log/ceph/ceph-client.admin.24226.log 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24200.log 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.24044.log.gz 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.23800.log.gz/var/log/ceph/ceph-client.admin.24226.log: 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23796.log 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.24226.log.gz 2024-03-28T16:52:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.24200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23395.log 2024-03-28T16:52:31.379 INFO:teuthology.orchestra.run.smithi119.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.24200.log.gz 2024-03-28T16:52:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23796.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.23796.log.gz 2024-03-28T16:52:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.23395.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.23395.log.gz 2024-03-28T16:52:31.391 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27213.log 2024-03-28T16:52:31.391 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32044.log 2024-03-28T16:52:31.391 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.27213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23753.log 2024-03-28T16:52:31.391 INFO:teuthology.orchestra.run.smithi170.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.27213.log.gz 2024-03-28T16:52:31.391 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.32044.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46794.log 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23753.log: /var/log/ceph/rgw.ceph.client.0.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.32044.log.gz 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65676.log 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.23753.log.gz 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph-client.0.46794.log: -5 --verbose -- /var/log/ceph/ceph-client.0.77452.log 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.65676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78036.log 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.77452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72166.log 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.78036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73936.log 2024-03-28T16:52:31.392 INFO:teuthology.orchestra.run.smithi170.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.65676.log.gz 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.77452.log.gz 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68036.log 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.72166.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.46794.log.gz 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.73936.log: 90.6% -- replaced with /var/log/ceph/ceph-client.0.78036.log.gz 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30280.log 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.68036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70396.log 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.30280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52104.log 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.70396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53874.log 2024-03-28T16:52:31.393 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.52104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37354.log 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.0.73936.log.gz 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.68036.log.gz 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.53874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75700.log 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.0.72166.log.gz 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.37354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23801.log 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.0.30280.log.gz 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.70396.log.gz 2024-03-28T16:52:31.394 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.75700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36174.log 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27237.log 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% 85.3% -- replaced with /var/log/ceph/ceph-client.0.52104.log.gz 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.53874.log.gz 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23801.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.43844.log 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.36174.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.37354.log.gz 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.75700.log.gz 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.27237.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-28T16:52:31.395 INFO:teuthology.orchestra.run.smithi170.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.27237.log.gz 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.43844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58594.log 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27920.log 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.36174.log.gz 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.58594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51514.log 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.43844.log.gz 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49154.log 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.27920.log: /var/log/ceph/ceph-client.0.51514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.28510.log 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.49154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56824.log 2024-03-28T16:52:31.396 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.28510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73346.log 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.58594.log.gz 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3%/var/log/ceph/ceph-client.0.56824.log: -- replaced with /var/log/ceph/ceph-client.0.51514.log.gz 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.27920.log.gz 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23825.log 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.49154.log.gz 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.73346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79794.log 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.28510.log.gz 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31454.log 2024-03-28T16:52:31.397 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.79794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62724.log 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.56824.log.gz 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.23825.log.gz 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.31454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54464.log 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.62724.log: gzip 91.5% -- replaced with /var/log/ceph/ceph-client.0.73346.log.gz -5 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr: --verbose -- /var/log/ceph/ceph-client.0.47974.log 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.54464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36764.log 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.79794.log.gz/var/log/ceph/ceph-client.0.47974.log: 2024-03-28T16:52:31.398 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72756.log 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.31454.log.gz 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.36764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75116.log 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.54464.log.gz 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.72756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32634.log 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8%/var/log/ceph/ceph-client.0.75116.log: -- replaced with /var/log/ceph/ceph-client.0.62724.log.gz 91.7% -- replaced with /var/log/ceph/ceph-client.0.47974.log.gz 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80387.log 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-28T16:52:31.399 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3%/var/log/ceph/ceph-client.0.32634.log: -- replaced with /var/log/ceph/ceph-client.0.36764.log.gz 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.0.75116.log.gz 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.80387.log: gzip 91.5% -- replaced with /var/log/ceph/ceph-client.0.72756.log.gz -5 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr: --verbose -- /var/log/ceph/ceph-client.0.74526.log 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56234.log 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.74526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30870.log 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.56234.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.32634.log.gz 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39124.log 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57414.log 2024-03-28T16:52:31.400 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.30870.log: /var/log/ceph/ceph-client.0.39124.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.74526.log.gz 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.0.30870.log.gz 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55644.log 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.80387.log.gz 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.57414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23777.log 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.56234.log.gz 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4%/var/log/ceph/ceph-client.0.55644.log: -- replaced with /var/log/ceph/ceph-client.0.39124.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.62134.log 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-28T16:52:31.401 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.23777.log: /var/log/ceph/ceph-client.0.62134.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.23777.log.gz 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42074.log 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.0.57414.log.gz 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29100.log 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.0.42074.log: 91.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52694.log 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.0.55644.log.gz 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.29100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78620.log 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.62134.log.gz 2024-03-28T16:52:31.402 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.52694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59774.log 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.42074.log.gz 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.78620.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.29100.log.gz 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.78620.log.gz 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.52694.log.gz 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38534.log 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40894.log 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.59774.log: /var/log/ceph/ceph-client.0.38534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69806.log 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-28T16:52:31.403 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.40894.log: /var/log/ceph/ceph-client.0.69806.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.38534.log.gz 2024-03-28T16:52:31.415 INFO:teuthology.orchestra.run.smithi170.stderr: 92.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60364.log 2024-03-28T16:52:31.416 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-03-28T16:52:31.416 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.40894.log.gz 2024-03-28T16:52:31.416 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27189.log 2024-03-28T16:52:31.416 INFO:teuthology.orchestra.run.smithi170.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.69806.log.gz 2024-03-28T16:52:31.416 INFO:teuthology.orchestra.run.smithi170.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.59774.log.gz 2024-03-28T16:52:31.416 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.60364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49744.log 2024-03-28T16:52:31.416 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.27189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34404.log 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.27189.log.gz/var/log/ceph/ceph-client.0.49744.log: 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33224.log 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46204.log 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.34404.log: /var/log/ceph/ceph-client.0.33224.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.49744.log.gz 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.60364.log.gz 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.33224.log.gz 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.34404.log.gz 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61544.log 2024-03-28T16:52:31.417 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.46204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66856.log 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.61544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55054.log 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.66856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59184.log 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.55054.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.46204.log.gz 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69216.log 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.61544.log.gz 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.59184.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.66856.log.gz 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.69216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53284.log 2024-03-28T16:52:31.418 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.55054.log.gz 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ops-log-ceph-client.0.log: 90.7% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.59184.log.gz 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71576.log 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.69216.log.gz 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66266.log 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.53284.log: /var/log/ceph/ceph-client.0.71576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67446.log 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.66266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60954.log 2024-03-28T16:52:31.419 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.67446.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.53284.log.gz 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.66266.log.gz 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.71576.log.gz 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.67446.log.gz 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64494.log 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.60954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48564.log 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.64494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63904.log 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.48564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43254.log 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.63904.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.60954.log.gz 2024-03-28T16:52:31.420 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.64494.log.gz 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.63904.log.gz 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.48564.log.gz 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70986.log 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.43254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63314.log 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.70986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68626.log 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.63314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44434.log 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.68626.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.43254.log.gz 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50924.log 2024-03-28T16:52:31.421 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.70986.log.gz 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.44434.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.63314.log.gz 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.68626.log.gz 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33814.log 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.50924.log: 91.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29690.log 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.0.44434.log.gz 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.33814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40304.log 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.29690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47384.log 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.40304.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.50924.log.gz 91.6% -- replaced with /var/log/ceph/ceph-client.0.33814.log.gz 2024-03-28T16:52:31.422 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.40304.log.gz 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.29690.log.gz 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42664.log 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.47384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45024.log 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.42664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41484.log 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.45024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37944.log 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.41484.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.47384.log.gz 2024-03-28T16:52:31.423 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.42664.log.gz 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.45024.log.gz 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.41484.log.gz 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65086.log 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.37944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39714.log 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.65086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79204.log 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.39714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45614.log 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.79204.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.37944.log.gz 2024-03-28T16:52:31.429 INFO:teuthology.orchestra.run.smithi170.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.65086.log.gz 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35584.log 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.39714.log.gz 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.45614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50334.log 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.35584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58004.log 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.79204.log.gz 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.50334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76284.log 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.58004.log: 92.2% -- replaced with /var/log/ceph/ceph-client.0.45614.log.gz 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.35584.log.gz 2024-03-28T16:52:31.430 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.50334.log.gz 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.58004.log.gz 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27261.log 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.76284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34994.log 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.27261.log: gzip 90.6% -- replaced with /var/log/ceph/ceph-client.0.76284.log.gz -5 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr: --verbose -- /var/log/ceph/ceph-client.0.76868.log 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.27261.log.gz 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.34994.log: /var/log/ceph/ceph-client.0.76868.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.76868.log.gz 2024-03-28T16:52:31.431 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.34994.log.gz 2024-03-28T16:52:31.435 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-28T16:52:31.512 INFO:teuthology.orchestra.run.smithi119.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-28T16:52:32.070 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-28T16:52:32.100 INFO:teuthology.orchestra.run.smithi119.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-28T16:52:32.101 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-03-28T16:52:32.101 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m0.771s 2024-03-28T16:52:32.102 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m0.916s 2024-03-28T16:52:32.102 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m0.075s 2024-03-28T16:52:32.146 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-28T16:52:32.147 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-28T16:52:32.148 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-03-28T16:52:32.148 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m0.806s 2024-03-28T16:52:32.148 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m2.408s 2024-03-28T16:52:32.148 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m0.156s 2024-03-28T16:52:32.149 INFO:tasks.ceph:Archiving logs... 2024-03-28T16:52:32.150 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/remote/smithi119/log 2024-03-28T16:52:32.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T16:52:32.312 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/remote/smithi170/log 2024-03-28T16:52:32.312 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T16:52:32.687 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-28T16:52:32.699 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-28T16:52:32.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T16:52:32.701 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T16:52:32.765 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-28T16:52:32.775 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-28T16:52:32.775 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T16:52:32.778 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T16:52:32.789 INFO:teuthology.orchestra.run.smithi119.stdout: remote refid st t when poll reach delay offset jitter 2024-03-28T16:52:32.790 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================== 2024-03-28T16:52:32.790 INFO:teuthology.orchestra.run.smithi119.stdout:*hv01.front.sepi 44.4.53.2 2 u 58 64 77 0.131 -1.065 0.838 2024-03-28T16:52:32.790 INFO:teuthology.orchestra.run.smithi119.stdout:+hv02.front.sepi 63.231.80.2 3 u 62 64 77 0.131 -1.219 0.902 2024-03-28T16:52:32.790 INFO:teuthology.orchestra.run.smithi119.stdout:+hv03.front.sepi 74.6.168.72 3 u 53 64 77 0.108 -0.242 0.843 2024-03-28T16:52:32.790 INFO:teuthology.orchestra.run.smithi119.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:52:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: remote refid st t when poll reach delay offset jitter 2024-03-28T16:52:32.792 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================== 2024-03-28T16:52:32.792 INFO:teuthology.orchestra.run.smithi170.stdout:*hv01.front.sepi 44.4.53.2 2 u 52 64 77 0.143 -0.590 0.986 2024-03-28T16:52:32.792 INFO:teuthology.orchestra.run.smithi170.stdout:+hv02.front.sepi 63.231.80.2 3 u 54 64 77 0.153 -0.758 1.058 2024-03-28T16:52:32.792 INFO:teuthology.orchestra.run.smithi170.stdout:+hv03.front.sepi 74.6.168.72 3 u 49 64 77 0.131 +0.170 1.034 2024-03-28T16:52:32.792 INFO:teuthology.orchestra.run.smithi170.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-28T16:52:32.793 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-28T16:52:32.801 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-28T16:52:32.802 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-28T16:52:32.810 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-28T16:52:32.818 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-28T16:52:32.826 INFO:teuthology.task.internal:Duration was 814.436984 seconds 2024-03-28T16:52:32.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-28T16:52:32.834 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-28T16:52:32.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T16:52:32.837 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T16:52:32.874 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-28T16:52:32.874 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-03-28T16:52:32.875 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T16:52:32.924 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-03-28T16:52:32.924 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T16:52:32.934 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-28T16:52:32.934 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T16:52:32.969 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T16:52:33.002 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-28T16:52:33.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T16:52:33.041 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T16:52:33.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-28T16:52:33.103 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-28T16:52:33.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T16:52:33.115 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T16:52:33.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-28T16:52:33.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T16:52:33.179 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T16:52:33.187 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-03-28T16:52:33.189 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-03-28T16:52:33.308 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T16:52:33.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:52:33.313 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T16:52:33.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T16:52:33.355 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-28T16:52:33.365 INFO:teuthology.task.internal:Transferring archived files... 2024-03-28T16:52:33.366 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/remote/smithi119 2024-03-28T16:52:33.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T16:52:33.388 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381/remote/smithi170 2024-03-28T16:52:33.388 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T16:52:33.419 INFO:teuthology.task.internal:Removing archive directory... 2024-03-28T16:52:33.419 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T16:52:33.429 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T16:52:33.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-28T16:52:33.475 INFO:teuthology.task.internal:Not uploading archives. 2024-03-28T16:52:33.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-28T16:52:33.484 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-28T16:52:33.484 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T16:52:33.486 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T16:52:33.489 INFO:teuthology.orchestra.run.smithi119.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 16:52 /home/ubuntu/cephtest 2024-03-28T16:52:33.511 INFO:teuthology.orchestra.run.smithi170.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 16:52 /home/ubuntu/cephtest 2024-03-28T16:52:33.513 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-28T16:52:33.523 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-28T16:52:33.543 INFO:teuthology.nuke:Checking targets against current locks 2024-03-28T16:52:33.568 DEBUG:teuthology.nuke:shortname: smithi119 2024-03-28T16:52:33.568 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T16:52:33.589 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 16:30:06.931945', 'locked_by': 'scheduled_soumyakoduri@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQ0qQqye1uKDXScKCoAVqSlWVI8KsfzUNX7YTdRQwYQ0cIEIhdtKHEz0DpNLvdjS7oZqx3yLvXv8aw5fbhdqxQ='} 2024-03-28T16:52:33.592 DEBUG:teuthology.nuke:shortname: smithi170 2024-03-28T16:52:33.593 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T16:52:33.608 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/soumyakoduri-2024-03-28_16:27:01-rgw:singleton-wip_thotz_update_attrs_from_state_setattrs-distro-default-smithi/7628381', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 16:30:06.933040', 'locked_by': 'scheduled_soumyakoduri@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpRTtw3n0zBm75s+ZMdSyBbyD9vw++ZFvRFb9QaQy+3ImgeT9ikr+Xk+DubNhB4IL2pmsjpXnrF6tn4Bnu/bEc='} 2024-03-28T16:52:33.638 INFO:teuthology.orchestra.console.smithi119:Power off 2024-03-28T16:52:33.639 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T16:52:33.665 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2024-03-28T16:52:33.665 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T16:52:33.685 INFO:teuthology.orchestra.console.smithi170:Power off 2024-03-28T16:52:33.686 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T16:52:33.698 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-03-28T16:52:33.711 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2024-03-28T16:52:33.711 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T16:52:33.736 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-28T16:52:37.699 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T16:52:37.737 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T16:52:37.813 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2024-03-28T16:52:37.849 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-28T16:52:41.817 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T16:52:41.850 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T16:52:41.929 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2024-03-28T16:52:41.960 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-28T16:52:42.031 INFO:teuthology.orchestra.console.smithi119:Power off completed 2024-03-28T16:52:45.965 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T16:52:46.078 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-03-28T16:52:46.179 INFO:teuthology.orchestra.console.smithi170:Power off completed 2024-03-28T16:52:46.283 INFO:teuthology.run:Summary data: description: rgw:singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} duration: 814.4369835853577 failure_reason: '''Key'' object has no attribute ''set_storage_class''' flavor: default owner: scheduled_soumyakoduri@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=69cdb6a1ff204c52bf229833670ca014 status: fail success: false 2024-03-28T16:52:46.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T16:52:46.370 INFO:teuthology.run:FAIL